Untamed Heart: Enough dream stones to run a sleep clinic!

Untamed Heart Klonoa Collective Untamed Heart Klonoa Collective Untamed Heart Klonoa Collective

Author Topic: Reverse engineering  (Read 8469 times)

0 Members and 1 Guest are viewing this topic.

Offline Voka~Daemyn

  • Cabbits for the Brotherhood of Nod!
  • Global Moderator
  • *
  • Posts: 1717
  • Dreamer Rating: 65
  • Praise the Klonoa/s
Re: Reverse engineering
« Reply #10 on: March 25, 2017, 03:31:16 AM »
I have some more addresses and it all just gets more confusing for me though I am inexperienced.  :sick:
Also a save file which I hope is the correct one. It should have the vision select availible.


Code: [Select]
0BFCCC   d   u   0   MainRAM   Ghadius's head Y axis in vision 1-2 cutscene
0BFCD0   d   u   0   MainRAM   Ghadius's head x-axis in vision 1-2 cutscene
0C44F8   d   s   0   MainRAM   Camera panning in 1-2 cutscene (X axis either way?)
0C44FC   d   s   0   MainRAM   1-2 cutscene cam ( Y axis sort of?)
0C4508   d   s   0   MainRAM   1-2 cutscene z-axis sort of
0C450C   d   s   0   MainRAM   1-2 cutscene y-axis sort of
0C4510   d   s   0   MainRAM   1-2 also sort of Z axis -40000 makes for mega zoom out
0C4518   d   s   0   MainRAM   1-2 another camera . I can't guess the axis.
0C96B0   d   s   0   MainRAM   Affects the bottom third of the background in 1-2. setting to 780 makes it spin pretty quicjkly in cutscene
0C96B8   d   s   0   MainRAM   Also affects the bottom third of the background in 1-2
0C96B4   d   s   0   MainRAM   bottom third of background inrelation to camera maybe
0BF074   d   s   0   MainRAM   set to 1 for invulnerability technically copied from old pec / playstation emulation cheater code. minus some zeros
0BF530   d   s   0   MainRAM   this affects Rongo Lango it can make him freeze or stay in place
0BF068   d   s   0   MainRAM   related to the sliding mechanic in game but not sliding down hills
0BF06C   d   s   0   MainRAM   Also connected to the above sliding mechanic
0BF018   d   s   0   MainRAM   set to zero and freeze for infinite jumps
0C981C   d   u   0   MainRAM   Amount of Phantomillians released 1 "this seems to hop around adresses:c"
0C9820   d   u   0   MainRAM   Amount of Phantomillians released 2 "this seems to hop around adresses:c"
10689C   d   u   0   MainRAM   windbullet. 0=right, 1=left 2=behind 3=forward
10E610   d   u   0   MainRAM   Amount of Phantomillians released 3? "this seems to hop around adresses:c"
0BE328   d   u   0   MainRAM   this is for key and 5-1 orbs. set to 1 for a key 2-5 for orbs
0BE8B4   d   u   0   MainRAM   related to Phantomillians at vision end screen 1 "this seems to hop around adresses:c"
0C96EC   d   u   0   MainRAM   related to Phantomillians at vision end screen 2 "this seems to hop around adresses:c"
0C96F0   d   u   0   MainRAM   related to Phantomillians at vision end screen 3 "this seems to hop around adresses:c"
0BEB60   d   u   0   MainRAM   Vision 5-2 Eclipse 1 you can disable the eclipse by freezing these are zero or have infinite eclipse by freezing at 1. does not affect Joka transformation
0BED0C   d   u   0   MainRAM   Vision 5-2 Eclipse 2
0C51A0   d   u   0   MainRAM   Vision 5-2 Eclipse 3
0BF028   d   u   0   MainRAM   Same as 0B83F4 appears to be the position you are in along the level track? Also does not always go 0-100 sometimes it goes out of order or in a loop
0B83F4   d   u   0   MainRAM   Seems to be the position along the track of curren't vision you are in. set to 29 to get directly under the bell in the middle of the rango longo fight
0BEE14   d   u   0   MainRAM   post game vision select 1
1069EC   d   u   0   MainRAM   post game vision select 2 2048 pops up during vision clear book thing with fireworks
106AB0   d   u   0   MainRAM   post game vision select 3 "other things also use this adress at times @[email protected]
125CAC   d   u   0   MainRAM   post game vision select 4
« Last Edit: March 25, 2017, 03:47:56 AM by Vokadae »

Offline cows

  • Moo
  • *
  • Posts: 24
  • Dreamer Rating: 5
Re: Reverse engineering
« Reply #11 on: March 27, 2017, 02:10:12 AM »
Wow, nice progress. Making me look bad by comparison.

I had some trouble getting the disassembler in BizHawk to actually do anything (breakpoints set in the UI don't seem to actually halt execution or do anything else, and the disassembly window seemingly never gets filled in). It looks like I might have better luck with the onmemoryread and onmemorywrite Lua functions, but I haven't tried that yet since I had a much easier time getting PSX 1.13 to do interactive disassembly. I'm pretty rusty at assembly and I've never worked with the R3000 instruction set specifically, but fortunately wikipedia has most of the needed info.

I think I've isolated a function that actually decreases the life counter when you die, but I'm not certain if it's the only one that handles it or even if it always lives in the same location. This is a useful step toward modifying the game's logic, but there's a lot of things I haven't even begun to work out. For instance, I'm still pretty much clueless about how PlayStation ROMs are stored. Presumably the machine code gets copied into RAM at some point before execution, but I don't know what the process for this is at all. In an ideal (and unlikely) world, all of the code gets loaded at once, meaning I'll be able to find that same code in the ROM, but what I've seen so far indicates that it's probably a bit more complicated than this.

I guess a good thing to try is looking for the same chunk of code I found in the emulator in the ROM itself, but my instinct is that it's extremely naive to assume it'll exist in exactly the same form. Shouldn't take long to test, at least.

Still planning to get the disassembly working in BizHawk, if only because it has more user-friendly tools (except for, apparently, the ones that don't work...) and the Lua scripting engine looks like a really good way to mock up some logic in lieu of actually replacing bits of machine code. Basically you can add an event handler that tells the emulator (rather than the PlayStation) to directly read and set certain values when certain things happen.

Offline Voka~Daemyn

  • Cabbits for the Brotherhood of Nod!
  • Global Moderator
  • *
  • Posts: 1717
  • Dreamer Rating: 65
  • Praise the Klonoa/s
Re: Reverse engineering
« Reply #12 on: March 28, 2017, 04:02:58 PM »
Ah this is where you way a head of me  :sick: I don't know much of anything about using those. I mostly just only know how to mess with what is in memory and I pretty much sit there for a long long time and search for this or that to change when I do something in game and attempt to modify or change,freeze the address that changed except stuff gets weird and 2-4+ address change at once sometimes like the vision 5-2 eclipse which seems to have multiple parts working together.

I think you are probably a lot more knowledgeable than me on this stuff.

If I find any new interesting addresses I'll post them when I get time.

Also I might need to upload a new saveram my save file seems to keep reverting to an old version and I don't know how yet.

EDIT: I found out that if you use a save state it won't save to the saveram unless you flush it. Sorry I uploaded the wrong saveram I didn't know about that. Anyways I made sure this one load the completed save.
« Last Edit: March 30, 2017, 07:53:19 PM by Vokadae, Reason: added new saveram file because I screwed up on the first »

Offline cows

  • Moo
  • *
  • Posts: 24
  • Dreamer Rating: 5
Re: Reverse engineering
« Reply #13 on: April 04, 2017, 03:22:12 AM »
OK, so I still have nothing to show for it, but I've finally had a minor breakthrough in that I've been able to find where (some of?) the code that gets loaded into RAM lives on the disk. I got lucky in this regard; it could have been the case that this code was compressed or encrypted in some way, with the game dynamically translating it into native code while the game is already running. That would have made it substantially more difficult, since I wouldn't have even known what to look for in the ROM. Fortunately, the code seems to exist in pretty much the same form on disk as it does in RAM.

I'm still not sure how much of the code is actually copied into RAM at a given time, but this should be fairly inconsequential as long as I'm careful not to change the way the game loads the code.

I'm currently trying to modify the game to prevent Klonoa from ever losing lives, just because it should be relatively easy. I'm not quite there yet (my version crashes as it tries to start the first level), but I at least know roughly where the code that controls lives is, and what assembly code I need to replace it with to make this work.

In retrospect, finding the code was actually pretty easy. Using the PSX 1.13 emulator from earlier (I still haven't gotten BizHawk to work exactly how I want), all I needed to do was add a breakpoint that fires when the "lives" value is written to, look for the instructions that are executing when the breakpoint is hit, dump the memory that holds the instructions to disk, and then search for that same code in the ROM itself (which should be doable with any hex editors; I used Visual Studio even though it's certainly not the ideal tool for this).

The substantial pain point I ran into was that, even though the code is essentially the same in both places, the bytes weren't ordered in exactly the same way in the ROM as they were in my dumped RAM memory, and I couldn't find any documentation about exactly what the difference is. I ended up needing to use a lot of trial and error, but eventually I managed to figure it out. Now that I've got that figured out, it shouldn't be a problem anymore, which is pretty exciting.

I'm hoping I can find the time some time soon to document this process in a bit more detail, as I think you'll be pleasantly surprised at how not actually difficult it is. Once you've seen it done a single time you'll probably be able to do it yourself without too much trouble. My free time is still unfortunately quite limited, so I don't want to promise that this will happen any time soon.

Actually reading and writing assembly code can definitely be tricky/annoying/slow, particularly since assembly code pulled from a binary doesn't have any of the comments, labels, etc. that developers put in to make the code much easier to follow. There are also a lot of conventions about how (usually) to do certain things and store certain things (arguments that get passed to subroutines, where to return after leaving a subroutine, etc.), and being familiar with them makes the code a bit easier to understand, but the basic instructions, registers, etc. that are the building blocks for all of these conventions aren't too scary. Also note that I'm by no means an especially good assembly programmer; it's not something that often comes up in my day to day work, since I essentially never work closely enough with hardware to need it.

Offline Voka~Daemyn

  • Cabbits for the Brotherhood of Nod!
  • Global Moderator
  • *
  • Posts: 1717
  • Dreamer Rating: 65
  • Praise the Klonoa/s
Re: Reverse engineering
« Reply #14 on: April 05, 2017, 07:44:44 PM »
Ah no need to promise things. Don't worry about that x_x I know how expectations can become hell. Some of this is beyond me or at least what I know but I'll definitely try it if you get that put out. I wish you your best :o / best of luck also.

I've not had as much free time my self lately or I would try to find more addresses though I'm running into more mundane stuff now I think.

Offline cows

  • Moo
  • *
  • Posts: 24
  • Dreamer Rating: 5
Re: Reverse engineering
« Reply #15 on: April 08, 2017, 06:45:01 AM »
I've successfully modified the ROM to give Klonoa infinite lives. Not especially interesting on its own, but it suggests that it should be possible to edit other game logic by the same method as well, hopefully without running into too many weird problems. As I mentioned before, it's fortunate that the game seems to work in a pretty typical and straightforward way. If for some reason you want to download the modified ROM, I've uploaded it at https://mega.nz/#!wCAA2ApS!Av0Dei2-uf9vbyKV7rFwfXBAbXyhgF64J0FduNIOYIk

Also, I've started using yet another emulator, from http://problemkaputt.de/psx.htm . It has the ability to edit assembly code that's loaded into RAM on the fly, and it also has a "trace" functionality. Tracing essentially just starts logging all of the instructions in the order that they're executed over some period of time. It also includes a bit of information about register values as they're read/written. Unfortunately it doesn't log much else, such as function call stacks and values that are read from memory.

The trace function turned out to be extremely useful for the infinite lives mod. The code that directly modifies the lives value is actually inside a function that gets called from a bunch of different locations, meaning that changing the logic within that function also causes other things to break. Having a log of the instructions that
were executed before that function was called made it relatively easy to isolate the lives-specific logic leading up to the function call, and then modify that part instead.

Offline Voka~Daemyn

  • Cabbits for the Brotherhood of Nod!
  • Global Moderator
  • *
  • Posts: 1717
  • Dreamer Rating: 65
  • Praise the Klonoa/s
Re: Reverse engineering
« Reply #16 on: April 08, 2017, 09:50:31 PM »
Hmm gives me yet another excuse to play through the whole game again. I'll try the new emulator. I don't know if I can successfully learn uhh lots of that newer coding stuff on the fly but I'll give it a shot.

I hardly know how any assembly code works :confused: . Last time I ever tried to edit any was back in 2010 I believe by following a tutorial on how mod a dll to cause a skill in an online game skip it's animation  :bad_straight_face: :embarassed: . But heh I'm willing to poke at it. If I fail miserably at the very least I can test stuff for you.

Edit: Would it be ok if I streamed the rom while testing and recording it? So long as my internet is working well enough to stream.
« Last Edit: April 08, 2017, 09:54:03 PM by Vokadae »

Offline cows

  • Moo
  • *
  • Posts: 24
  • Dreamer Rating: 5
Re: Reverse engineering
« Reply #17 on: April 09, 2017, 03:26:03 AM »
Yeah, do whatever you want with it (well, the changes I've made at least, since I obviously don't own the material in the game itself). That said, I can't guarantee that it works properly, so don't rely on it for anything super-serious. I actually made a bit of a mistake, in that it it updates the number of "lives" from some arbitrary location in memory rather than the constant value I'd meant to use. The effect should be the same, as long as the value I'm copying doesn't change. This could be a big problem if that value ever gets set to 0 by some other code.

Anyway, assume everything I'm doing is under the CC0 license, unless I specifically say otherwise. That means you're free to sell it, copy it, modify it, include it in other things, etc.

Another tool I've started looking into is http://www.backerstreet.com/rec/recdload.htm , as it explicitly supports Playstation's executable format (even though it's actually possible to use any MIPS disassembler without too much more work. I haven't quite figured it out, but it definitely is able to find a few functions and even reverse-compile them into C code. It definitely doesn't identify all of the functions in the binary, so I'm thinking that perhaps the code's entry point isn't exactly where the program expects it to be. Then again, I'm not sure how that even makes sense, since I'd assume all executables have the same entry point; otherwise how would the Playstation even no how to start running the game? Or maybe the Playstation reads a header that includes a pointer to the actual entry point.

Being able to separate the code out into functions is extremely useful. The generated C code is probably even less readable than the assembly (even though I have much more experience with C than ASM) but it can be manually be turned into something much more readable than assembly, and it's potentially also easier to add new logic via C, although I haven't gotten to a point where I can actually test that.

Offline Voka~Daemyn

  • Cabbits for the Brotherhood of Nod!
  • Global Moderator
  • *
  • Posts: 1717
  • Dreamer Rating: 65
  • Praise the Klonoa/s
Re: Reverse engineering
« Reply #18 on: April 10, 2017, 08:10:36 AM »
Ah I got to test it finally. The mod works on "was damaged by enemy" deaths but it seems to not work when falling or falling in water. But then again one enemy damage death and you get infinite 88 lives.

I had one glitch I couldn't get to happen again when recording where if I get hit Klonoa would auto hover / ear flap which was actually somewhat handy. But it only happened on one new game.

the mod seems to work as it should on epsxe as well. I  about thought I would have to record on it instead.

My internet didn't cooperate so no stream  :sick: But I recorded a playthrough and extra vision.


Offline cows

  • Moo
  • *
  • Posts: 24
  • Dreamer Rating: 5
Re: Reverse engineering
« Reply #19 on: April 11, 2017, 03:52:55 AM »
Neat! Interesting that the value it's reading from just happens to be 0 during fall deaths.

I have no guess as to where the flying bug would have come from, but I wonder if it's a bug in the original as well. Normally I'd assume it was unrelated to the changes I made simply because it's such a minor change, but I've found Klonoa to be one of the least buggy games I can think of, so it seems unlikely that a bug like that in the original game would just happen to first show up in my version. If you do find a way to reproduce it, that'd be pretty interesting either way. If I had to guess, it seems like the "lives" value I'm writing to gets used for something completely different in some situations, since it frequently holds values that don't match the number of lives.

If that's the case, AND it uses the same function that decrements the number of lives (since that bit of code normally just decreases that value by one), it could also be cause the other values to assume incorrect values. The only reason I'm skeptical of this conjecture is that the Game Shark "infinite lives" code works by forcing that same memory location to remain some fixed number, so whatever else is being stored there would also be invalid when the Game Shark code is in use. That suggests that the non-lives values stored there are either not very important or not meaningful at all.

Anyway, I'm probably not going to spend any more time on this particular behavior, because I'm too eager to start looking at some of the more complex logic, even though I haven't chosen anything in particular yet. Being able to understand how the levels are stored (geometry, walkable path, etc.) and modify them definitely seems to be a good long-term goal, but it's probably beyond the scope of where we are now.

Another great thing would be to understand enemy placement and behavior, but that's also probably a bit tricky (almost certainly still easier than the level geometry though), mainly because the number of active enemies changes a lot, meaning that the game probably chooses where the information about any one particular enemy on the fly. That'd require understanding how this memory is allocated as well as where it is for a particular enemy that's onscreen in a given moment. In the best case, there's a static array of ALL enemy structures laid out in memory, including inactive ones. That'd also mean that the game would store a running count of "active" enemies so that it knows how many to loop through and update each frame. All of the enemy information would be particularly useful to my secret project (which still may or may not ever actually exist), so that's probably what I'll look into next.