To note that PS1 was the first games console on the home market[0] to fully embrace C on its SDK, until then it was Assembly all the way, hence why so many studios were happy to jump into it.
[0] - The arcades adopted it much earlier, and there were UNIX and VAX/VMS devkits with cross-compiling capabilities for them.
I'm reverse-engineering a PlayStation video game that is rather unoptimized and I'm amazed that the game managed to do so much with such horrible game code. That SDK really was fairly forgiving for its time to new, inexperienced developers.
Yet to squeeze out performance out of the in-order, single-issue 33 MHz MIPS CPU you had to be very mindful of low-level details. The 4 KiB instruction cache hates code bloat, the lack of data cache means spilling registers anywhere but onto the 1 KiB of scratchpad memory is costly, yet the god-awful C compilers of the era liked to do both of these things. That's just for the CPU, the rest of the system also has plenty of details you need to pay attention to for performance.
I haven't actually programmed for the console, but the achievements done by talented developers such as Naughty Dogs on this dirt-cheap system are truly impressive. The console might be easy to develop for, but to make it really fly still takes a lot of skill.
Yes, for Crash Bandicoot we had to entirely skip the C SDK for everything performance-critical; pushing arguments onto the stack and the SDK popping them off for each function call used more cycles than the underlying operation in many of these SDK calls.
Sony explicitly forbade this, presumably because they envisioned the API established by the C SDK as a way to ensure future backward-compatibility. We just ignored the rules and hoped the superior performance we could achieve would overcome any bureaucratic obstacles.
We also had to reverse engineer some of the PS1’s really nice capabilities. I only learned the hardware supported vector pipelining for strip rendering by noticing the coordinate values move from one register set to the next in the debugger.
Seeing that was a revelation: when rendering a polygonal mesh, you naively have to project three coordinates for each triangle. But if you convert your mesh into sequences of polygonal strips where each triangle shares an edge with the next triangle in the mesh, you only need to project a single additional vertex for each additional polygon in the strip. Observing the behavior in the debugger, it was obvious to me that the Sony hardware had in fact been optimized for this approach. But not only were we not given any documentation on this, we were instead told to use the C SDK, which didn’t expose this capability at all.
The PS1 also had 2KB of “scratchpad” that was slower than registers but much faster than RAM. Hearsay was this was a way for the CPU designers to make use of the physical space on the die meant for floating point instructions which the MIPS CPU in the PS1 didn’t have.
I used the scratchpad to store projected 2D “stamps” of the 3D world (stored in an octree) and Crash: a kind of low-res Crash Flatland. I could then check for X/Y collisions between Crash and the world very rapidly by inspecting the flatland bitmap stored in the 2K scratchpad. This was Mark Cerny’s idea; he was our producer on the Crash games and has also been responsible for PS2 through PS5 hardware.
Hey I just wanted to chime in and say that I spent a lot of time in my infancy/youth playing crash bandicoot, thank you very much.
Also, i’ve seen many videos on YouTube about all the crazy things you had to do to make stuff work. And every time i get to read some additional detail it’s always super intriguing…
Have you considered doing something like a documentary? Like, a series of long form videos on the topic? With interviews to people involved etc?
Hopefully you’ve read the Polygon article and seen the videos with Andy Gavin talking about the underlying tech. I was going to do part two of the video series but COVID killed it. Maybe we’ll revive that; it would be fun.
Not only do you only have 4KB of instruction cache, but it's direct mapped.
So if the linker just so happens to place the function you are calling at some address that's some multiple of 4KB away from your current hot loop, then the code for your hot loop, and the function you call will get continually evicted and reloaded for each loop iteration.
A small change to a random function could have large impacts on the alignment of all other functions in your game, and could result in large performance impact. The simple solution is to just inline everything your hot loops call. That avoids the chance of the pathological cases (and has the bonus of avoiding function call overhead), but at the cost of increasing code bloat in general (which the cache hates).
I've been vaguely considering making linker that's aware of direct mapped caches, and uses the call graph to guide the placement of functions (and data too, for machines with direct mapped data caches). Not only would it be useful for retro consoles with direct mapped caches like the ps1 and n64 (and probably help for the PS2 too, which is only 2-way set associative) but it would be useful for modern microcontrollers that do XIP execution of code from flash. They often have simplistic direct mapped caches too.
Have you published anything? There are a few games that I would love to be able to round trip between the original disc contents and a build with a few tweaks or experimental features. I know there is a Wipeout decompile/source leak, but having more examples is always wonderful.
Something like the Mario 64 reverse engineering project has made the game playable effectively everywhere with the ability to take advantage of modern hardware (16:9 displays). It would be great to see efforts like that for any console.
I'm documenting my progress on my personal blog (https://boricj.net). Currently I'm in the process of porting data from a leftover SYM file onto a closely matching EXE, the next article should be out by the end of the week-end.
Be warned, I'm going about this with a very different approach than most decompilation projects out there. I want to delink these executables back into working, reusable object files. The plan is to make a native Linux MIPS port on the cheap using PsyCross, then to rewrite each piece Ship-of-Theseus style until I have a complete, portable set of source code for the game.
It's Tenchu: Stealth Assassins. More specifically, the updated Japanese re-release version, which has goodies like a level editor we didn't have in the West until Tenchu 2: Birth of the Stealth Assassins.
I do not expect to finish it anytime soon, although finding debugging symbols in a leftover file inside a proprietary archive format has been a very lucky find. Most of the past two years have actually been spent on the tooling to delink executables back into object files, which should prove extremely useful if this crazy approach of mine works out.
[0] - The arcades adopted it much earlier, and there were UNIX and VAX/VMS devkits with cross-compiling capabilities for them.