[ Beneath the Waves ]

This Dust Remembers What It Once Was

article and software by Ben Lincoln

 

Table of contents

  1. Introduction
  2. Components
  3. Walkthroughs
  4. Results
  5. Known Issues
  6. Update History
  7. Downloads

Introduction

This Dust Remembers What It Once Was ("TDR") is a reverse-engineering toolkit I wrote for use with the NSA'a amazing tool Ghidra. Ghidra is a completely free, open-source binary reverse-engineering toolkit that includes not only a disassembler, but a decompiler that must have been written using black magic. I can't thank its authors and the NSA enough for releasing it last year.

I wanted to use Ghidra to help reverse engineer Soul Reaver, my favourite game of all time, but at least when I started, there were a couple of obstacles in my way: Ghidra doesn't support the proprietary PSX-EXE format used for PlayStation binaries, and it also doesn't support the PsyQ .SYM debug symbol format.

I originally started writing TDR specifically for that one project, but I've tried to generalize it enough to work with any PlayStation title that has PsyQ debug symbols available. The PSX-EXE-to-ELF converter means that any PlayStation binary should be importable into Ghidra, even if it wasn't written using PsyQ at all. It also means that any PlayStation binary can be imported into other tools that support ELF binaries, such as RetDec.

I have some additional componnents in mind for later that will extend it to other gaming platforms, but I'm not sure when I'll have time to get around to that.

Be warned, the current version of TDR should be considered an alpha release, in the traditional sense: it's feature-complete, but it's probably full of bugs. I don't know how frequently I'll be able to work on it, so I wanted to get it out there in case it was useful to someone even in its current state.

TDR is a highly-specialized reverse-engineering tool. The documentation below is pretty barebones at the moment, and assumes extensive pre-existing knowledge. I'd like to expand it in the future.

TDR itself is open-source, licensed under the GPLv3. Warning: you may regret looking at some of the code. This is a project that grew organically over about eight months. It involved lots of on-the-fly design changes because I was learning about some of the low-level details as I went.

If you just want to be able to load PlayStation games into Ghidra and don't have .SYM files, you can also take a look at DrMefistO's outstanding PSX Loader for Ghidra. A future version of this content will include an alternative walkthrough for using that loader in place of the EXE-to-ELF step, for those who wish to compare.

Components

The current version of TDR is made up of four tools (in addition to Ghidra itself, which you'll need to install separately):

CreateSkeleton.exe does the bulk of the work in the current version of TDR. From the input data, it generates the following:

Walkthroughs

I've written four walkthroughs to help people jump into the basics of using TDR. Following these walkthroughs will get you a long way, much faster than manually reverse-engineering these games from scratch, but you'll still be doing a lot of manual work in Ghidra if you want to generate code that will compile back to a working game binary. If all you're looking for is mostly-accurate decompiled code to use as a reference for e.g. reverse-engineering file formats, you might not have to do any additional work, though.

Important:

When a new version is released, be sure to delete all of the generated files, start over from scratch, and re-read the walkthrough. This is still alpha-quality software, and the "right" way to use it is changing essentially with every release. Mixing files from different versions will result in poor output or crashes.

  1. TDR: Practice Using EdgeCase- a basic walkthrough of decompiling a very simple PlayStation PSX-EXE binary which has source code included so you can compare the decompiled code with the corresponding original source.
  2. TDR: Soul Reaver - steps that should work with most debug builds of Soul Reaver, but specifically tested with the 1999-06-01 prototype.
  3. TDR: Need For Speed 4 - tested with the 1999-02-22 (v9.0) version of the game.
  4. TDR: Biohazard 2 - tested against the 1997-10-30 build of Biohazard 2.

Results

This section will be greatly expanded in the future.

TDR works really well with all of the debug builds of Soul Reaver I've tested it against.

It does a solid job against the 1997-10-30 beta build of Biohazard 2.

As of version 0.6, it does a pretty phenomenal job with the 1999-02-22 prototype of Need for Speed 4, especially considering the extremely complex codebase for that game.

It does not do so well with the 1996-08-05 prototype version of Wipeout XL, because that build was created without the -g flag for CCPSX.EXE, which means the .SYM file essentially only includes labels, not other types of symbols. It's better than nothing, but a significant additional amount of manual work would be required decompile it to working code.

Known Issues

Update History

In reverse chronological order:

Version 0.7 - released 2019-08-22

Version 0.6 - released 2019-08-19

Version 0.5 - released 2019-08-14

Version 0.4 - released 2019-08-14

Version 0.3 - released 2019-08-14

Downloads

 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was 578 KiB 0.7 2019-08-22 Ben Lincoln
This is the .NET executable version of the TDR suite. If you want to use the tool, this is probably the file you want to download.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was (Source Code) 1 MiB 0.7 2019-08-22 Ben Lincoln
This is the .NET source code for the TDR suite.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was 575 KiB 0.6 2019-08-19 Ben Lincoln
This is the .NET executable version of the TDR suite. If you want to use the tool, this is probably the file you want to download.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was (Source Code) 1 MiB 0.6 2019-08-19 Ben Lincoln
This is the .NET source code for the TDR suite.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was 571 KiB 0.5 2019-08-14 Ben Lincoln
This is the .NET executable version of the TDR suite. If you want to use the tool, this is probably the file you want to download.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was (Source Code) 1 MiB 0.5 2019-08-14 Ben Lincoln
This is the .NET source code for the TDR suite.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was 571 KiB 0.4 2019-08-14 Ben Lincoln
This is the .NET executable version of the TDR suite. If you want to use the tool, this is probably the file you want to download.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was (Source Code) 1 MiB 0.4 2019-08-14 Ben Lincoln
This is the .NET source code for the TDR suite.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was 568 KiB 0.3 2019-08-13 Ben Lincoln
This is the .NET executable version of the TDR suite. If you want to use the tool, this is probably the file you want to download.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was (Source Code) 1 MiB 0.3 2019-08-13 Ben Lincoln
This is the .NET source code for the TDR suite.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was 558 KiB 0.2 2019-08-06 Ben Lincoln
This is the .NET executable version of the TDR suite. If you want to use the tool, this is probably the file you want to download.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was (Source Code) 1 MiB 0.2 2019-08-06 Ben Lincoln
This is the .NET source code for the TDR suite.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was (Documentation) 118 KiB 0.2 2019-08-13 Ben Lincoln
The documentation for version 0.2 of the TDR suite. Available for anyone who wants to use the old version for some reason.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was 557 KiB 0.1 2019-08-06 Ben Lincoln
This is the .NET executable version of the TDR suite. If you want to use the tool, this is probably the file you want to download.
 
Download
File Size Version Release Date Author
This Dust Remembers What It Once Was (Source Code) 1 MiB 0.1 2019-08-06 Ben Lincoln
This is the .NET source code for the TDR suite.
 
[ Page Icon ]