Yet another GameCube/Wii decompilation toolkit.
decomp-toolkit functions both as a command-line tool for developers, and as a replacement for various parts of a decompilation project's build system.
For use in a new decompilation project, see dtk-template, which provides a project structure and build system that uses decomp-toolkit under the hood.
- Goals
- Background
- Analyzer features
- Other approaches
- Terminology
- Commands
- ar create
- ar extract
- demangle
- disc info
- disc extract
- disc convert
- disc verify
- dol info
- dol split
- dol diff
- dol apply
- dol config
- dwarf dump
- elf disasm
- elf fixup
- elf2dol
- map
- rel info
- rel merge
- rso info
- shasum
- nlzss decompress
- rarc list
- rarc extract
- u8 list
- u8 extract
- yay0 decompress
- yay0 compress
- yaz0 decompress
- yaz0 compress
- Automate as much as possible, allowing developers to focus on matching code rather than months-long tedious setup.
- Provide highly accurate and performant analysis and tooling.
- Provide everything in a single portable binary. This simplifies project setup: a script can simply fetch the binary from GitHub.
- Replace common usages of msys2 and GNU assembler, eliminating the need to depend on devkitPro.
- Integrate well with other decompilation tooling like objdiff and decomp.me.
The goal of a matching decompilation project is to write C/C++ code that compiles back to the exact same binary as the original game. This often requires using the same compiler as the original game. (For GameCube and Wii, Metrowerks CodeWarrior)
When compiling C/C++ code, the compiler (in our case, mwcceppc) generates an object file (.o) for every source file.
This object file contains the compiled machine code, as well as information that the linker (mwldeppc) uses to
generate the final executable.
One way to verify that our code is a match is by taking any code that has been decompiled, and linking it alongside portions of the original binary that have not been decompiled yet. First, we create relocatable objects from the original binary:
(Heavily simplified)
Then, each object can be replaced by a decompiled version as matching code is written. If the linker still generates a binary that is byte-for-byte identical to the original, then we know that the decompiled code is a match.
decomp-toolkit provides tooling for analyzing and splitting the original binary into relocatable objects, as well as generating the linker script and other files needed to link the decompiled code.
Function boundary analysis
Discovers function boundaries with high accuracy. Uses various heuristics to disambiguate tail calls from
inner-function control flow.
Signature analysis
Utilizes a built-in signature database to identify common Metrowerks and SDK functions and objects.
This also helps decomp-toolkit automatically generate required splits, like __init_cpp_exceptions.
Relocation analysis
Performs control-flow analysis and rebuilds relocations with high accuracy.
With some manual tweaking (mainly in data), this should generate fully-shiftable objects.
Section analysis
Automatically identifies DOL and REL sections based on information from signature and relocation analysis.
Object analysis
Attempts to identify the type and size of data objects by analyzing usage.
Also attempts to identify string literals, wide string literals, and string tables.
Splitting
Generates split object files in memory based on user configuration.
In order to support relinking with mwldeppc.exe, any unsplit .ctors, .dtors, extab and extabindex entries
are analyzed and automatically split along with their associated functions. This ensures that the linker will properly
generate these sections without any additional configuration.
A topological sort is performed to determine the final link order of the split objects.
Object file writing
Writes object files directly, with no assembler required. (Bye devkitPPC!)
If desired, optionally writes GNU assembler-compatible files alongside the object files.
Linker script generation
Generates ldscript.lcf for mwldeppc.exe.
Future work
- Support RSO files
- Add more signatures
Create a static library (.a) from the input objects.
$ dtk ar create out.a input_1.o input_2.o
# or
$ echo input_1.o >> rspfile
$ echo input_2.o >> rspfile
$ dtk ar create out.a @rspfileExtracts the contents of static library (.a) files.
Accepts multiple files, glob patterns (e.g. *.a) and response files (e.g. @rspfile).
Options:
-o,--out <output-dir>: Output directory. Defaults to the current directory.-v,--verbose: Verbose output.-q,--quiet: Suppresses all output except errors.
# Extracts to outdir
$ dtk ar extract lib.a -o outdir
# With multiple inputs, extracts to separate directories
# Extracts to outdir/lib1, outdir/lib2
$ dtk ar extract lib1.a lib2.a -o outdirDemangles CodeWarrior C++ symbols. A thin wrapper for cwdemangle.
$ dtk demangle 'BuildLight__9CGuiLightCFv'
CGuiLight::BuildLight() constdisc commands are wrappers around the nod library
and its nodtool command line tool.
Displays information about disc images.
Supported disc image formats:
- ISO (GCM)
- WIA / RVZ
- WBFS (+ NKit 2 lossless)
- CISO (+ NKit 2 lossless)
- NFS (Wii U VC)
- GCZ
$ dtk disc info /path/to/game.isoExtracts the contents of disc images to a directory.
See disc info for supported formats.
$ dtk disc extract /path/to/game.iso [outdir]By default, only the main data partition is extracted.
Use the -p/--partition option to choose a different partition.
(Options: all, data, update, channel, or a partition index)
Converts any supported disc image to raw ISO (GCM).
If the format is lossless, the output will be identical to the original disc image.
See disc info for supported formats.
$ dtk disc convert /path/to/game.wia /path/to/game.isoHashes the contents of a disc image and verifies it against a built-in Redump database.
See disc info for supported formats.
$ dtk disc verify /path/to/game.isoAnalyzes a DOL file and outputs information section and symbol information.
$ dtk dol info input.dolNote
This command is a work-in-progress.
Analyzes and splits a DOL file into relocatable objects based on user configuration.
This command is intended to be used as part of a decompilation project's build system.
For an example project structure and for documentation on the configuration, see
dtk-template.
$ dtk dol split config.yml targetSimple diff tool for issues in a linked ELF. (Yes, not DOL. It's misnamed.)
Tries to find the most obvious difference causing a mismatch.
Pass in the project configuration file, and the path to the linked ELF file to compare against.
$ dtk dol diff config.yml build/main.elfApplies updated symbols from a linked ELF to the project configuration. (Again, misnamed.)
Useful after matching a file. It will pull updated symbol information from the final result.
$ dtk dol apply config.yml build/main.elfGenerates an initial project configuration file from a DOL (& RELs).
Pass in the DOL file, and any REL files that are linked with it.
Or, for Wii games, pass in the selfile.sel. (Not RSOs)
$ dtk dol config main.dol rels/*.rel -o config.ymlDumps DWARF 1.1 information from an ELF file. (Does not support DWARF 2+)
$ dtk dwarf dump input.elfDisassemble an unstripped CodeWarrior ELF file. Attempts to automatically split objects and rebuild relocations when possible.
$ dtk elf disasm input.elf outFixes issues with GNU assembler-built objects to ensure compatibility with mwldeppc.exe.
- Strips empty sections
- Generates section symbols for all allocatable sections
- Where possible, replaces section-relative relocations with direct relocations.
- Adds an
(asm)suffix to the file symbol. (For matching progress calculation)
# input and output can be the same
$ dtk elf fixup file.o file.oCreates a DOL file from the provided ELF file.
$ dtk elf2dol input.elf output.dolProcesses CodeWarrior map files and provides information about symbols and TUs.
$ dtk map entries Game.MAP 'Unit.o'
# Outputs all symbols that are referenced by Unit.o
# This is useful for finding deduplicated weak functions,
# which only show on first use in the link map.
$ dtk map symbol Game.MAP 'Function__5ClassFv'
# Outputs reference information for Function__5ClassFv
# CodeWarrior link maps can get very deeply nested,
# so this is useful for emitting direct references
# in a readable format.Prints information about a REL file.
$ dtk rel info input.relMerges a DOL file and associated RELs into a single ELF file, suitable for analysis in your favorite reverse engineering software.
$ dtk rel info main.dol rels/*.rel -o merged.elfWarning
This command is not yet functional.
Prints information about an RSO file.
$ dtk rso info input.rsoCalculate and verify SHA-1 hashes.
$ dtk shasum baserom.dol
949c5ed7368aef547e0b0db1c3678f466e2afbff baserom.dol
$ dtk shasum -c baserom.sha1
baserom.dol: OKDecompresses NLZSS-compressed files.
$ dtk nlzss decompress input.bin.lz -o output.bin
# or, for batch processing
$ dtk nlzss decompress rels/*.lz -o relsLists the contents of an RARC (older .arc) archive.
$ dtk rarc list input.arcExtracts the contents of an RARC (older .arc) archive.
$ dtk rarc extract input.arc -o output_dirExtracts the contents of a U8 (newer .arc) archive.
$ dtk u8 list input.arcExtracts the contents of a U8 (newer .arc) archive.
$ dtk u8 extract input.arc -o output_dirDecompresses Yay0-compressed files.
$ dtk yay0 decompress input.bin.yay0 -o output.bin
# or, for batch processing
$ dtk yay0 decompress rels/*.yay0 -o relsCompresses files using Yay0 compression.
$ dtk yay0 compress input.bin -o output.bin.yay0
# or, for batch processing
$ dtk yay0 compress rels/* -o relsDecompresses Yaz0-compressed files.
$ dtk yaz0 decompress input.bin.yaz0 -o output.bin
# or, for batch processing
$ dtk yaz0 decompress rels/*.yaz0 -o relsCompresses files using Yaz0 compression.
$ dtk yaz0 compress input.bin -o output.bin.yaz0
# or, for batch processing
$ dtk yaz0 compress rels/* -o rels
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent