encounter / decomp-toolkit Goto Github PK
View Code? Open in Web Editor NEWA GameCube & Wii decompilation toolkit
License: Apache License 2.0
A GameCube & Wii decompilation toolkit
License: Apache License 2.0
https://github.com/Coockie1173/Goldeneye-Wii
GoldenEye Wii
USA
When splitting the game, it throws the following error:
build\tools\dtk.exe dol split config\SJBE52\config.yml build\SJBE52
INFO Loading config\SJBE52\config.yml
INFO Loading and analyzing 1 module (using 1 thread)
WARN module{name=main}: Control flow from 3:0x803AB514 hit known function 3:0x803AB55C (instruction: 3:0x803AB55C)
INFO Initial analysis completed in 0.969s (found 14036 functions)
INFO Rebuilding relocationninja: error: rebuilding 'build.ninja': subcommand failed
s and splitting
Failed: While processing object 'main.dol' (module ID 0)
Currently, there is no way to disambiguate between multiple symbols of the same name when adding relocations via config.yml. A very notable case would be adding a relocation to a string pool, as there could be hundreds or thousands of @stringbase0
symbols that exist in a project.
This could be done by splitting the target
field into two targetName
and targetLocation
fields, which hold the symbol name and address respectively. There could also be some way of combining the two fields (this is what I instinctively tried in the config.yml - @stringbase0:0x80b895e8
), but it could be difficult to determine where the symbol ends and where the location begins.
Section alignment is handled haphazardly. Ways to improve:
rel make
Right now, a symbol size mismatch will error and halt the diff. This is annoying for symbols where the true size was discovered and symbols.txt
hasn't yet been updated to reflect. If the next symbol is still at the correct address (meaning that alignment was not broken), omit a warning instead and continue.
It would be nice to have something like ninja diff
working for RELs. A use case can be found in https://github.com/robojumper/ss-dtk/tree/rel-wip (SOUE01) - with the latest commit on that branch, objdiff shows no difference, but checksum verification for the d_t_tackle REL fails for not immediately obvious reasons.
The reason in this case is that two virtual function overrides were (accidentally) swapped in the symbols, which resulted in an incorrect vtable after linking, but to figure this out I had to binary diff the whole REL in a hex editor - which is doable for a small REL like this but scales poorly.
>
on Windows writes UTF-16. An -o
option lets us write a UTF-8 file directly.
No response
Spongebob Squarepants: Creature from the Krusty Krab
USA V0
When I run dwarf dump
the structures are not populated correctly. Instead of being able to parse the struct, they get hit by and anonymous union at the tail end of it.
I did a simple fix by changing src/util/dwarf.rs
line 1838
and changing it from let name = name.ok_or_else(|| anyhow!("Member without name: {:?}", tag))?;
to let name = name.unwrap_or("".to_string());
I don't think that is a good fix but it did it for me. I think there might need to be a more specific fix for this.
When I try to do a dtk dwarf dump
on FIFA_d.elf from FIFA2004 Gamecube I get the error Failed: Unhandled user defined type ClassType
. I've tried to create the class type, and that fixed the error, but when i tried the command again i got the error Failed: Unhandled ClassType child Inheritance
Tools parsing splits.txt separately will be able to use this info. Keep it as a comment maybe?
No response
Wii Sports Resort
USA v1.1
While trying to analyze the game, I get the error of "Multiple sections with name .rodata". From what I understand, the dol file has a duplicate segment with the same name, but looking at it in IDA it seems like both contain different data. Is there any possible way of specifying the segment it should use?
objdiff joins the battle
While following dtk-template Getting Started using the OoT JP Wii VC 00000001.app (.app sha1 47546e48467ae14d712b8c207e911821586d1043) as the dol (renaming 00000001.app -> main.dol) I encountered the following crash:
$ ninja
[1/3] TOOL build/tools/dtk
Downloading https://github.com/encounter/decomp-toolkit/releases/download/v0.6.2/dtk-linux-x86_64 to build/tools/dtk
[2/3] SPLIT config/NACJ/config.yml
FAILED: build/NACJ/config.json
build/tools/dtk dol split config/NACJ/config.yml build/NACJ
INFO Loading config/NACJ/config.yml
INFO Loading and analyzing 1 module (using 1 thread)
WARN module{name=oot_vc}: Conflicting size for Control: was 0x190, now 0x18C
WARN module{name=oot_vc}: Conflicting size for Control: was 0x190, now 0x18C
WARN module{name=oot_vc}: Conflicting size for InitMetroTRK: was 0x94, now 0x98
INFO Initial analysis completed in 0.825s (found 4323 functions)
INFO Rebuilding relocations and splitting
thread '<unnamed>' panicked at src/obj/symbols.rs:308:36:
index out of bounds: the len is 0 but the index is 18446744073709551615
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted (core dumped)
ninja: error: rebuilding 'build.ninja': subcommand failed
The same crash occurs if I manually run (on the 0.6.4 release)
./dtk dol config main.dol -o config.yml
./dtk dol split config.yml output
dtk fails to split all versions Inazuma Eleven Strikers when symbols are applied.
With symbols_known turned off the analyzer doesn't seem to know these functions are local
Failed: While loading object 'main.dol'
Caused by:
Multiple symbols with name longlong2str: 11989 Function 0x801E7D10 and 12018 Function 0x801EAC5C
Enabling symbols_known
causes a cyclic dependency error (also on local functions it seems)
Failed: While processing object 'main.dol' (module ID 0)
Caused by:
Cyclic dependency (involving auto_load_match_tex__Fv_text) encountered while resolving link order
repo can be found here
In the GC mario party games, there is a branch to every .text symbol in a giant list to make the symbols visible to the rels. When dtk encounter this, it throws an error about trying to disassemble data as code
No response
F-Zero GX
USA, Japan
When attempting to analyze the F-Zero GX dol with the dol info
command, control flow errors are returned. Furthermore, when attempting to merge the DOL and REL files together to create an ELF which can be statically analyzed, even more control flow errors are returned, along with a "Tail call analysis" failure.
As in the title. This stops other functions from running due to it being returned as an error.
Could you clarify in the README to what degree this workflow is applicable to Wii U? In the middle it is mentioned that the Wii U disc format is supported, but that's far from the whole flow.
Thank you!
Currently working on this branch: https://github.com/rjkiv/rb3/tree/binstream
Trying to match binstream.cpp, but am unable to merge BinStream::Cached() and BinStream::GetPlatform() with fn_8077BAA0, the original function these link to in the elf.
I should mention that fn_8077BAA0 hasn't actually been defined, but I'm not sure if defining that would fix this issue.
Lots of elf files in the debugging.games archive fail to disassemble with similar errors. These look like linker-defined symbols which do not actually correspond to source locations. For example:
Megaman.elf:
Section symbol without section: Symbol { name: "_stack_end", address: 2149095232, size: 0, kind: Data, section: Absolute, scope: Dynamic, weak: false, flags: Elf { st_info: 17, st_other: 0 } }
smash.elf:
Section symbol without section: Symbol { name: "__SBSS2_END__", address: 2149515028, size: 0, kind: Data, section: Undefined, scope: Unknown, weak: false, flags: Elf { st_info: 17, st_other: 0 } }
I have to add these little tags to track these areas:
.4byte 0x4C4C2069
.4byte 0x6E202564
.2byte 0x0A00
.endobj "@633"
.2byte 0x0000
.obj "@634", local @ C44
.4byte 0x4F534368
.4byte 0x65636B41
.4byte 0x63746976
.4byte 0x65546872
.4byte 0x65616473
.4byte 0x3A204661
.4byte 0x696C6564
.4byte 0x20436865
.4byte 0x636B5468
.4byte 0x72656164
.4byte 0x51756575
.4byte 0x65287468
.4byte 0x72656164
.4byte 0x2D3E7175
.4byte 0x65756529
.4byte 0x20696E20
.4byte 0x25640A00
.endobj "@634"
.obj "@635", local @ C88
.4byte 0x4F534368
.4byte 0x65636B41
.4byte 0x63746976
The @ xxx areas would be nice if automatically emitted by DTK. It makes string lookup awful as I have to make sure I grab the right data for each symbol im adding to my WIP code.
Use case: Large assets embedded in the binary. decomp-toolkit should be able to extract and convert them to C headers that can be included.
DTK currently throws a "cryptic" error when the start address of a split is greater than the end of the split.
INFO Loading config\SB4E01\config.yml
INFO Loading and analyzing 1 module (using 1 thread)
INFO Initial analysis completed in 6.283s (found 50349 functions)
INFO Rebuilding relocations and splitting
thread '<unnamed>' panicked at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce\library\alloc\src\collections\btree\search.rs:120:21:
range start is greater than ninja: error: rebuilding 'build.ninja': subcommand failed
range end in BTreeMap
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
INFO Loading default.elf
INFO Splitting 949 objects
Failed: Range 0x00000000-0x80004380 outside of section .init: 0x80004000-0x80006728
relevant part from the info output:
.text | 0x800562E4 | 0x4 | sqrt
.text | 0x800562E8 | 0x4 | strdup
.text | 0x800562EC | 0x54 | strlwr
.text | 0x80056340 | 0xA0 | stricmp
.init | 0x80004380 | 0x0 | gTRKInterruptVectorTable
.init | 0x800062B4 | 0x0 | gTRKInterruptVectorTableEnd
.text | 0x800563E0 | 0x0 | TRKAccessFile
.text | 0x800563E8 | 0x0 | TRKOpenFile
.text | 0x800563F0 | 0x0 | TRKCloseFile
Initially I thought this was because the symbol was 0 sized, but then had the same issue with a different ELF
Failed: Range 0x00000000-0x80004300 outside of section .init: 0x80004000-0x800066BC
.bss | 0x804DF8E0 | 0x30 | __OSExpireAlarm
.text | 0x80293190 | 0x158 | __OSInitPlayTime
.rodata | 0x80465688 | 0x40 | @LOCAL@OSCalcCRC32__FPCvUl@crc32_table
.text | 0x802932F0 | 0x134 | OSCalcCRC32
.text | 0x80293430 | 0x204 | __OSRelaunchTitle
.init | 0x80004300 | 0x24 | __init_hardware
.text | 0x80293640 | 0x20 | __init_user
.text | 0x802936B0 | 0x4C | exit
.init | 0x80004330 | 0x34 | __flush_cache
.text | 0x80293700 | 0x2C | PSMTXIdentity
.text | 0x80293730 | 0x34 | PSMTXCopy
https://github.com/robojumper/ss-dtk/tree/dtk-invalid-rel
Skyward Sword
NTSC v1.0
Provide the main.dol and RELs, then run ninja in the provided branch
Failed: While processing object 'd_a_b_bigboss2NP.rel' (module d_a_b_bigboss2NP ID 122)
Caused by:
Found invalid relocation 0:0x99C4 Lo(
Address(
4:0x10FC,
),
) (target 4:0x10FC) in relocatable object
Adding
block_relocations:
- source: .text:0x000099C4
does not seem to work around this.
Right now it has a few bugs:
gap_
symbols are addedSymbol name differences that come from symbol globalization are more often than not just noise, and end up masking other issues whenever you're trying to find a real issue. Currently, the only way to make this difference go away is to temporarily set the affected symbol's scope to global. Ignoring these differences (or changing them to warnings) would avoid this issue entirely.
Had this happen on multiple dols (eg Wrath of Cortex (GC)), when running dtk dol info sys/main.dol
WARN Failed to locate _rom_copy_info
Failed: While processing 1:0x800C4E74: FunctionSlices {
blocks: {
1:0x800C4E74: Some(
1:0x800C4EB4,
),
1:0x800C4EB4: None,
1:0x800C4ED8: Some(
1:0x800C4F04,
),
1:0x800C4F04: None,
1:0x800C4F14: Some(
1:0x800C4F7C,
),
1:0x800C4F7C: None,
1:0x800C4F8C: Some(
1:0x800C4FF4,
),
1:0x800C4FF4: None,
1:0x800C5004: Some(
1:0x800C506C,
),
1:0x800C506C: None,
1:0x800C507C: None,
},
branches: {
1:0x800C4EB0: [
1:0x800C4EB4,
1:0x800C4ED8,
],
1:0x800C4F00: [
1:0x800C4F04,
1:0x800C4F14,
],
1:0x800C4F78: [
1:0x800C4F7C,
1:0x800C4F8C,
],
1:0x800C4FF0: [
1:0x800C4FF4,
1:0x800C5004,
],
1:0x800C5068: [
1:0x800C506C,
1:0x800C507C,
],
},
function_references: {},
jump_table_references: {},
prologue: Some(
1:0x800C4E74,
),
epilogue: None,
possible_blocks: {},
has_conditional_blr: false,
has_rfi: false,
finalized: false,
has_r1_load: false,
}
Caused by:
Found duplicate prologue: 1:0x800C4E74 and 1:0x800C517C
When running elf disasm
on an (unstripped) ELF file (from a Wii game), it fails with the message:
Failed: Range 0x00000000-{e_entry} outside of section .init: {.init->sh_addr}-{.init->sh_addr + .init->sh_size}
It may not happen on every ELF file, but I tried it on at least 10 and they all had the same result, so I think it probably does.
An actual example, from an actual game:
Failed: Range 0x00000000-0x80004134 outside of section .init: 0x80004000-0x800064C4
Seems to be emitted from obj/sections.rs:152:
ensure!(
self.contains_range(start..end),
"Range {:#010X}-{:#010X} outside of section {}: {:#010X}-{:#010X}",
start,
end,
self.name,
self.address,
self.address + self.size
);
I guess the problem is that it thinks the entry point is a range? No idea where that's determined, though.
i also don't know any rust i just grepped for the string
dtk elf disasm <path/to/file.elf> .
OS: Windows 10 22H2
dtk -V
: dtk 0.5.5 36bb5ddcc67dabaf5d00d7335a9e764c4c668ee7
also tried on macOS Catalina with 0.5.4
same stuff but im not on that computer rn so no specs
https://github.com/DarkRTA/rb3
Rock Band 3
SZBE69_B8
The lis
/subic.
instruction pair here is not marked as a relocation like it should be.
So far I've identified this issue in the following locations:
band3/meta_band/AccomplishmentCategory.cpp
- AccomplishmentCategory::HasAward()
band3/meta_band/AccomplishmentGroup.cpp
- AccomplishmentGroup::HasAward()
system/utl/HxGuid.cpp
- HxGuid::Generate()
For MKDD. Could automatically extract binary symbols in this section and include them in ldscript.lcf.
Just Dance 2 (Best Buy Edition), along with presumably any game using MetaFortress, deliberately create invalid instructions as an anti-piracy measure. Currently, dtk hard-fails when encountering an invalid instruction, such as when splitting out the dol. Instead, it should silently ignore invalid instructions and continue with analysis.
data:4byte |
data:int |
---|---|
.4byte 0xFFFFFF80 |
.int -128 |
Game: Bloody Roar: Primal Fury
Revision: 0
Region: NTSC (US)
INFO Loading config\GBLE52\config.yml
INFO Loading and analyzing 1 module (using 1 thread)
Failed: While loading object 'main.dol'
Caused by:
Failed to finalize block @ 3:0x80084054
I came across this when I was decomposing MSL; in particular, there's a file named sysenv.GCN.o
as specified by some unstripped .elf
files, so I added it to my splits
extern/msl/libc/embedded/sysenv.GCN.c:
.text start:0x804385FC end:0x80438604
but running dtk dol split
(dtk/config.yml disasm
) produces the following:
> ls -la disasm/obj/extern/msl/libc/embedded | grep sysenv
-rw-r--r-- 1 muffin None 540 Dec 17 15:15 sysenv.o
splits.txt
with a name with multiple periods in it like a.b.c
OS: Windows 10 22H2
dtk -V
: dtk 0.6.4 0cfc5df20b4d03604649afb88897d4e011fc9b26
dtk fails to detect a relocation in GetMesMaxSizeSub in mario party 4.
/* 800473F0 00044250 88 7F 00 00 */ lbz r3, 0x0(r31) /* 800473F4 00044254 38 63 89 A9 */ subi r3, r3, 0x7657 /* 800473F8 00044258 7F A3 68 AE */ lbzx r29, r3, r13
the relevant code is this. 0x7657 points to mesWInsert-0x1 which is equivalent to winTabSize
this code translates tovar_r29 = mesWInsert[*var_r31 - 1];
https://github.com/wowjinxy/dtk-brawl
Super Smash Brothers Brawl
RSBE01
E:\dtk-brawl>ninja
[1/3] TOOL build\tools\dtk.exe
Downloading https://github.com/encounter/decomp-toolkit/releases/download/v0.7.5/dtk-windows-x86_64.exe to build\tools\dtk.exe
[2/3] SPLIT config\RSBE01\config.yml
FAILED: build/RSBE01/config.json
build\tools\dtk.exe dol split config\RSBE01\config.yml build\RSBE01
INFO Loading config\RSBE01\config.yml
INFO Loading and analyzing 127 modules (using 16 threads)
ERROR module{name=sora_scene}: Failed to finalize functions:
ERROR module{name=sora_scene}: 0:0x000221D0
ERROR module{name=sora_scene}: 0:0x00022D3C
ERROR module{name=sora_scene}: 0:0x000234F4
ERROR module{name=sora_scene}: 0:0x0002442C
ERROR module{name=sora_scene}: 0:0x00024BA0
ERROR module{name=sora_scene}: 0:0x00032C3C
ERROR module{name=sora_scene}: 0:0x00036990
ERROR module{name=sora_scene}: 0:0x0003B16C
ERROR module{name=sora_scene}: 0:0x0003C26C
ERROR module{name=sora_scene}: 0:0x0003CFC8
Failed: While loading object 'main.dol'
Caused by:
Overlapping functions 3:0x802F7728-3:0x802F7910 -> 3:0x802F7ninja: error: rebuilding 'build.ninja': subcommand failed
744
https://github.com/ieee802dot11ac/krp
Karaoke Revolution Party
NTSC-U/C
the SPLIT stage of the ninja script fails with the following error:
[1/2] SPLIT config/GK9EA4/config.yml
FAILED: build/GK9EA4/config.json
build/tools/dtk dol split config/GK9EA4/config.yml build/GK9EA4
INFO Loading config/GK9EA4/config.yml
INFO Loading and analyzing 1 module (using 1 thread)
WARN module{name=main}: Failed to locate _rom_copy_info
WARN module{name=main}: Failed to locate BSS sections
Failed: While loading object 'main.dol'
Caused by:
Couldn't determine alignment for section '.data5' (0x802CC660 -> 0x803B3F80)
ninja: error: rebuilding 'build.ninja': subcommand failed
This appears to be caused by the fact that ProDG GCC for GameCube emits less data sections in the dol than MetroWerks for GameCube does, only having .data0 through .data4 before hitting the .bss sections.
Luigi's Mansion's executable contains YAY0-compressed assets, so support for YAY0 decompression would be appreciated. A C implementation of the decompression algorithm can be found here.
I've tried to dump the Dwarf from Gamecube's FIFA 2004 and with the last version it worked really well, but I noticed that the dump missed some functions and global variables, even with their data present in elf. For example, the data from global gPassTri is present here:
But this data and the data around that are not present in the dump
https://github.com/Yanis42/oot-gc-dtk
Zelda: Ocarina of Time - Master Quest and Collector's Edition N64 Emulator
Any
dtk is interpreting N64 addresses written in code as pointers, this makes the assembly harder to understand once decompiled (and it's giving a fake diff), the current workaround is to use noreloc
(which is what I used on the systemSetupGameALL
symbol)
to find these addresses in the source files you can look for calls of cpuSetCodeHack
https://github.com/ribbanya/melee/tree/8daa8eb261ccd111d7595bc73405607b2fb4cd5d
Super Smash Bros. Melee
GALE01
dtk keeps placing a function, whose definition is in .text
, but is referenced from .ctors
, in .ctors
. Well, more accurately, it keeps creating a ctors split where that function is. The function is __sinit_trigf_c
.
I'm not sure this is a bug. I would be entirely unsurprised if this is due to us doing something wrong, since as you can see __ctors
are not properly implemented in C.
make
)https://github.com/Rainchus/mp5-dtk
Mario Party 5
No response
With dtk 0.9.1, this error occurs when attempting to build RELs that are built only with the empty.c
C file
Given a TU with correctly assigned symbol sizes, dtk should be able to infer the unpadded endings of splits, like this:
melee/ft/chara/ftPurin/ftPr_SpecialHi.c:
.text start:0x8013C9C8 end:0x8013CE8C
- .sdata2 start:0x804D9C10 end:0x804D9C20
+ .sdata2 start:0x804D9C10 end:0x804D9C1C
melee/ft/chara/ftPurin/ftPr_SpecialLw.c:
.text start:0x8013CE8C end:0x8013D234
.sdata2 start:0x804D9C20 end:0x804D9C2C
Templates notably break module processing.
Causes issues in build.ninja
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.