diasurgical / devilution-comparer Goto Github PK
View Code? Open in Web Editor NEWSmall helper tool to aid comparing functions between devilution and the original binary
License: The Unlicense
devilution-comparer's People
Contributors
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Stargazers
Watchers
devilution-comparer's Issues
Complete the function mapping
- Add missing sizes for all functions with tail call optimizations: https://gist.github.com/seritools/9a565ccc3aba405b1a3f074fa781c109
You can find the size of the "function chunk" (since IDA doesn't treat that as an actual function) like this:
BTW: the rest of the unmapped functions in devilution:
WARN: Function 'FindRep' was not found in the config.
WARN: Function 'Expand' was not found in the config.
WARN: Function 'FlushBuf' was not found in the config.
WARN: Function 'OutputBits' was not found in the config.
WARN: Function '__SETS__' was not found in the config.
WARN: Function 'implode' was not found in the config.
WARN: Function 'GenDecodeTabs' was not found in the config.
WARN: Function 'explode' was not found in the config.
WARN: Function 'GenAscTabs' was not found in the config.
WARN: Function '__OFSUB__' was not found in the config.
WARN: Function 'DecodeDist' was not found in the config.
WARN: Function 'WriteCmpData' was not found in the config.
WARN: Function '$E1' was not found in the config.
WARN: Function '$E4' was not found in the config.
WARN: Function '$E5' was not found in the config.
WARN: Function '__ROL__' was not found in the config.
WARN: Function 'operator delete' was not found in the config.
WARN: Function '$E2' was not found in the config.
WARN: Function 'SortBuffer' was not found in the config.
WARN: Function 'WasteBits' was not found in the config.
WARN: Function 'DecodeLit' was not found in the config.
$E<number> are the c++ struct initializers, __NAME__ are IDA remains, and the rest are from the PKWare explore.cpp and implode.cpp.
PM_DoBlock is to long
devilution-comparer/comparer-config.toml
Line 7547 in ae4515d
Looks like it's including the next function:
Switch jump table is treated as part of the function
I don't know if it's related to #11 and/or deficiency of pdb but seems like switch jump table is added to diff and also incorrectly parsed as a code. You can easily test it on OperateObject function:
Make displacements and indirect calls "local"
--no-mem-disp does a good job of cleaning up the diff and still show you the sizes. But leaves you in a situation where you can't know if you swaped one for another. Still even with out --no-mem-disp enabled it can be hard to track, take this as an exampel:
It's easy to miss that line 32 is actually referencing the wrong value.
If we could instead lable them with a local reference the issue becomes much clear then what we get even with out --no-mem-disp:
Add all functions with their respective names and offsets/lengths
To make using devilution-comparer even easier, a full mapping from a function name to its respective offset and length in the binary would be awesome. We could remove the DIABLO_OFFSET_START parameter or make it optional then.
This would make using the tool to generate a full diff easier as well.
- Gather the full function mapping for the 1.09b version and integrate that into devilution-comparer
- Clean up the function definitions
-
Add size information to all function definitionsTail-call-optimized fucntions are still missing - Implement a global check mode for the riivaaja builds
Show relative jumps as relative jumps instead of their calculated address
This should make the "diff noise" considerably smaller, for example in cases where there is an additional instruction shifting everything after that a few bytes.
This causes any line with a relative jump to show a diff, since the calculated absolute address is different, even though they all are correct/binary exact.
capstone-rs currently doesn't expose this info, so we'll probably either have to fork it or use the underlying capstone-sys, which are raw bindings to the lib.
Compiler warning about unhandled errors
When compiling with Rust version 1.33.0.
Compiling devilution-comparer v0.3.5 (devilution-comparer)
warning: unused `std::result::Result` that must be used
--> src/generate_full.rs:57:25
|
57 | / writeln!(
58 | | stdout_lock,
59 | | "Note: Skipping '{}' because no size was defined.",
60 | | func.name
61 | | );
| |__________________________^
|
= note: #[warn(unused_must_use)] on by default
= note: this `Result` may be an `Err` variant, which should be handled
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
warning: unused `std::result::Result` that must be used
--> src/generate_full.rs:126:21
|
126 | / writeln!(
127 | | stdout_lock,
128 | | "WARN: Function '{}' was not found in the PDB.",
129 | | func.name
130 | | );
| |______________________^
|
= note: this `Result` may be an `Err` variant, which should be handled
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
warning: unused `std::result::Result` that must be used
--> src/generate_full.rs:134:17
|
134 | / writeln!(
135 | | stdout_lock,
136 | | "WARN: Function '{}' was not found in the config.",
137 | | func.1.name
138 | | );
| |__________________^
|
= note: this `Result` may be an `Err` variant, which should be handled
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
warning: unused `std::result::Result` that must be used
--> src/hexformat.rs:17:17
|
17 | write!(f, "+");
| ^^^^^^^^^^^^^^^
|
= note: this `Result` may be an `Err` variant, which should be handled
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
warning: unused `std::result::Result` that must be used
--> src/hexformat.rs:20:17
|
20 | write!(f, "0x");
| ^^^^^^^^^^^^^^^^
|
= note: this `Result` may be an `Err` variant, which should be handled
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
warning: unused `std::result::Result` that must be used
--> src/hexformat.rs:24:13
|
24 | write!(f, "-");
| ^^^^^^^^^^^^^^^
|
= note: this `Result` may be an `Err` variant, which should be handled
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
warning: unused `std::result::Result` that must be used
--> src/hexformat.rs:26:17
|
26 | write!(f, "0x");
| ^^^^^^^^^^^^^^^^
|
= note: this `Result` may be an `Err` variant, which should be handled
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
Finished dev [unoptimized + debuginfo] target(s) in 4m 03s
Make travis build devilution-comparer properly
Right now, the build fails because of zydis requiring C99, and travis' gcc (4.8.4) defaulting to that gnu90 standard.
Use .MAP file instead of .PDB
Currently we have the awkward situation where the VC6 linker is used to generate builds with a PDB file, since the VC5 linker can't process PDBs produced by the VC6 compiler. This means the builds end up with different layout, different startup stub, different static constructors etc...
But the VC5 linker can produce a .map file which includes the locations of all functions and variables in the final build, just like the PDB information. Could we switch to using it instead so we don't have to make separate builds?
Update function names
Some functions have recently been renamed meaning we get mismatches on full diff. Also part of render.cpp has been split in to inline sub-function to deal with the expected ASM usage in the original src so we might have to think about how to handle that when diffing.
WARN: Function 'encrypt_decrypt_block' was not found in the PDB.
WARN: Function 'encrypt_encrypt_block' was not found in the PDB.
WARN: Function 'encrypt_hash' was not found in the PDB.
WARN: Function 'encrypt_init_lookup_table' was not found in the PDB.
WARN: Function 'encrypt_compress' was not found in the PDB.
WARN: Function 'encrypt_pkware_read' was not found in the PDB.
WARN: Function 'encrypt_pkware_write' was not found in the PDB.
WARN: Function 'encrypt_decompress' was not found in the PDB.
WARN: Function 'mainmenu_action' was not found in the PDB.
WARN: Function 'Decrypt' was not found in the config.
WARN: Function 'GenAscTabs' was not found in the config.
WARN: Function '$E4' was not found in the config.
WARN: Function 'mainmenu_loop' was not found in the config.
WARN: Function '$E2' was not found in the config.
WARN: Function 'PkwareDecompress' was not found in the config.
WARN: Function 'PkwareCompress' was not found in the config.
WARN: Function 'SortBuffer' was not found in the config.
WARN: Function 'Hash' was not found in the config.
WARN: Function 'WriteCmpData' was not found in the config.
WARN: Function 'asm_trans_light_edge_1_3' was not found in the config.
WARN: Function 'DecodeDist' was not found in the config.
WARN: Function '__OFSUB__' was not found in the config.
WARN: Function 'asm_cel_light_edge' was not found in the config.
WARN: Function 'FindRep' was not found in the config.
WARN: Function 'explode' was not found in the config.
WARN: Function '$E5' was not found in the config.
WARN: Function '$E1' was not found in the config.
WARN: Function 'PkwareBufferRead' was not found in the config.
WARN: Function 'GenDecodeTabs' was not found in the config.
WARN: Function 'asm_trans_light_cel_1_3' was not found in the config.
WARN: Function 'PM_DoNewLvl' was not found in the config.
WARN: Function 'PkwareBufferWrite' was not found in the config.
WARN: Function 'implode' was not found in the config.
WARN: Function 'asm_trans_light_edge_0_2' was not found in the config.
WARN: Function 'WasteBits' was not found in the config.
WARN: Function 'OutputBits' was not found in the config.
WARN: Function 'DecodeLit' was not found in the config.
WARN: Function 'Encrypt' was not found in the config.
WARN: Function 'FlushBuf' was not found in the config.
WARN: Function 'InitHash' was not found in the config.
WARN: Function '__ROL__' was not found in the config.
WARN: Function '__SETS__' was not found in the config.
WARN: Function 'asm_cel_light_square' was not found in the config.
WARN: Function 'Expand' was not found in the config.
WARN: Function 'asm_trans_light_cel_0_2' was not found in the config.
A diff for a specific function
In devilution's #1122 @AJenbo responded about checking bin-exactness of a function in CI:
@sskras https://travis-ci.org/diasurgical/devilution/jobs/528380611 line 595
it's hard to have it any more automatic than that. It has been used to detect issues previously but for something like this, it's not of much help, though it does confirm the fix.
Thanks. The output looks still a little bit too scarse to be sure. I think it can be advanced. Will post details below. Please point me if I am missing something.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.