pcrain / slippc Goto Github PK
View Code? Open in Web Editor NEWSlippi replay (.slp) parser, compressor, JSON converter, and basic analysis program written in C++
License: GNU General Public License v3.0
Slippi replay (.slp) parser, compressor, JSON converter, and basic analysis program written in C++
License: GNU General Public License v3.0
$ ./src/slippc -i crashes/crash5.slp -d
Loading crashes/crash5.slp
Parsing header
Slippi Header Matched
Raw portion = 25730 bytes
Parsing event descriptions
Event description length = 5 bytes
Payload size for event 0x36: 417 bytes
Payload size for event 0x37: 63 bytes
Parsing events proper
Warning: unknown event code 0x0 encountered
Warning: unknown event code 0x3f encountered
=================================================================
==99581==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61a0000040c0 at pc 0x00010b2557c0 bp 0x7ffee49c7ce0 sp 0x7ffee49c7cd8
READ of size 8 at 0x61a0000040c0 thread T0
#0 0x10b2557bf in slip::Parser::_parsePostFrame() parser.cpp:223
#1 0x10b242cb4 in slip::Parser::_parseEvents() parser.cpp:121
#2 0x10b23e78f in slip::Parser::_parse() parser.cpp:48
#3 0x10b23dded in slip::Parser::load(char const*) parser.cpp:36
#4 0x10b239986 in main main.cpp:28
#5 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
Address 0x61a0000040c0 is a wild pointer.
SUMMARY: AddressSanitizer: heap-buffer-overflow parser.cpp:223 in slip::Parser::_parsePostFrame()
This one wasn't found by fuzzing! It's just a raw .slp file I downloaded from slippi.gg.
$ ./slippc -i Game_20190324T113415.slp -a -
=================================================================
==620==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6250000048e0 at pc 0x000108aa9200 bp 0x7ffee71a9f20 sp 0x7ffee71a9f18
READ of size 4 at 0x6250000048e0 thread T0
#0 0x108aa91ff in slip::Analyzer::analyzePunishes(slip::SlippiReplay const&, slip::Analysis*) const analyzer.cpp:581
#1 0x108ab1ad2 in slip::Analyzer::analyze(slip::SlippiReplay const&) analyzer.cpp:707
#2 0x108b20a49 in slip::Parser::analyze() parser.cpp:459
#3 0x108adab92 in main main.cpp:64
#4 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
0x6250000048e0 is located 0 bytes to the right of 8160-byte region [0x625000002900,0x6250000048e0)
allocated by thread T0 here:
#0 0x108cd5c3d in wrap__Znam asan_new_delete.cc:133
#1 0x108ab305d in slip::AnalysisPlayer::AnalysisPlayer() analysis.h:68
#2 0x108ab25da in slip::Analysis::Analysis() analysis.h:65
#3 0x108ab1268 in slip::Analyzer::analyze(slip::SlippiReplay const&) analyzer.cpp:691
#4 0x108b20a49 in slip::Parser::analyze() parser.cpp:459
#5 0x108adab92 in main main.cpp:64
#6 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
SUMMARY: AddressSanitizer: heap-buffer-overflow analyzer.cpp:581 in slip::Analyzer::analyzePunishes(slip::SlippiReplay const&, slip::Analysis*) const
AddressSanitizer:DEADLYSIGNAL
=================================================================
==74712==ERROR: AddressSanitizer: BUS on unknown address 0x61a200000082 (pc 0x000108c68656 bp 0x7ffee6fa2df0 sp 0x7ffee6fa2cc0 T0)
#0 0x108c68655 in slip::Parser::_parseEvents() parser.cpp:143
#1 0x108c62dae in slip::Parser::_parse() parser.cpp:51
#2 0x108c6231d in slip::Parser::load(char const*) parser.cpp:39
#3 0x108c5dd07 in main main.cpp:29
#4 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
crash8.slp
Found before your latest changes, so might be fixed already, but I suspect probably not:
=================================================================
==15275==ERROR: AddressSanitizer: requested allocation size 0xb58d4c00c6b13340 (0xb58d4c00c6b14340 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0)
#0 0x102ae4b2d in wrap__Znwm asan_new_delete.cc:130
#1 0x7fff5b0837b1 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__grow_by_and_replace(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, char const*) (libc++.1.dylib:x86_64+0x37b1)
#2 0x7fff5b0836e2 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::assign(char const*, unsigned long) (libc++.1.dylib:x86_64+0x36e2)
#3 0x7fff5b083670 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::operator=(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (libc++.1.dylib:x86_64+0x3670)
#4 0x1028b3262 in slip::Analyzer::analyze(slip::SlippiReplay const&) analyzer.cpp:701
#5 0x102924209 in slip::Parser::analyze() parser.cpp:459
#6 0x1028dcb6f in main main.cpp:64
#7 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
==15275==HINT: if you don't care about these errors you may set allocator_may_return_null=1
SUMMARY: AddressSanitizer: allocation-size-too-big asan_new_delete.cc:130 in wrap__Znwm
==15275==ABORTING
Abort trap: 6
This will be crash12.slp and I will upload to my fork now.
I use this program extensively to compress all my slippi files, but it seems 3.8.0 isn't supported. Could you update this?
=================================================================
==74808==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x625000002650 at pc 0x00010c73d370 bp 0x7ffee34ddc80 sp 0x7ffee34ddc78
WRITE of size 4 at 0x625000002650 thread T0
#0 0x10c73d36f in slip::Parser::_parsePreFrame() parser.cpp:232
#1 0x10c72d173 in slip::Parser::_parseEvents() parser.cpp:133
#2 0x10c727dae in slip::Parser::_parse() parser.cpp:51
#3 0x10c72731d in slip::Parser::load(char const*) parser.cpp:39
#4 0x10c722d07 in main main.cpp:29
#5 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
0x625000002650 is located 4 bytes to the right of 9548-byte region [0x625000000100,0x62500000264c)
allocated by thread T0 here:
#0 0x10c7d0c3d in wrap__Znam asan_new_delete.cc:133
#1 0x10c75c59f in slip::SlippiReplay::setFrames(int) replay.cpp:10
#2 0x10c7391f0 in slip::Parser::_parseGameStart() parser.cpp:219
#3 0x10c72d0fd in slip::Parser::_parseEvents() parser.cpp:132
#4 0x10c727dae in slip::Parser::_parse() parser.cpp:51
#5 0x10c72731d in slip::Parser::load(char const*) parser.cpp:39
#6 0x10c722d07 in main main.cpp:29
#7 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
SUMMARY: AddressSanitizer: heap-buffer-overflow parser.cpp:232 in slip::Parser::_parsePreFrame()
crash10.slp
In file included from analysis.cpp:1:
In file included from ./analysis.h:11:
./util.h:168:24: error: implicit instantiation of undefined template 'std::__1::basic_ostringstream<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
std::ostringstream o;
^
/usr/local/bin/../include/c++/v1/iosfwd:136:32: note: template is declared here
class _LIBCPP_TEMPLATE_VIS basic_ostringstream;
^
analysis.cpp:13:21: error: implicit instantiation of undefined template 'std::__1::basic_stringstream<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
std::stringstream ss;
^
/usr/local/bin/../include/c++/v1/iosfwd:139:32: note: template is declared here
class _LIBCPP_TEMPLATE_VIS basic_stringstream;
^
Both can be solved by adding #include <sstream>
to util.h
$ ./src/slippc -i crashes/crash4.slp -d
Loading crashes/crash4.slp
Parsing header
Slippi Header Matched
Raw portion = 25730 bytes
Parsing event descriptions
Event description length = 13 bytes
Payload size for event 0x36: 417 bytes
Payload size for event 0x37: 63 bytes
Payload size for event 0x38: 37 bytes
Payload size for event 0x39: 1 bytes
Parsing events proper
Estimated 125 (+123) frames
Estimated 125 (+123) frames
Warning: unknown event code 0x15 encountered
Warning: unknown event code 0x1 encountered
Warning: unknown event code 0x42 encountered
Warning: unknown event code 0xc2 encountered
AddressSanitizer:DEADLYSIGNAL
=================================================================
==97544==ERROR: AddressSanitizer: BUS on unknown address 0x61a200000024 (pc 0x0001028e4278 bp 0x7ffeed326e30 sp 0x7ffeed326d20 T0)
#0 0x1028e4277 in slip::Parser::_parseEvents() parser.cpp:130
#1 0x1028df78f in slip::Parser::_parse() parser.cpp:48
#2 0x1028deded in slip::Parser::load(char const*) parser.cpp:36
#3 0x1028da986 in main main.cpp:28
#4 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
See details in #1.
==74759==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60c0000028c8 at pc 0x00010210c960 bp 0x7ffeedb10c80 sp 0x7ffeedb10c78
WRITE of size 1 at 0x60c0000028c8 thread T0
#0 0x10210c95f in slip::Parser::_parsePostFrame() parser.cpp:265
#1 0x1020fa15f in slip::Parser::_parseEvents() parser.cpp:134
#2 0x1020f4dae in slip::Parser::_parse() parser.cpp:51
#3 0x1020f431d in slip::Parser::load(char const*) parser.cpp:39
#4 0x1020efd07 in main main.cpp:29
#5 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
Address 0x60c0000028c8 is a wild pointer.
SUMMARY: AddressSanitizer: heap-buffer-overflow parser.cpp:265 in slip::Parser::_parsePostFrame()
crash9.slp
Program always exits with the same error message:
libc++abi.dylib: terminating with uncaught exception of type std::__1::regex_error: The expression contained mismatched ( and ).
Abort trap: 6
I can only reproduce this on my MacOS machine, not on Ubuntu. The way I've been dealing with this is to just remove the code that uses comma_killer
.
Any time a file that doesn't exist is passed to slippc, the program prints the "file does not exist" error message, and then this happens:
AddressSanitizer:DEADLYSIGNAL
=================================================================
==59048==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000100d93588 bp 0x7ffeef14b950 sp 0x7ffeef14b920 T0)
==59048==The signal is caused by a READ memory access.
==59048==Hint: address points to the zero page.
#0 0x100d93587 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) sanitizer_atomic_clang.h:80
#1 0x100de20b5 in wrap__ZdaPv asan_new_delete.cc:205
#2 0x100b55f6c in slip::Parser::~Parser() (slippc:x86_64+0x1000a2f6c)
#3 0x100b2ebbc in main (slippc:x86_64+0x10007bbbc)
#4 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
Looks like you're trying to delete an array that was never new'ed
, because the file not existing means that code never executed. Should be fixable with a simple conditional.
Getting tired of copy/pasting all the info, I'm sure you can manage without it. I've just pushed them to my fork. Reproduced on both MacOS and Ubuntu.
As reported by address sanitizer:
==82068==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x622000001654 at pc 0x000102b994a0 bp 0x7ffeed07fcc0 sp 0x7ffeed07fcb8
WRITE of size 4 at 0x622000001654 thread T0
#0 0x102b9949f in slip::Parser::_parsePreFrame() parser.cpp:195
#1 0x102b8aa2f in slip::Parser::_parseEvents() parser.cpp:118
#2 0x102b8695f in slip::Parser::_parse() parser.cpp:48
#3 0x102b85ded in slip::Parser::load(char const*) parser.cpp:36
#4 0x102b81a68 in main main.cpp:28
#5 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
0x622000001654 is located 4 bytes to the right of 5456-byte region [0x622000000100,0x622000001650)
allocated by thread T0 here:
#0 0x102c2dc3d in wrap__Znam asan_new_delete.cc:133
#1 0x102bb7a5b in slip::SlippiReplay::setFrames(int) replay.cpp:10
#2 0x102b95cd1 in slip::Parser::_parseGameStart() parser.cpp:182
#3 0x102b8a85f in slip::Parser::_parseEvents() parser.cpp:115
#4 0x102b8695f in slip::Parser::_parse() parser.cpp:48
#5 0x102b85ded in slip::Parser::load(char const*) parser.cpp:36
#6 0x102b81a68 in main main.cpp:28
#7 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
SUMMARY: AddressSanitizer: heap-buffer-overflow parser.cpp:195 in slip::Parser::_parsePreFrame()
Note that I reformatted the code for my own taste, so the line numbers won't match up exactly, but the stack trace should be accurate.
Compiled with:
afl-cc 2.51b by <[email protected]>
clang version 9.0.0 (https://github.com/llvm/llvm-project.git 8c5555c39a7a2f0cbb4720cacba32c53d601f785)
Target: x86_64-apple-darwin18.5.0
I will be reporting a few other crashes shortly. Please note the "reformatting disclaimer" and compiler information remains the same for these issues as well.
What is the best way to get the offending file to you, so that you can reproduce?
AddressSanitizer:DEADLYSIGNAL
=================================================================
==92814==ERROR: AddressSanitizer: BUS on unknown address 0x61a200000026 (pc 0x000102ce02fb bp 0x7ffeecf28e50 sp 0x7ffeecf28ce0 T0)
#0 0x102ce02fa in slip::Parser::_parseEventDescriptions() parser.cpp:98
#1 0x102cdd73e in slip::Parser::_parse() parser.cpp:44
#2 0x102cdcded in slip::Parser::load(char const*) parser.cpp:36
#3 0x102cd8986 in main main.cpp:28
#4 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
See more detailed info in #1.
=================================================================
==74585==ERROR: AddressSanitizer: BUS on unknown address 0x61a1fffffff0 (pc 0x0001049b8a8e bp 0x7ffeeb24fdf0 sp 0x7ffeeb24fc40 T0)
#0 0x1049b8a8d in slip::Parser::_parseEventDescriptions() parser.cpp:102
#1 0x1049b5d9e in slip::Parser::_parse() parser.cpp:47
#2 0x1049b531d in slip::Parser::load(char const*) parser.cpp:39
#3 0x1049b0d07 in main main.cpp:29
#4 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
As usual, the line numbers are gonna be off. The details are the same as before. I'll upload the files to my fork after I create all the issues. This one is crash6.slp
.
==74676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x622000003fc4 at pc 0x000107a93370 bp 0x7ffee8187c80 sp 0x7ffee8187c78
WRITE of size 4 at 0x622000003fc4 thread T0
#0 0x107a9336f in slip::Parser::_parsePreFrame() parser.cpp:232
#1 0x107a83173 in slip::Parser::_parseEvents() parser.cpp:133
#2 0x107a7ddae in slip::Parser::_parse() parser.cpp:51
#3 0x107a7d31d in slip::Parser::load(char const*) parser.cpp:39
#4 0x107a78d07 in main main.cpp:29
#5 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
Address 0x622000003fc4 is a wild pointer.
SUMMARY: AddressSanitizer: heap-buffer-overflow parser.cpp:232 in slip::Parser::_parsePreFrame()
crash7.slp
I get a weird issue claiming std::filesystem has not been declared in util.h. However there is a #include <filesystem>
line there that should work. Googling told me that this is an issue for older versions of g++, but I have a new version that should fully support C++17.
nick@nick-lenovo:~/bin/slippc$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
nick@nick-lenovo:~/bin/slippc$ make static
Building file: src/parser.cpp
Invoking: GCC C++ Compiler
cc src/parser.cpp -c -o build/parser.o
In file included from src/parser.h:9,
from src/parser.cpp:1:
src/util.h: In function ‘bool slip::makeDirectoryIfNotExists(const char*)’:
src/util.h:639:15: error: ‘std::filesystem’ has not been declared
639 | return std::filesystem::create_directories(path);
| ^~~~~~~~~~
make: *** [makefile:96: build/parser.o] Error 1
nick@nick-lenovo:~/bin/slippc$ g++ --version
g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE
==30005==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000104ad2928 at pc 0x00010494239a bp 0x7ffeeb30aec0 sp 0x7ffeeb30aeb8
READ of size 4 at 0x000104ad2928 thread T0
#0 0x104942399 in slip::Analyzer::analyzeInteractions(slip::SlippiReplay const&, slip::Analysis*) const analyzer.h:48
#1 0x10494dceb in slip::Analyzer::analyze(slip::SlippiReplay const&) analyzer.cpp:704
#2 0x1049c10a9 in slip::Parser::analyze() parser.cpp:470
#3 0x104977ccf in main main.cpp:64
#4 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
0x000104ad2928 is located 24 bytes to the left of global variable '<string literal>' defined in 'parser.cpp:382:22' (0x104ad2940) of size 2
'<string literal>' is ascii string ','
0x000104ad2928 is located 29 bytes to the right of global variable '<string literal>' defined in 'parser.cpp:373:29' (0x104ad2900) of size 11
'<string literal>' is ascii string '(,)(\s*\})'
SUMMARY: AddressSanitizer: global-buffer-overflow analyzer.h:48 in slip::Analyzer::analyzeInteractions(slip::SlippiReplay const&, slip::Analysis*) const
crash13.slp, coming up in a moment in the usual location :)
=================================================================
==74918==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61a0000027c0 at pc 0x00010687b890 bp 0x7ffee93a2c60 sp 0x7ffee93a2c58
READ of size 8 at 0x61a0000027c0 thread T0
#0 0x10687b88f in slip::Parser::_parsePostFrame() parser.cpp:265
#1 0x10686815f in slip::Parser::_parseEvents() parser.cpp:134
#2 0x106862dae in slip::Parser::_parse() parser.cpp:51
#3 0x10686231d in slip::Parser::load(char const*) parser.cpp:39
#4 0x10685dd07 in main main.cpp:29
#5 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
Address 0x61a0000027c0 is a wild pointer.
crash11.slp
Sorry I got kind of lazy with the issue descriptions; I think you get the point though. Let me know if you need anything else. About to push these files to my fork now.
Showing additional output this time:
$ ./src/slippc -i crashes/crash3.slp -d
Loading crashes/crash3.slp
Parsing header
Slippi Header Matched
Raw portion = 25730 bytes
Parsing event descriptions
Event description length = 13 bytes
Payload size for event 0x36: 417 bytes
Payload size for event 0x37: 63 bytes
Payload size for event 0x38: 37 bytes
Payload size for event 0x39: 9 bytes
Parsing events proper
Estimated 124 (+123) frames
Warning: unknown event code 0x74 encountered
Warning: unknown event code 0x61 encountered
Warning: unknown event code 0x7b encountered
Warning: unknown event code 0x55 encountered
Warning: unknown event code 0x7 encountered
Warning: unknown event code 0x73 encountered
Warning: unknown event code 0x74 encountered
Warning: unknown event code 0x61 encountered
Warning: unknown event code 0x72 encountered
Warning: unknown event code 0x74 encountered
Warning: unknown event code 0x41 encountered
Warning: unknown event code 0x74 encountered
Warning: unknown event code 0x53 encountered
Warning: unknown event code 0x55 encountered
Warning: unknown event code 0x13 encountered
Warning: unknown event code 0x32 encountered
Warning: unknown event code 0x30 encountered
Warning: unknown event code 0x31 encountered
Warning: unknown event code 0x3a encountered
Warning: unknown event code 0x30 encountered
Warning: unknown event code 0x31 encountered
Warning: unknown event code 0x3a encountered
Warning: unknown event code 0x34 encountered
=================================================================
==95584==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61a000001690 at pc 0x00010a9c7dfe bp 0x7ffee5252ce0 sp 0x7ffee5252cd8
READ of size 8 at 0x61a000001690 thread T0
#0 0x10a9c7dfd in slip::Parser::_parsePreFrame() parser.cpp:195
#1 0x10a9b7dea in slip::Parser::_parseEvents() parser.cpp:118
#2 0x10a9b378f in slip::Parser::_parse() parser.cpp:48
#3 0x10a9b2ded in slip::Parser::load(char const*) parser.cpp:36
#4 0x10a9ae986 in main main.cpp:28
#5 0x7fff5e0793d4 in start (libdyld.dylib:x86_64+0x163d4)
Address 0x61a000001690 is a wild pointer.
SUMMARY: AddressSanitizer: heap-buffer-overflow parser.cpp:195 in slip::Parser::_parsePreFrame()
See details in #1.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.