Comments (10)
Correct. (and excellent debugging by the way)
Zstd is not yet protected against invalid input.
This is an objective, obviously.
Part of the work is done, since FSE seems to properly handle invalid input.
Now, Zstd proper must receive the same attention.
from zstd.
I found a variation of this crash that is most likely exploitable by people with malicious intent. Let me know where I can send the details, or I can publish the details here.
from zstd.
Hi Brian
It's just a bit too soon.
Zstd is not yet protected against such scenario, it's that simple.
Investigating specific situations will be useful later on, when protection code will be settled.
I'll keep this issue opened to track progresses.
from zstd.
#22 contains some interesting testing code initiated by @luben
from zstd.
Latest version of Zstd is now supposed to be safe against invalid input.
from zstd.
Not completely though. Here's another example file:
00000000 fd 2f b5 1e 00 00 31 40 00 20 31 32 32 32 32 32
00000010 32 31 33 31 32 33 31 31 31 32 32 32 32 32 33 31
00000020 32 32 31 31 32 32 32 32 32 0a 03 00 20 00 00 00
00000030 00 a1 00 10 00 08 42 10 c0 00 00
If you'll run zstd -d
on it (as of commit 79d3b1e), it will segfault with this backtrace:
#0 0x000000000040c9ac in FSE_read64 (memPtr=0x70101103b) at fse.c:202
#1 0x000000000040c8f5 in FSE_readLE64 (memPtr=0x70101103b) at fse.c:277
#2 0x000000000040474b in FSE_readLEST (memPtr=0x70101103b) at fse.c:311
#3 0x0000000000404a1e in FSE_reloadDStream (bitD=0x7fffffffe470) at fse.c:1594
#4 0x0000000000404a7c in FSE_initDState (DStatePtr=0x7fffffffe490, bitD=0x7fffffffe470, dt=0x801006000) at fse.c:1604
#5 0x000000000040a5d9 in ZSTD_decompressSequences (ctx=0x801006000, dst=0x801032000, maxDstSize=524288, seqStart=0x801011023, seqSize=14, litStart=0x801011003 "1222222131231112222231221122222\n\003", litSize=32) at ../lib/zstd.c:1543
#6 0x000000000040a387 in ZSTD_decompressBlock (ctx=0x801006000, dst=0x801032000, maxDstSize=524288, src=0x801011000, srcSize=14) at ../lib/zstd.c:1591
#7 0x000000000040a1df in ZSTD_decompressContinue (dctx=0x801006000, dst=0x801032000, maxDstSize=524288, src=0x801011000, srcSize=49) at ../lib/zstd.c:1734
#8 0x00000000004167d5 in FIO_decompressFilename (output_filename=0x4181d6 "-", input_filename=0x7fffffffec50 "--file-path-here--") at fileio.c:362
#9 0x00000000004174ee in main (argc=3, argv=0x7fffffffe998) at zstdcli.c:334
BTW, I'm using this fuzzer to find these crashes; I fully recommend you to use it too.
from zstd.
Thanks @magv, this is exactly the kind of feedback I'm looking for right now.
Thanks also for the link, I did not know this tool, and I'll certainly have a look at it !
from zstd.
This specific error fixed into fee8e24
from zstd.
Thanks for the link. AFL is really an excellent fuzzer tool.
It managed to find 2 more bugs that internal fuzzer tools did not triggered yet.
What's more, it did it without knowing the internal data format, just "guessing" the edges, and progressing along promising test cases. This is truly impressive.
After a few hours, it did not found any new bug, so I guess we are starting to see a relatively robust release ...
from zstd.
Yeah, AFL is magical. High yield with low effort? Yes, please.
I also see no crashes in the most recent commit, which is great, but I'd like to leave the fuzzer running for a while longer (a few more days would be good, I think).
from zstd.
Related Issues (20)
- MSVC CMake build failed on v1.5.6
- v1.5.6 Windows binary downloads are double zipped HOT 4
- Raise version's in win32 binaries header HOT 3
- Why was the new release 1.5.6 removed? HOT 15
- long file names are cut off in output HOT 3
- Should zstd check archive consistency before overwriting files? HOT 1
- Should zstd delete incomplete archives? HOT 5
- 32-bit x86 build failure with 1.5.6 HOT 3
- v1.5.6 breaks 32-bit Windows clang-cl build HOT 3
- Decompress multiple zstaa backups on FAT32 drives HOT 4
- Replication of bug #3517 HOT 24
- Separate dictionary references to enable dictionary usage for any combination of window size and content size HOT 1
- Decompression speed regression in zstd 1.5.6 (win)
- Embed hash of raw dictionary in compressed resource (optionally) HOT 4
- Decompression crash after upgrading from zstd 1.4.5 to 1.5.6 HOT 12
- Missing check on failed allocation leads to NULL-ptr dereference
- libzstd.lib missed in package, also VC sample seems include wrong mem.h or ambigious including!
- Environment variable for --memory HOT 2
- Improve misleading wording in the streaming decompression howto HOT 1
- erro
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zstd.