Comments (6)
Thanks for notifying. It's clearly described.
I'll look into it.
Original comment by [email protected]
on 29 Oct 2012 at 11:06
- Changed state: Accepted
from lz4.
The first point is fairly clear.
The decoder exits silently becauses it believes it completed its job, which is
true when the compressed stream is correct. But not if the stream is
hand-crafted to be wrong, or if the output size is wrong.
The second point is clear too, but its interest more debatable.
If the "delta" is zero, it means, necessarily, that the decoding will be
incorrect : we are essentially copying "noise" or more probably initialised
values, such as "00".
When such a case does happen ?
Well, it cannot happen if the stream is produced by an LZ4 encoder, so the
stream must be modified, either by error, or hand-crafted for this situation to
be triggered.
If the stream is maliciously hand-crafted, then of course it is "wrong".
It's not possible to prevent such a scenario. There are many ways to modify a
compressed stream "transparently". For example, just modify a few literals, or
provide a wrong delta value. LZ4 will not detect that : it's not an encryption
layer.
So we don't even try to do that. What we want to ensure is the LZ4 format
cannot be exploited by attackers to silently crash a process or for any kind of
buffer overflow scenario : LZ4 shall never read nor write outside of provided
buffers.
Copying from a distance range of zero does not help for such a scenario. Since
it's useless, we don't even bother with it.
So now, we are targeting "errors".
How many times this specific error will happen ?
I mean, we suppose only the delta value is wrong, AND its value is zero.
So, on a collection of 65536 possible delta values, 1 is correct, 65535 are
wrong, and one of the wrong ones is 0. So we will detect one error among 65535
possible ones. Does that really improve anything ?
To be fair, if it could be demonstrated that, when a delta value is wrong, it's
more likely to be "zero", then okay, detecting this case could be useful.
Note that this detection has some cost : one test, one branch, so a potential
burden to performance. Better be useful...
Original comment by [email protected]
on 30 Oct 2012 at 2:46
from lz4.
Thanks for the details, Yann. I just wanted to make LZ4 fail sooner when
inconsistencies are encountered, these are not security issues: if one of them
proves to make uncompression slower then I fully agree that it should not be
fixed.
Thank you again!
Original comment by [email protected]
on 30 Oct 2012 at 3:10
from lz4.
The following attached file is supposed to implement your first suggestion
(regarding undetected end of file reached before oend).
I did not had time (yet) to test if the detection of delta=0 does cost some
performance.
Original comment by [email protected]
on 1 Nov 2012 at 5:46
Attachments:
from lz4.
Corrected into r82.
Well i have only corrected the main part, regarding detecting early exit (when
provided osize has wrong size). For the "delta 0" part, i'll have to start a
more thorough benchmark session.
Thanks for the clear description and proposed correction.
Original comment by [email protected]
on 3 Nov 2012 at 9:08
- Changed state: Fixed
from lz4.
Thanks for the fix!
Original comment by [email protected]
on 3 Nov 2012 at 9:12
from lz4.
Related Issues (20)
- Clang compiler warnings HOT 6
- LZ4F_compressBound() using null PreferencesPtr HOT 3
- lz4io license HOT 6
- Commandline lz4 stops when decompressing a concatenated lz4 file representing an empty file HOT 4
- Kernel wont boot with CONFIG_DYNAMIC_DEBUG=y HOT 2
- Bad alignment detection & setting on Sparc Solaris HOT 2
- lz4 r124 breaks ABI without bumping soname HOT 22
- Do not call sudo in Makefile HOT 3
- LZ4 HC r126 crashes when compiled with GCC 4.9 on highest optimization settings HOT 1
- lz4frame should correctly handle skipped frames HOT 4
- LZ4 cli invocation should match other compressors for non-option arguments HOT 14
- sudo make install does not cp files to /usr/local HOT 6
- lz4 auto-decompress does not work HOT 3
- Force should pass thru bytes unchanged HOT 5
- lz4 cli should support sparse file HOT 22
- decompression same file on different platforms HOT 3
- Static code analysis warnings HOT 10
- Google Code will close HOT 3
- [Query]Can LZo algorithm optimized for ARM NEON (SIMD instructions). HOT 3
- Potential Size Extension Problem in LZ4F_readLE64 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lz4.