bisqwit / tinydeflate Goto Github PK

View Code? Open in Web Editor NEW

169.0 169.0 17.0 112 KB

A deflate/gzip decompressor that requires minimal amount of memory to work

C++ 100.00%

compression deflate embedded gzip

tinydeflate's People

Contributors

Stargazers

Watchers

Forkers

iterswap shekkbuilder templeblock sailfish009 eg0xy doc0160 f4568525153 happy-ferret paulwratt akj7 3xlogickevinhan drony 00mjk icodein stevenchen1976 rveips

tinydeflate's Issues

Response to certain lies

This is a response to some false claims made which use the respected word "Engineering" in it.
Hey
@bisqwit just wanted to say that Engineering has nothing to do with sitting behind a keyboard and writing code. There is no such thing as "reverse" "engineering". Engineering is a real subject. Engineering means designing and building real mechanical systems, like vehicular engines and turbojets. Software devs are claiming the most ridiculous of false claims these days.

Software dev is easy and for kids, and has absolutely nothing to do with a real and serious complex subject like engineering.

Mechanical Engineering is the only Engineering.

bye bye

Not a bug, C++20 removal of std::result_of

As title suggests building for C++20 where std::result_of has been removed after being deprecated in C++17, fails. I've implemented a generic solution, and what's actually being used simply copied from headers , thus not using a pull request. If you would consider adding proper support that would be ideal.

Putting this snipped at the top of the gunzip.hh then simply removing std:: from instances of result_of_t.

#if __cplusplus > 202000UL

// Normal solution for result_of , not what's being used
template <typename F, typename... Args>
struct result_of<F(Args...)> : std::invoke_result<F, Args...> {};

template <class _Fty>
struct result_of { // explain usage
    static_assert(_Always_false<_Fty>, "result_of<CallableType> is invalid; use "
        "result_of<CallableType(zero or more argument types)> instead.");
};

template <class _Ty>
using result_of_t = typename result_of<_Ty>::type;

#endif

Wanings compiling gunzip.hh on Windows

When compiling on Windows with the Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29914 for x64 compiler and enabling warnings I get the warnings:

...\TinyDeflate\gunzip.hh(983): warning C4800: Implicit conversion from 'uint_least64_t' to bool. Possible information loss
...\TinyDeflate\gunzip.hh(983): note: consider using explicit cast or comparison to 0 to avoid this warning
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include\stdint.h(34): note: see declaration of 'uint_least64_t'

Changing the lines 983 and 984 from:

            if(header&0x0800) for(;;) { GetBits(8, bool q); if(!q) break; } // NAME: Skip filename if FNAME was present
            if(header&0x1000) for(;;) { GetBits(8, bool q); if(!q) break; } // COMMENT: Skip comment if FCOMMENT was present

            if(header&0x0800) for(;;) { GetBits(8, std::uint_least64_t q); if(!q) break; } // NAME: Skip filename if FNAME was present
            if(header&0x1000) for(;;) { GetBits(8, std::uint_least64_t q); if(!q) break; } // COMMENT: Skip comment if FCOMMENT was present

solves the problem, though it might be even better to change them in:

            if(header&0x0800) for(;;) { GetBits(8, std::uint_least64_t q); if(q == 0) break; } // NAME: Skip filename if FNAME was present
            if(header&0x1000) for(;;) { GetBits(8, std::uint_least64_t q); if(q == 0) break; } // COMMENT: Skip comment if FCOMMENT was present

Divergence in the number of bytes read

I'm doing some tests with TinyDeflate, but I noticed a divergence in the number of bytes read.

In both cases I have the same gzip file loaded into a std::vector<uint8_t> (this approach is just for a proof of concept).

The code below returns 80275 bytes consumed:

std::vector<uint8_t> gzip_content{ /*...*/ };  // 80283 bytes
std::vector<uint8_t> bin_content;

auto result = Deflate(
    [&]() {
        static size_t i = 0;
        if (i < gzip_content.size())
            return (int)gzip_content[i++];
        return EOF;
    },
    [&](uint8_t data) { bin_content.push_back(data); },
    DeflateTrackBothSize{});

// result.first = 0
// result.second.first = 80275 (8 bytes less: chesksum + trailer ?)
// result.second.second = 221863
// bin_content is OK

However, the code below returns 81850 bytes consumed:

std::vector<uint8_t> gzip_content{ /*...*/ };  // 80283 bytes

size_t n = /* ... */;
uint8_t *bin_content = new uint8_t[n];

auto result = Deflate(
    (uint8_t *)gzip_content.data(),
    (uint8_t *)gzip_content.data() + gzip_content.size(),
    (uint8_t *)bin_content,
    (uint8_t *)bin_content + n,
    DeflateTrackBothSize{});

// result.first = 0
// result.second.first = 81850 (1567 bytes more)
// result.second.second = 221863
// bin_content is OK

Is this because of sentence (15)?

"This method is backtrackable, meaning that some bytes in the input may be read twice."

README.md example code triggers static_assert.

    extern const char compressed_data[];
    extern unsigned compressed_data_length;
    extern unsigned char outbuffer[131072];
    
    int result = Deflate(compressed_data+0, compressed_data_length, outbuffer, outbuffer + sizeof(outbuffer));
    if(result != 0) std::fprintf(stderr, "Error\n");

This is the code in question.

The result:

In file included from ../../project/test/test.cpp:13:
../../project/test/TinyDeflate/gunzip.hh: In instantiation of ‘auto gunzip_ns::DeflateOutputDispatch(BtFun&&, InFun&&, T1&&, T2&&) [with unsigned char code = 1; BtFun = DeflateInputDispatch<0, dummy, const char*, unsigned int&, unsigned char (&)[131072], unsigned char*>(dummy&&, const char*&&, unsigned int&, unsigned char (&)[131072], unsigned char*&&)::<lambda(bool)>&; InFun = DeflateInputDispatch<0, dummy, const char*, unsigned int&, unsigned char (&)[131072], unsigned char*>(dummy&&, const char*&&, unsigned int&, unsigned char (&)[131072], unsigned char*&&)::<lambda()>&; T1 = unsigned char (&)[131072]; T2 = unsigned char*]’:
../../project/test/TinyDeflate/gunzip.hh:1386:67:   required from ‘auto gunzip_ns::DeflateInputDispatch(BtFun&&, T1&&, T2&&, T&& ...) [with unsigned char code = 0; BtFun = dummy; T1 = const char*; T2 = unsigned int&; T = {unsigned char (&)[131072], unsigned char*}]’
../../project/test/TinyDeflate/gunzip.hh:1427:46:   required from ‘auto Deflate(T&& ...) [with T = {const char*, unsigned int&, unsigned char (&)[131072], unsigned char*}]’
../../project/test/test.cpp:22:23:   required from here
../../project/test/TinyDeflate/gunzip.hh:1311:31: error: static assertion failed: Deflate: Unknown output parameter type
 1311 |             static_assert(code==0xFF, "Deflate: Unknown output parameter type");
      |                           ~~~~^~~~~~
../../project/test/TinyDeflate/gunzip.hh:1311:31: note: the comparison reduces to ‘(1 == 255)’

I'm using GCC 12 with -std=c++20 if that makes a difference.

Any usage instructions?

As a not-so-experienced C programmer, I am not sure what to do with the .hh file. Could you kindly give a complete working solution?

bisqwit / tinydeflate Goto Github PK

tinydeflate's People

Contributors

Stargazers

Watchers

Forkers

tinydeflate's Issues

Response to certain lies

Not a bug, C++20 removal of std::result_of

Wanings compiling gunzip.hh on Windows

Divergence in the number of bytes read

README.md example code triggers static_assert.

Any usage instructions?

Could you provide a demo how to use your code?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent