codingkoopa / bmpng Goto Github PK

I think these are the main bottlenecks:

• Looking up the symbol info in _get_symbol_info. I think we probably want to make a dictionary mapping every possible length/distance directly to the LengthSymbolInfo.
• Searching for backreferences. We should be using the Rabin-Karp algorithm.

I wanted to make slidingwindow support streaming data, but had trouble getting it to be performant. I think this line is the issue:

        outf.write(bytes(itertools.islice(self.deque, self.idx, self.idx + n)))

Deques are not optimized for sequential accesses, so this decimates performance. The solution would probably be to have a bytearray and be smart about moving data in and out in sufficiently large chunks.

We should either implement this, or remove the slow slidingwindow.