Memory leak about ruby-xz

[Quintus]

Hmm. I'll take a look later, but for now: in your loop, you are creating the string "Lorem Ipsum ..." again each time the loop iterates. Please try again with the input string allocation outside the loop (maybe even outside the memory measurement block), e.g.: data = "Lorem Ipsum is simply ..." iterations.times do compressed = XZ.compress(data) XZ.decompress(compressed) end XZ#compress and XZ#decompress use Ruby's StringIO under the hood. It's ::new method makes a copy of its argument, thus there should be no need to create a new string for each iteration. I vaguely remember that StringIO has some memory problems, but I will have to look that up. You could also try with the XZ#compress_stream and XZ#decompress_stream methods, which are the workhouse methods that do the actual calls to libxz. If the memory leak appears in those methods, we can exclude StringIO as the cause. -quintus

…

-- Dipl.-Jur. M. Gülker | https://mg.guelker.eu | For security: Passau, Germany | [email protected] | () Avoid HTML e-mail European Union | PGP: see homepage | /\ http://asciiribbon.org

[xTRiM]

@Quintus, I've updated the gist to use streaming ops, there's clearly a leak in both compression and decompression.
https://gist.github.com/xTRiM/afdd9ea25519a479672153637637f0f7

Here's the output:

[Compress run with 10000 iterations]
Before - Process Memory: 179.988 mb
Before - Objects count: 581026
Before - Symbols count: 11449
After - Process Memory: 269.4609375 mb, changed by 89.473 mb
After - Objects count: 580998, changed by 25
After - Symbols count: 11449, changed by 0

[Decompress run with 10000 iterations]
Before - Process Memory: 269.559 mb
Before - Objects count: 580999
Before - Symbols count: 11449
After - Process Memory: 350.0859375 mb, changed by 80.527 mb
After - Objects count: 581001, changed by -5
After - Symbols count: 11449, changed by 0

[Compress/decompress run with 10000 iterations]
Before - Process Memory: 350.105 mb
Before - Objects count: 580997
Before - Symbols count: 11449
After - Process Memory: 511.17578125 mb, changed by 161.071 mb
After - Objects count: 580998, changed by -4
After - Symbols count: 11449, changed by 0

[Quintus]

I have decided to abaondon this project due to lack of time to maintain it, and I don’t use it anymore. Hence, I close this issue now. Please fork if you want to continue maintaining it.