jdmaturen / javafastpfor Goto Github PK

JavaFastPFOR: A simple integer compression library in Java
Main contributors
Daniel Lemire, http://lemire.me/en/

Minor contributors
Di Wu, http://www.facebook.com/diwu1989

== What does this do  ==

It is a library to compress and uncompress arrays of integers 
very fast. The assumption is that most (but not all) values in
your array use less than 32 bits. These sort of arrays often come up
when using differential coding.


Some CODECs ("integrated codecs") assume that the integers are
in sorted orders. Most others do not.

== Why? ==

I found a few libraries that implemented Binary Packing, NewPFD, OptPFD,
Variable Byte, Simple 9 and so on in Java. However, I could not find
one that I liked.

I threw in a cool little benchmark program.

== How does it compare to the Kamikaze PForDelta library? ==

In our tests, Kamikaze PForDelta does not fare well. See
the benchmarkresults directory for some results. 


== License ==

This code is released under the
Apache License Version 2.0 http://www.apache.org/licenses/.

== Requirements ==

A recent Java compiler.

== What can I do? ==

Compile the code and execute integercompression.benchmark.

I recommend running all the benchmarks with the "-server" flag.

Speed is always reported in millions of integers per second.


=== For Maven users ===

mvn exec:java

=== For ant users ===

When you use Apache ant, please try this:

    $ ant Benchmark

or:

    $ ant Benchmark -Dbenchmark.target=BenchmarkBitPacking


== Usage ==

See example.java for a simple demonstration.

== what to read more? ==

We wrote a research paper which documents many of the CODECs implemented here:

Daniel Lemire and Leonid Boytsov, Decoding billions of integers per second through vectorization, Software Pratice & Experience (to appear)
http://arxiv.org/abs/1209.2137




== Limitation ==

Currently this library compresses and uncompresses arrays in bulk.
An alternative strategy is to compress them block-by-block.

== Installing java 7 (optional) ==

Good instructions:
http://forums.linuxmint.com/viewtopic.php?f=42&t=93052