Issues with 10s of thousands of contigs about pybigwig

[dpryan79]

In fact, the issue is that nperblock is set incorrectly with > 65535 contigs!

[dpryan79]

In general, writing doesn't work properly when a non-leaf node for chromosomes is needed.

[dpryan79]

The good new is that there can't be more than 4 billion contigs (uin32_t), so a single non-leaf node of size 0xFFFF suffices and we don't need to do multiple levels. The issue was essentially that nperblock was the wrong size, so setting it to -1 was producing an illegal result and the iterator overflowed. In addition, the non-leaf node was never properly constructed to begin with. The modifications should really be tested more, but at least in theory this should fix the issue.

[dpryan79]

This apparently makes deepTools happy, but not IGV.

[dpryan79]

So IGV is broken because java doesn't do unsigned integers, which exist in every other language. I'll halve my maximum values per leaf...

[dpryan79]

The fix36_many_contigs branch should now work in IGV too and generally be a bit more similar to what UCSC is doing (with the exception that I don't use a fixed nperblock of 256, but go up to 0x7FFF). There's currently a limit of ~1 billion contigs, which is a bit artificial when the same library can read up to ~4 billion, but since java-based tools set a limit at ~2 billion contigs (int32_t rather than uint32_t) I guess I'm not alone. I also don't expect anything with more than a billion contigs will ever be created, since it'd be useless to make a bigWig of that.