Error when handling DNG files larger than 256Mb about rawspeed

[klauspost]

Nice find. I will probably modify the "generic" function for this, since it will otherwise be a generic slowdown (keeping things in level 1 cache is critical)

[LibRaw]

I cannot understand your statement.
maximum slices count is 15 (so, 4 bit), offsets[] table is slices+1 in size;
So, for 32-bit offsets, the table will fit in 64 bytes (16*4), for 64-bit offsets - in 128 bytes.So, 2 or 4 cache lines.
L1 cache is about 500 times larger.

[LibRaw]

I've changed offset[] to offset64[] for easy refactoring (so, compiler warns until all places was fixed).
Here is my patch: https://www.dropbox.com/s/i64gott37ekx791/offset64.patch?dl=0
Sorry for do not use github pull mechanics, my repo is not synced with github.

Maximum data offset is now 32bit, this is enough for 2GPix bayer images and for 0.5-0.7Gpix 4/3-component linear dngs. Enough if we want to keep RawSpeed 32-bit compatible.

[klauspost]

See PR #52 ... If that works for you, I will merge it.

[klauspost]

The reason for not using 64 bits is that 64 bit shifts are very slow on 32 bit platforms.

I don't see any reason to slow down 99% of all decodes, so we handle this situation separately.

It still decodes at ~80MP/sec on my 5 year old Q6600 system.

[LibRaw]

Offset calculation is called once per slice. With maximum 16 slices per image, the speed difference is not measureable.

[LibRaw]

#52: unable to check right now, I use my own 64-bit offset code. Sometimes in future I'll rebase to current rawspeed from Github. Anyway, I'll wait until development branch will be merged to master.

[klauspost]

No, it is used "slices" times per line, see:

uint32 slices = (uint32)slicesW.size() * (frame.h - skipY);

In all DNG's, that is 1 per line (slices are only a CR2 concept).

[klauspost]

I have merged code, that uses the generic decoder, if 28 bits for offsets isn't enough.