Implementing SuperCDC 2022 about fastcdc-rs

[nlfiedler]

There are a couple of reasons why SuperCDC would not make sense in this crate. First the name is different, so that's a no-go right off the bat. Second, FastCDC is suited only to chunking a single stream of bytes, while SuperCDC and its ilk are about managing whole systems comprised of many files. These solutions employ additional storage to track chunks (and files) in order to predict the size of the upcoming chunks in the hope of speeding up the process slightly. That's really neat, but it's far more complex and grander in scheme than the modest and simple FastCDC. Implementing SuperCDC involves making decisions that are better suited to an application rather than a library. It's either that or the library would have to offer several options for tuning the performance vs accuracy dial to suit the application.