richarddurbin / modimizer Goto Github PK

hi @richarddurbin, @luizirber pointed me at this repo and I wanted to drop you a note --

we have been using an analogous concept in sourmash, that is identical in concept to modulo hash but uses a slightly different technical approach. (modulo hash was coined by Broder in his 1997 paper on MinHash; see also the mash screen blog post from Adam Phillippy.)

briefly, in sourmash we choose a max_hash value below which all hash values are kept; this max_hash is the inverse of the density, which we refer to as the scaled value. (max_hash = 2**64/scaled) This gives us slightly more options for resolution than modulo hash, and also has the advantage of being interconvertible with mash-style MinHash approaches.

we have this set of notes on the advantage of modulo hash, as well as some API documentation that goes further into depth on interconvertibility.

our code is not so pretty but here is the max_hash implementation.

we have a draft f1000research software paper nearly written that I'd be happy to send you if you want to see more polished prose :)

happy to discuss more!

other than saying hi, the main purpose of this issue is to recommend that you investigate (or at least support) mash compatibility by using murmurhash hashing with a seed of 42 and reverse complement handling. see our example Python code here. murmurhash is less efficient than rolling hash but on the other hand it's nice to be able to interconvert between these efforts!