Try caching the query vector's FloatVector segments when computing distance about elastiknn

[alexklibisz]

I took a first pass at implementing this in #605. The benchmark looks promising on my M1 Macbook, but not so much on my EC2 r6i.8xlarge instance.

On M1 Macbook:

[info] Benchmark                            Mode  Cnt        Score       Error  Units
[info] CachedL2DistanceBenchmark.baseline  thrpt    6  2584985.052 ±  9945.960  ops/s
[info] CachedL2DistanceBenchmark.cached    thrpt    6  4461915.029 ± 43007.018  ops/s

On EC2 r6i.8xlarge:

[info] Benchmark                            Mode  Cnt        Score        Error  Units
[info] CachedL2DistanceBenchmark.baseline  thrpt    6  2939742.574 ± 273069.456  ops/s
[info] CachedL2DistanceBenchmark.cached    thrpt    6  2698700.860 ±  13063.707  ops/s

It seems on the M1, we actually benefit from pre-computing and fetching the chunks from memory. Not so much on standard x86 architecture on the EC2 instance.

[alexklibisz]

Closing as I'm skeptical that this would work.