Faiss indexing assertion about colbert

[okhat]

That's correct: the assertion was being triggered on every call of _flush_to_cpu; it should be after the last _flush_to_cpu only. (This issue doesn't show when _flush_to_cpu is getting called only once, which happens to be my setup.)

Just pushed a relaxation of the assertion. It should work for you now.

[okhat]

For FAISS indexing, here's a sample command for MS MARCO:

python -m colbert.index_faiss --index_root /root/to/indexes/ --index_name IndexName --root /root/to/experiments --experiment ExperimentName --run OptionalRunName --partitions 65536 --sample 0.3

You can probably remove --sample 0.3 which will use a default (0.05). This will be faster and probably lose no effectiveness (unless you increase the partitions further).

If this run cannot use faiss-gpu for any reason, you can also reduce --partitions to just 16384. This will be fast to index on CPU. It will make retrieval slightly slower, so it's a tradeoff. (I just realized this is not relevant to you here, since the error above is only for faiss-gpu. In this case, you can use 65536 or even twice or 4x that quickly enough during indexing.)

The original runs in the paper used just 2000 partitions. Using up to 200k can yield much faster retrieval (much faster than the paper's results too) but you pay the cost upfront during faiss indexing. This is not an issue at all when indexing with faiss-gpu, though.

If you use a large number of partitions, I recommend --nprobe 16 or --nprobe 32 during retrieval. Otherwise, --nprobe 10 is more than enough for small partition numbers.

Let me know if any other parameters are unclear. I will eventually move those to their own set of instructions.

[cmacdonald]

Thanks Omar. Looking forward to seeing you tomorrow!

[okhat]

Looking forward to seeing you tomorrow as well!

[okhat]

Seems resolved. Closing but feel free to re-open if needed.