Query not using IVFFLAT index about pgvector

[xfalcox]

After going into a rabbit hole, this is caused by an abnormal high ivfflat.probes parameter.

I automated index creation at

https://github.com/discourse/discourse-ai/blob/main/lib/embeddings/vector_representations/base.rb#L23-L108

but since that we have also added another table for post embeddings. There are many more posts than topics, and the automation was setting probes based on the number of posts globally.

In the end this is yet another point for #235, which would make the ergonomics much simpler to reason about.

[jkatz]

@xfalcox Curious -- is there a reason you went with IVFFLAT instead of HNSW?

[xfalcox]

Well, @jkatz, HNSW wasn't a thing when I started doing this, I think most of my databases both in-house and in RDS are below pgvector 0.5 still.

I'd like to experiment with it, but with all the moving pieces, we running embeddings backfill and indexing automatically on many instances, with some hosted by us and some hosted by people running our software which may not be as technical, I need something that is as hands-free and safe as possible.

[jkatz]

@xfalcox Thanks for the explanation, that's really helpful! One nice thing about HNSW is that it's a bit easier to maintain, because it supports iterative inserts/updates, so you don't have to have the dataset pre-loaded like with IVFFLAT. From observations and talking to users, HNSW tends to be easier for folks who are less familiar with working with approximate nearest neighbor indexing.

Amazon RDS does support pgvector 0.5 and above since Oct 23 (Disclosure, I work at AWS).

Hope this helps. I'm also happy to hear more about your experience working with pgvector.