FAISS score filtering is done twice with and without applied relevance_fn about langchain

[habanoz]

@ozgurdemir I have been debugging sth. else and found your issue here. A quick question. Are you sure the filtering is applied twice?

The first line in "VectorStore.similarity_search_with_relevance_scores" removes "score_threshold".
score_threshold = kwargs.pop("score_threshold", None)

This should prevent "FAISS.similarity_search_with_score_by_vector" (no preceding underscore here) from applying filtering...

Just wanted to help.
Regards

[ozgurdemir]

Hey @habanoz ,

thx for your reply. I'm not working on this particular project anymore. I remember that we stumbled on this as we had a weird behaviour when using the relevance_score_fn.

However, looking at the code again it seems like you're right.
VectorStoreRetreiver calls the VectorStore.similarity_search_with_relevance_scores which removes the score_threshold parameter and then calls _similarity_search_with_relevance_scores without this parameter. So the filtering step should be circumvented.

In other words the relevance_score_fn is applied to the returned scores but the filtering happens on the raw scores.
All in all it seems like a very confusing implementation and probably not what is intended. The filtering by score depends on the similarity_metric used. So it should be either smaller_than or greater_than depending on the similarity_metric but this code path is not executed due to the above mentioned logic.

So I still think there's a bug here :(