Relevancy changes in rc0 - are they expected? about meilisearch

[dureuill]

The fix for nested is nice, but the global change looks less convenient to me. So, personally, no strong opinion.

we can have one without the other so it shouldn't be a factor in the decision about the weights when searchableAttributes isn't provided

it's always good to check with @meilisearch/product-team 😊

agreed

[curquiza]

For anything touching the product, even if it's not breaking, it's always good to check with @meilisearch/product-team 😊 Relevancy is central in the product, so at least to ensure they understand the changes 👍

The fix for nested is nice, but the global change looks less convenient to me. So, personally, no strong opinion.

[dureuill]

Hello 👋,

the discrepancies are indeed caused by #4639

• As these tests don't specify the searchableAttributes, before (v1.8.1), the fields would be ordered in decreasing order of relevancy: id (attributeRankingOrderScore=1.0), title (attributeRankingOrderScore=0.857), poster (attributeRankingOrderScore=0.714), overview (attributeRankingOrderScore=0.571), release_date (attributeRankingOrderScore=0.429), languages (attributeRankingOrderScore=0.286), genres (attributeRankingOrderScore=0.143). Now, the fields are no longer ordered (all attributeRankingOrderScore=1.0 regardless of the field).
• As these tests use the default ranking rule order, attributes is ranked above sort
• in v1.8.1, attributeRankingOrderScore was what would break the tie between documents, hence the previous order (it would favor documents where and would appear in title rather than overview)
• in v1.9.0 it is now queryWordDistanceScore that breaks the tie, hence the new order (it favors documents where and appears near the beginning of either title or overview, without a preference for one or the other).

---

Possible fixes:

1. Update the tests to match the new results,
2. Explicitly set searchableAttributes to ["title", "overview"], which should revert the previous behavior and results
3. If the idea of tests (1) and (2) is to test sort, consider changing the default ranking rule to have sort at first (I'm not sure, depends on what the tests want to check)

[curquiza]

No worries about the tests, I will update them accordingly.

---

As these tests don't specify the searchableAttributes, before (v1.8.1), the fields would be ordered in decreasing order of relevancy: id (attributeRankingOrderScore=1.0), title (attributeRankingOrderScore=0.857), poster (attributeRankingOrderScore=0.714), overview (attributeRankingOrderScore=0.571), release_date (attributeRankingOrderScore=0.429), languages (attributeRankingOrderScore=0.286), genres (attributeRankingOrderScore=0.143). Now, the fields are no longer ordered (all attributeRankingOrderScore=1.0 regardless of the field).

This is a huge impact, I did not realize it with the #4639, my bad. Is it something we really want? I mean, it's less convenient for users regarding product perspective (before you entered most of the dataset, with title at top, and it worked), so what do we gain technically?
Did you discuss with @meilisearch/product-team before doing it? (real question, I don't know)

[irevoire]

I mean, it's less convenient for users regarding product perspective (before you entered most of the dataset, with title at top, and it worked), so what do we gain technically?

Huuum, why is it less convenient? It still works, right? It’s just not ordered in the same way, and we didn’t rely on the fact that maybe the ordering of the first time we’ve discovered every field was important, but that was the point.

Did you discuss with @meilisearch/product-team before doing it? (real question, I don't know)

No, I think we were in a call with all the engineers, saying that it’s what we originally wanted to do but couldn’t. Since it was non-breaking and only a relevancy change, we didn’t communicate much about it except for the issue.

Is it something we really want?

No strong opinion on my side; I don't think it’s hard to revert to the previous behavior.

But we should really get @ManyTheFish and @Kerollmops in the discussion because they’re the real relevancy expert here 👀

[dureuill]

Thanks @irevoire for summarizing the situation here

Is it something we really want?

No strong opinion either way, there's a definite benefit to avoiding "automagic" behavior, but I agree that this particular one was useful, and that the change is probably disruptive.

The important parts for me are:

gives us the ability to finally untie the importance (or weight) of each field with their order.

and the fix on the priority of nested fields.

[curquiza]

Closing this issue, decision has been made: the behavior is expected, and we keep it as it is! 🎉