Performing search CPU spikes about lifti

[markwriter]

Hi, I discovered that I may have caused the problem by loading the thesaurus incorrectly. Currently I have about 50 items in it and when I comment out all but a few of them the CPU spiking goes away (and that fantastic Lifti-speed returns!). I will take a bit of time to figure out which item(s) are the culprit and get back to you.

[markwriter]

Sorry!! I never was able to test it long enough to figure out why the slowdown occurred. I would like to get back to it.
Also, I didn't mention that we finally went to production with this and you have made some users extremely happy with 1 second search times compared to the 10 seconds that it was previously.

[mikegoatly]

Are the queries that are being run the same? I would expect different operators to exhibit different cpu cost, so if the queries aren't identical then that could be part of it.

[mikegoatly]

Assuming that what you're experiencing isn't a bug in LIFTI, the most likely cause will be due to an increased number of matches that need to be merged in the case of a query containing lots of search terms, or there is a wildcard in the query that's becoming computationally expensive to run.

Let me know what you find out - if there's a concrete example I can work with I'd be happy to find some time to take a look.

[mikegoatly]

@markwriter I'm interested to know if you're able to share where you got to with this?

[mikegoatly]

That's great to hear, @markwriter. I'll keep this issue open for a while in case you get chance to look into it any more.