Make it more robust and intelligent about horizon

[krixano]

You can make the search engine appear like it's going faster by, on a first query, limiting to the amount you want to show on the first page, then immediately doing another query for the next page - still with a limit. Then, when they click the next page, immediately show those results and query for the next page.

One thing you should do with this though, is to make sure you do your sorting in sql rather than javascript that way the sorting happens for everything, before the limiting, rather than per page.

[blurHY]

@krixano
Merge pages that is the same zite.I think it could be done with group_concat,but we need to find a separator.If it can return a array for each item,the better.

[blurHY]

I just do some complex things with javascript after sql query,but i won't use js to sort or filter 5 million data 😂
https://github.com/blurHY/Horizon/projects/1#card-13092259

[krixano]

Btw, in the code for Important Zites (and KxoNetwork), I have code that constructs an sql query to query for zites based on relevance by seeing how many words in the query match whatever is in the database, as well as putting priority over certain fields. If you open the console on Important Zites or KxoNetwork while searching, you can see this.

It could perhaps be useful to you, although, I think your search query could probably be simpler than mine since you are searching from selected keywords, rather than, with mine, I'm searching through all text.

I'll link you to the exact code when I find the time, but I did post some ZeroMe posts about this a while ago, so you may be able to find it on my ZeroMe profile.

[blurHY]

I used to write some title rank * 2 too.
And I'm going to implement PageRank algorithm for ranking.The site descriptions text data has been already published.So there is a lot of things need to be calculated for the final score.
Uh,i don't know it should be title rank * 2 or title rank * 1.5 or something else.

[blurHY]

Search 'b'
Takes 4.619s returns 73918 rows
Takes 3.743s with limit 70000,3000
Takes 3.312s with limit 10,3000
Maybe it will not be much quicker to query.

[krixano]

Did you try limiting to 10 or 20 or so (that's what I was meaning above, since only 10-20 will be shown on a page at a time)? Also, a half a second minimum is definitely significant imo, especially for search.

[blurHY]

It filter the database then order the result.If we just limit it,the result won't be the best match.Or we still need to scan the whole database

[krixano]

If you do sorting/ordering in SQL, then that happens before limiting.

[blurHY]

So it's hard to improve.But if the new plugin system has been implemented and zeronet has moved to py3,it would be possible.

[krixano]

What things are you doing in js that you can't do in SQL currently?