Search about hackweek

[jordimassaguerpla]

I'd love to work on this. Please assign it to me :)

[jordimassaguerpla]

We have a simple search in place now that use solr. However, there are still 2 issues to fix:

• substring search does not work. For example, searching for node does not match nodejs.
• we should have a monitoring process since if solr were not working, any insert into the db would create an exception.

[jordimassaguerpla]

a part from substring, it would be great to do other advanced stuff like github does with the limiter, for example:

if q is @user, it searches only users

[jordimassaguerpla]

for substring

#57

[jordimassaguerpla]

Regarding the parser. The default parser includes operators like "+/-" and "NOT", as well as phrases (use " ). However it does not include extended search capabilities.

[vpereira]

I'm late on that ticket, but wouldn't be better to go with ElasticSearch? One advantage, would be to be able to import the wiki files directly, without parse and convert to csv and maybe some realtime search function..

[jordimassaguerpla]

You are right: you are late :-)

Why don't you send us a pull request ;-) ?

[vpereira]

hm, as a wise woman once said:

ich habe leider kein foto pull request für dich :)

the question is if the pull request would be accepted :) because actually, adding the gem tire, would be basically to remove the searchable block and add on the model project ( warning untested code) :

include Tire::Model::Search
include Tire::Model::Callbacks

mapping do
    indexes :title, analyzer => 'snowball', :boost => 100
    indexes :description, :analyzer => 'snowball'
    indexes :created_at, :type => 'date'
end

def self.search(params)
  tire.search(load: true) do
    query { string params[:q], default_operator: "AND" } if params[:q].present?
    sort { by :created_at, 'desc' }
end

I just stumbled on that, cause I was kind of playing with the tools (you can see my changes here https://github.com/vpereira/hackweek/blob/master/tools/lib/hackweek_tool.rb and at https://github.com/vpereira/hackweek/blob/master/tools) and maybe the whole wiki conversion before import wouldn't be necessary, cause wikimedia files can be directly indexed using elasticsearch..

[jordimassaguerpla]

Hi! I wanted to finish the search this week ( that is why I said you were late :-) ) because it is hackweek in SUSE. I was not aware of the tools and the import wiki. I selected solr because we are using it in another project and so I could look at how was implemented / deployed there, and because I did not find any feature that was in elastic search and not in solr that we require (I was not aware of the import wiki one).

That said, which is your "timeframe" for your features? It is a hackweek project, too? Can we plan to do that on the next hackweek?

[jordimassaguerpla]

By the way, it is not only those changes you mention, but also we should make sure the tests pass:

#61

that travis runs properly: see the before and after scripst at:

https://github.com/SUSE/hackweek/blob/master/.travis.yml

that we update the deploy configuration

https://github.com/SUSE/hackweek/blob/master/config/deploy.rb

and that we setup and configure properly elasticsearch to at least have the same analyzers we have now: lemmer and gram (see the tests to know what does this mean)

[vpereira]

my timeframe is: I must pack my stuffs and run, cause my train living in 45 minutes :)

the test cases, weren't there when I went through this tickets, so that's why I didn't wrote a line about it, sorry. But IMO it shouldn't be tested by unit testing, instead it should be stubbed (opinion is opinion, it doesn't mean that I'm right), mainly because as soon as you call the search method, you are testing solr or tire that is already well tested. For acceptance, maybe make sense, well kind of :)

anyway, maybe for the next hackweek or next week from the hotel room :))

[hennevogel]

This is implemented now...