Comments (9)
Yes, you have to set this up in the mapping
block (or, generally speaking, in the mapping for the index), have a look at documentation and an example.
You will have to re-index all the data, if it's a development setup, the easiest way is to re-import it (Rake task or the import
method).
from elasticsearch-rails.
Thank you for your fast reply!
I guess what I am asking is that I am not sure how to do this with Elasticsearch::Model
I have added it to the mappings like so:
settings index: { number_of_shards: 1, number_of_replicas: 0 },
analysis: {
filter: {
synonym: {
type: "synonym",
synonyms: File.readlines(Rails.root.join("config", "analysis", "data"), "synonym.txt")).map(&:chomp)
}
},
analyzer: {
synonym: {
tokenizer: "whitespace",
filter: ["synonym"]
}
}
} do
mapping do
indexes :my_field, analyzer: 'synonym'
end
end
But does not seem to work so I think I might have done this wrong?
from elasticsearch-rails.
I'll need more info than "it does not seem to work" I'm afraid :)
So, can you configure the synonyms inline (not by external file), and post the config? Are you sure you have re-created the index with correct mappings? (Check localhost:9200/MYINDEX/_mapping
or MyModel.mappings.to_hash) How do you verify whether it "works", by searching, or by using the analyze API?
from elasticsearch-rails.
Ok, sorry, will try to be more elaborate :)
I have this mapping in my searchable model (i excluded other fields where i do not need the synonyms for the sake of readability here):
settings index: { number_of_shards: 1, number_of_replicas: 0 },
analysis: {
filter: {
synonym: {
type: "synonym",
synonyms:[
"developer, programmer, hacker, web developer, coder, sofwtare developer, software engineer",
"designer, web designer, graphic",
"ux, user experience"
]
}
},
analyzer: {
synonym: {
tokenizer: "whitespace",
filter: ["synonym"]
}
}
} do
mapping do
indexes :i_am_role, analyzer: 'synonym'
end
end
Then i recreated the index and this is what i get if i inspect it with Marvel:
GET /my_application/_mapping/
{
"my_application": {
"mappings": {
"profile": {
"properties": {
"i_am_role": {
"type": "string",
"analyzer": "synonym"
}
}
}
}
}
}
I am verifying by searching through Marvel. I have 7 models indexed and 5 of them have the field set to be either Developer, developer, or Programmer
But when i do the search, only the record that has "developer" is returned. I am doing something wrong, I am sure. I just can't spot it :)
from elasticsearch-rails.
uhm.. i just tried it with the API and i think the synonym analyzer is not properly loaded or something because i get an error:
Profile.__elasticsearch__.client.indices.analyze text: 'programmer', analyzer: 'synonym'
Elasticsearch::Transport::Transport::Errors::BadRequest: [400] {"error":"ElasticsearchIllegalArgumentException[failed to find analyzer [synonym]]","status":400}
from /Users/tanja/.rvm/gems/ruby-2.1.1@headhunted/bundler/gems/elasticsearch-ruby-e4794be90094/elasticsearch-transport/lib/elasticsearch/transport/transport/base.rb:132:in `__raise_transport_error'
from elasticsearch-rails.
You forgot to create the index with proper settings and mappings, I think -- Profile.__elasticsearch__.create_index! force: true
Working code:
require 'pry'
require 'logger'
require 'ansi/core'
require 'active_record'
require 'active_support/core_ext/numeric'
require 'active_support/core_ext/hash'
require 'elasticsearch/model'
ActiveRecord::Base.logger = ActiveSupport::Logger.new(STDOUT)
ActiveRecord::Base.establish_connection( adapter: 'sqlite3', database: ":memory:" )
ActiveRecord::Schema.define(version: 1) do
create_table :people do |t|
t.string :name
t.string :occupation
t.timestamps
end
end
Elasticsearch::Model.client= Elasticsearch::Client.new log: true
class Person < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
settings index: { number_of_shards: 1, number_of_replicas: 0 },
analysis: {
filter: {
synonym: {
type: "synonym",
synonyms:[
"developer, programmer, hacker, web developer, coder, sofwtare developer, software engineer",
"designer, web designer, graphic",
"ux, user experience"
]
}
},
analyzer: {
synonym: {
tokenizer: "whitespace",
filter: ["synonym"]
}
}
} do
mapping do
indexes :name
indexes :occupation, analyzer: 'synonym'
end
end
end
Person.__elasticsearch__.create_index! force: true
# Store data
#
Person.delete_all
Person.create name: 'John', occupation: 'developer'
Person.__elasticsearch__.refresh_index!
puts '', '-'*Pry::Terminal.width!
p Person.__elasticsearch__.client.indices.analyze(index: 'people', text: 'developer', analyzer: 'synonym')['tokens'].map { |d| d['token'] }
# => ["developer",
# "programmer",
# "hacker",
# "web",
# "coder",
# "sofwtare",
# "software",
# "developer",
# "developer",
# "engineer"]
p Person.search('occupation:hacker').to_a.first.name
# => "John"
binding.pry;
from elasticsearch-rails.
Uhm, yeah. That is weird. I tried everything again, and now it works.. I must have forgotten to reindex, although i could swear i did it.
Sorry about that. Thanks for looking at it! Closing this because it is working now.
this is my final settings and mappings:
settings index: { number_of_shards: 1, number_of_replicas: 0 },
analysis: {
filter: {
synonym: {
type: "synonym",
ignore_case: true,
synonyms:[
"developer,programmer,hacker,coder,software,engineer",
"designer,web designer,graphic",
"ux,user experience",
"copywriter,writer,blogger,journalist"
]
}
},
analyzer: {
synonym: {
tokenizer: "whitespace",
filter: ["synonym", "lowercase", "stop", "snowball"]
}
}
} do
mapping do ....
👍 :)
from elasticsearch-rails.
Great!, glad it's working :)
from elasticsearch-rails.
Hi guys. I'm not sure if this should be a new issue, or remain on this thread, but I'll mention it here and go from there.
In the example above, multi-word / phrase synonyms are being treated as single terms ("web developer" ~> "web", "developer"). How would we incorporate / treat phrase synonyms? The docs mention contraction, but don't give code / json examples of how to incorporate these.
from elasticsearch-rails.
Related Issues (20)
- NPE happens on short prefix query on a field with index_prefixes
- Indexes are not refreshing, even with refresh_interval being set
- Disable automatic callbacks in rspec
- Not getting any records from elasticsearch HOT 1
- FYI gem version 6.1.1 was pushed without any changes HOT 1
- elasticsearch-model 7.2.1 introduces circular dependency with ruby 2.7 and rubygems 3.2.5 HOT 1
- specify an alias near the index_name in an AR model HOT 1
- Illegal argument "type" sent in all requests upgrading to 8.x HOT 1
- Not able to update single record too please help me to here HOT 1
- How to import into two clusters
- Mapping Two Ruby on Rails Models with Different Attribute Names to a Shared Elasticsearch Index using 'elasticsearch-rails'
- update_document not selective enough
- Backward compatibility for older Elasticsearch version HOT 1
- Faraday::ConnectionFailed (end of file reached) HOT 1
- Elasticsearch - Unexpected Character Error (Minus Sign) HOT 1
- Unexpected behavior when searching from a scope
- Feature Suggestion - Get Indexed Document for Instance
- Deprecation warning in Rails 7.1 (support Rails 7.2) HOT 2
- ElasticSearch::Model does not register a subclass HOT 3
- rake elasticsearch:import:model - ArgumentError: unknown keyword: :type (ArgumentError)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elasticsearch-rails.