how to disable the round robin behavior about elasticsearch-rails

[karmi]

Hmm, this is weird. Also, it doesn't have any connection to "round robin", since the clients are separate, it's not the same instance.

So, I have generated a fresh Rails app with the template, and added two models:

# article.rb
#
class Article < ActiveRecord::Base
  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks

  __elasticsearch__.client.transport.logger = Logger.new(STDERR)
end

# book.rb
#
class Book < ActiveRecord::Base
  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks

  __elasticsearch__.client = Elasticsearch::Client.new host: 'localhost:9250'
  __elasticsearch__.client.transport.logger = Logger.new(STDERR)
end

Now, I've started a separate cluster elasticsearch --cluster.name=books --node-name=book1 --http.port=9250 and when I run bundle exec rails console:

>> Article.__elasticsearch__.create_index!
F, [2014-03-17T11:36:59.417716 #49843] FATAL -- : [404] 
I, [2014-03-17T11:36:59.548716 #49843]  INFO -- : PUT http://localhost:9200/articles [status:200, request:0.128s, query:n/a]
=> {"acknowledged"=>true}
>> Book.__elasticsearch__.create_index!
F, [2014-03-17T11:37:05.342066 #49843] FATAL -- : [404] 
I, [2014-03-17T11:37:06.609309 #49843]  INFO -- : PUT http://localhost:9250/books [status:200, request:1.267s, query:n/a]
=> {"acknowledged"=>true}

So each model talks to a different ES cluster. How are you configuring the models?

[cleesmith]

In config/initializers/loader.rb I do this:
(1) to cover most models:

 Elasticsearch::Model.client = Elasticsearch::Client.new(host: 'localhost:9200')

(2) for the remote cluster model:

Share.__elasticsearch__.client = Elasticsearch::Client.new(host: "1.2.3.4:9500")

Is "(1)" causing the round robin ?
Should I move "(2)" into the model and out of the initializer ?
As the Share (remote cluster) model requires a template to be present (or put there),
it seemed like an initializer was the best location for that code.
It is strange ... and for several records it goes to the correct cluster, but then some
will go to the local cluster.
Also, when I do ".transport.inspect" I can see both IP's and port's in the output, and
the output is rather lengthly ... is there a way to just display the IP/port that the
client is using ?
I do love the gem, it's great, and I'm probably doing something wrong here.

[karmi]

So, first let me summarize what I have on my part.

In the config/initializers/elasticsearch.rb I have:

Elasticsearch::Model.client = Elasticsearch::Client.new(host: 'localhost:9200')
Elasticsearch::Model.client.transport.logger = Logger.new(STDERR)

Book.__elasticsearch__.client = Elasticsearch::Client.new host: 'localhost:9250'
Book.__elasticsearch__.client.transport.logger = Logger.new(STDERR)

And when I run this in rails console:

>> Article.__elasticsearch__.create_index!
I, [2014-03-17T17:06:06.097307 #52199]  INFO -- : HEAD http://localhost:9200/articles [status:200, request:0.031s, query:n/a]
D, [2014-03-17T17:06:06.097670 #52199] DEBUG -- : < 
=> nil
>> Book.__elasticsearch__.create_index!
I, [2014-03-17T17:06:09.030384 #52199]  INFO -- : HEAD http://localhost:9250/books [status:200, request:0.009s, query:n/a]
D, [2014-03-17T17:06:09.030477 #52199] DEBUG -- : < 
=> nil
>> Article.__elasticsearch__.client.transport.connections.map(&:host)
=> [{:host=>"localhost", :port=>"9200", :protocol=>"http"}]
>> Book.__elasticsearch__.client.transport.connections.map(&:host)
=> [{:host=>"localhost", :port=>"9250", :protocol=>"http"}]

[karmi]

To adress the other issues:

• The round robin rotation across nodes is only happenning when you a) actively pass the nodes to the client, b) tell it to "sniff" them from Nodes Info API
• I assume localhost:9200 and 1.2.3.4:9500 not running as part of the same cluster? Since I cannot guess the reason why you'd like to talk to a specific node within a cluster, for this model.

So, something is definitely mixed up here. Can you generate a Rails app with the linked template, add the second model, the initializer, and try it out? So we can start isolating the issue?

[cleesmith]

Hmm, that's exactly what I have in the initializer.
My issue isn't only creating the index, but as records are created in the database and
mirrored into the elasticsearch index some of them are indexed into the local
cluster and some of them are indexed into the remote cluster. As this rails app was an existing app that I'm converting to using elasticsearch, I did not start by using the rails app template.

No, localhost:9200 and 1.2.3.4:9500 are not a part of the same cluster. The reason one model
needs to use a different cluster (1.2.3.4:9500, not it's real IP) is to allow users to share certain
objects with each other. Basically the remote cluster at 1.2.3.4:9500 is a central location to
hold all share objects, so other installed instances of the same rails app can see the shares.
I hope my explanation is clear and helpful. I will experiment some more and get back to you,
and if the problem persists I will create an example rails app to emulate the issue.

[karmi]

Thanks for the explanation, all understood on the separate nodes now.

The client should be definitely separated, no mixup between them. Try to generate the app with two models, and replicate what you're doing there, so we get an isolated, shared environment.

[cleesmith]

I left the initializer as is, but putting this into the Share (remote) model:

  __elasticsearch__.client = Elasticsearch::Client.new(host: "1.2.3.4:9500")

after:

  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks
  index_name    "#{APP_CONFIG[:host]}_shares"
  document_type "share"

... seems to fix the issue and all records for the Share model go to the remote cluster.
It's just odd to specify the client connection info again in the model when it's already in
the initializer.

Thanks ... the gem is great, plus having the "as_indexed_json" method in the model
is very helpful.

[karmi]

Hmm, really, you shouldn't need to do that -- placing it into the initializer should have the same effect, I verified it with the example I posted...

[cleesmith]

I'm happy it's working :-) You're example just verified an index was created on the correct cluster, but within an app where a user can "bounce" around between different models while adding/updating/deleting records ... I'm not sure but perhaps that makes a difference somehow ... not a very well tested or scientific answer. If I notice the behavior again I will open an issue.
Thanks again for your prompt and helpful responses.

[karmi]

Great, open a new issue if you encounter further issues!