Current status of this project? about elasticsearch-hadoop

[utsengar]

Also wanted to let you know, I am getting a performance of 25,000 documents (with 13 columns)/min. There were 6 mappers allocated to the job and I am running pig 0.9.1

I get a better performance by accessing the cassandra data via a river, details: https://groups.google.com/forum/?fromgroups=#!topic/hector-users/OzAN3ji_gec

[costin]

@utkarsh2012 Thanks for the kind words. You can use the issue list or use the ElasticSearch group to post question (will update the home page to better reflect this).

Since it's a young project there are still a couple of things to iron out first before thinking about an official production release. However we do work on pushing out nightly builds as well as milestones.

Note that production wise, the data is stored within ES - meaning while this project can improve the read/write to/from ES, once within ES, your data is handled by the ES production policy (i.e. if you use a stable version, things should be okay).

Time-wise, I don't want to give any ETAs yet since there are still some critical design points that I'd like to get right first before doing estimates - otherwise I'm just guessing which is bad for everyone involved :)

[costin]

@utkarsh2012 by the way, regarding performance there's a lot when can improve on. Currently the code uses only 1 mapper/reducer task since the main aim is to handle the pig/hive integration properly. This will be changed shortly to be:

1. pluggable
2. use proper parallel querying/insertion
3. use an improved json/object serialization

This should improve performance a lot.

[utsengar]

Sounds good. Will be closely watching the development of this project :)