Support for List of Maps (Nested objects) about elasticsearch-hadoop

[costin]

Hmm - I think there's a problem with your data set. You're using a map but have the same key declared twice which is actually illegal (it's not a valid JSON object).
What happens is that the JSON Object gets converted to a map in which the entries are added one by one but since the key is the same, each entry causes an update instead of insert.

[abhishek376]

Costin,

Thanks that makes sense.

I will hard code map from hive map type [{1 : 3}, {4: 5}] to ES Query String
[
{
"mapid": "1",
"value": "3"
},
{
"mapid": "4",
"value": "5"
}
]

in BufferedRestClient.

Thanks for the support.

[costin]

Instead of hard-coding why not apply some transformation to the data, such as converting the map to a bag/list? Note that a wrong JSON implies the data might be properly represented in ES which can trigger issues when doing queries.
Note that while functionally we could support a map with duplicates (using identity instead of the hash code), the problem of duplicates still persist and is likely to cause issues later down the road.

[abhishek376]

Makes sense thanks costin.

[costin]

@abhishek376 I'm closing this issue as the problem seems to be fixed. Note that mapping is part of the roadmap to cope with the different data structures however it will take a bit until I get to it.
Watch this project for future updates.