Elasticsearch Bulk API Output error handeling issues about lua_sandbox_extensions

[trink]

1. IIRC it was by design; the thought was we would cache and not block the pipeline while elastic search was unavailable. If it never comes back there aren't a lot of options you could terminate the plugin and optionally stop Hindsight. An infinite retry would eventually backpressure the entire system and the reason for not discarding is because that is how it was configured.

2. No the retry applies to the message already added to the batch so no data will be lost when it resumes.

3. Only a failure return (-1) will log a warning

4. Correct there isn't any mechanism to feed that information back at the moment.

5. If you have set 'preserve_data = true' set then batch_count is restored along with the retry state

[trink]

I think in this case the best thing to do would be add a termination option for the case you don't want the batch file to keep growing and you don't want to discard data. You would simply retry X times, terminate the plugin (based on your configuration you could even have that cause HS to shutdown) and then restart HS when elastic search is back on line. For the second option you could simply have retry X == 0 mean infinite retry where the data will build up in the HS queues until it backpressures due to limited disk space or elastic search comes back on line and it can be drained down in batch size chunks.

[champtar]

Hi,

1. I'll try to add some options
2. I was off by one :), the message that we use for the retry is already in the batch, so we don't drop it ...
   3)4) #89 i guess
3. didn't know about this one, nice

Thanks