Lyra recover mechanism can prevent from the JVM to shutdown about lyra

[DimaGolomozy]

Started to debug this.
added to the lyra code in ConnectionHandler a small list to hold all the "new" delegates that been created
in line 225 (ConnectionHandler#createConnection function)
Found that, after a recovery, a new connection is been created (this is fine) but the old one is still "alive" because of the automaticRecovery flag is true in the ConnectionFactory class.
Tried to set it to false, seems to be working, the JVM does shuts down. but not right away.
any idea why?

[michaelklishin]

Because Java client starting with 4.0 enables automatic recovery by default and uses non-daemon threads (also by default). Lyra should disable automatic recovery in the Java client because it performs its own.

[michaelklishin]

@acogoluegnes there is a way to reproduce here. We will look into adding a new property, just like in the Java client.

[acogoluegnes]

There's a static executor service that is never shut down. With a modified version of @DimaGolomozy's test, the JVM shutdown after 1 minute, which is the default value of keep-alive time for the executor service.
Not sure how to fix this without regression.

[acogoluegnes]

My bad, the executor service is shut down in shutdown hook. Chicken-and-egg problem.

[acogoluegnes]

#83 should do part of the job, but #84 is the biggest problem. There may also other cases where the executor service used for connections is not correctly closed.