Buffered atomic increments aren't guaranteed to be flushed on time about asynchbase

[tsuna]

I added an integration test to reproduce this problem. Note that shutdown() always calls flush(), and your workaround just happens to work by chance.

The real issue is that flush() kicks off flushBufferedIncrements() but doesn't tie it to the Deferred it returns. So when you .join() on either shutdown() or flush(), the call may return before all the increments are actually flushed to HBase (edit: this is false, see 3 comments below). This is non-trivial to fix due to how the increments get flushed, I'm looking for a solution right now – the good thing is that there is now a test for this case.

[tsuna]

For reference the test case I added is at tsuna/asynchbase@addcea8 – it wasn't originally supposed to test this bug, but it turns out that it does.

[shrijeet]

👍 Good stuff tsuna. Glad to see the tests coming through.

[tsuna]

OK, so I understand where the problem is. flush() correctly triggers a flush of buffered counter increments, but they aren't tied to the Deferred it returns.

What I said earlier was incorrect: "when you .join() on either shutdown() or flush(), the call may return before all the increments are actually flushed to HBase" – this is false. shutdown() properly enforces that all in-flight RPCs complete before shutting down.

Really the only issue here is that the way we flush buffered counter increments makes it hard to tie them to a Deferred.

[tsuna]

I see how to fix this: we either make flush() wait on all outstanding RPCs, or we add a new public API sync() that does that (that would be analogous to Linux's sync() system call).

I started a discussion on the mailing list asking for input.

Ideally flush() would be able to gain access to the Deferreds of the buffered atomic increments, however due to the way increment coalescing is implemented with a Guava cache, this is rather tricky to implement and I don't see a more elegant solution than the one proposed above.

[tsuna]

Have a proposed fix in tsuna/asynchbase@00598d0