Client fails to recover on full HBase cluster restarts about asynchbase

[tsuna]

I think I finally nailed down this one. It's not a race condition, but a flaw in the logic that handles NSRE. I need to write a unit test to try to reproduce this in a controlled fashion, but I think the problem goes like this:

• One RPC get NSRE'd on some region that's being split
• After a small timeout, an exists probe is sent to check if the region is still NSRE'd.
• The exists probe causes a .META. lookup and its target region is set to be another region, because the target is now a daughter region.
• The probe reaches the RS but the daughter region is also NSRE'd.
• The probe comes back to handleNSRE but the lookup in got_nsre now searches for the daughter's name.
• A new probe is created for the daughter, and the first probe is queued up.
• The new probe doesn't succeed because the daughter is still NSRE'd, and for some reason the code doesn't register a timeout to retry again later.

The scenario is not entirely clear yet.

[tsuna]

I wrote a unit test for the scenario above, it works fine. So this isn't it. There's something else I'm missing. How come there's an entry in got_nsre without a NSRETimer for it. I've seen this bug during a loadtest a few days ago, but I still understand how it crops up.

[tsuna]

Fixed by 86c82ab