Deterministically pick the next future to poll about futures-concurrency

[yoshuawuyts]

@eholk thanks for opening this!

I think you're correctly pointing out that fairness and non-determinism are not the same, and don't need to be implemented through the same mechanism. While we may want fairness, non-determinism comes at a cost.

Also, determinism is somewhat in conflict with fairness. If we did something like [ready(1), ready(2), ready(3)].race() then by our intuitive definition of fairness we should expect to get either 1, 2, or 3 out. But any deterministic scheduler will always return the same answer. So a deterministic scheduler can only be fair in the long run, but not for short-lived processes.

This to me is quite interesting! - To me the example you used is pretty much the "perfect" counter-example, and most cases wouldn't quite be like this. In practice it seems rare to expect someone to explicitly race two synchronous operations. In the case someone races 1 async + 1 sync operation, we already deterministically know that the synchronous one will complete before the async one. So the poll ordering practically doesn't matter.

The main example I'm thinking of is: we want to ensure that if we have a set of iterators, one iterator isn't starved before the other. We don't want Merge to accidentally provide Chain semantics when operating on non-async sequences. But as you say, we can trivially guarantee that by simply doing a round-robin iteration. [repeat(1), repeat(2), repeat(3)].merge() would then always yield 1, 2, 3, 1, 2, 3, 1, 2, 3. Which is fair and also deterministic.

Conclusion

I'm on board with this change ✅. Probably for the implementation it'd be nice if we could create a shared Indexer struct or something which replaces the current RandomGenerator and just cycles through the starting index. Basically the same semantics as (1..N).cycle() - but as a separate struct.