Flaky test - FluxBlackboxProcessorVerification about reactor-core

[chemicL]

Be my guest @injae-kim :)

[injae-kim]

Thank you! I'll investigate the cause of failed test and create fix PR soon :)

[chemicL]

It looks like it's not the test configuration that is broken but some operator along doesn't honour the demand. If the TCK's Subscriber demands 15 items, any configuration that we come up with should not deliver more items. So it seems that at least one operator along the way is broken with that regard. Instead of changing the pipeline to avoid this problematic outcome, perhaps it's best identifying which operator delivers more than was demanded. Some race occurs which doesn't properly synchronise delivery with demand.

[injae-kim]

May I handle this issue?

[injae-kim]

  @Override @Test
  public void required_spec312_cancelMustMakeThePublisherToEventuallyStopSignaling() throws Throwable {
    // the publisher is able to signal more elements than the subscriber will be requesting in total
    final int publisherElements = 20;

    final int demand1 = 10;
    final int demand2 = 5;
    final int totalDemand = demand1 + demand2;

    activePublisherTest(publisherElements, false, new PublisherTestRun<T>() {
      @Override @SuppressWarnings("ThrowableResultOfMethodCallIgnored")
      public void run(Publisher<T> pub) throws Throwable {
        final ManualSubscriber<T> sub = env.newManualSubscriber(pub);

        sub.request(demand1);
        sub.request(demand2);
        sub.nextElement();
        sub.cancel();
...
          // if the Publisher tries to emit more elements than was requested (and/or ignores cancellation) this will throw
          assertTrue(onNextsSignalled <= totalDemand,
                     String.format("Publisher signalled [%d] elements, which is more than the signalled demand: %d",
                                   onNextsSignalled, totalDemand));

I checked that required_spec312_cancelMustMakeThePublisherToEventuallyStopSignaling TCK test requests 15 elements to upstream and then cancel it, finally assertThat(onNextsSignalled <= 15)

reactor-core/reactor-core/src/tckTest/java/reactor/core/publisher/tck/FluxBlackboxProcessorVerification.java

Lines 45 to 48 in e6d6aeb

	return f.publishOn(sharedGroup)
	.parallel(2)
	.groups()
	.flatMap(stream -> stream.publishOn(asyncGroup)

(I'm not sure) but on our FluxBlackboxProcessorVerification with parallel, I think publisher can produce element that larger than 15 as you can see on above screen shot.

So I think publisher can produce total 16 items (e.g. 1 -> 3 -> 6 -> .. -> 20 (15th) -> Complete(16th)) and TCK test can be failed!

So I suggest to fix our FluxBlackboxProcessorVerification#ftransformFlux to always produce 15 items properly. WDYT?

[injae-kim]

It's best identifying which operator delivers more than was demanded. Some race occurs which doesn't properly synchronise delivery with demand.

I agree! good point. I think I found some clue on above investigation, so I'll check it more and share the root cause to you soon.

Thank you for your guide!