Messages go into infinite immediate retries in case there are a lot of messages (over 100) with persistent issue while processing. about nservicebus.rabbitmq

[bording]

I can reproduce the issue, and as I suspected, it only occurs when classic queues are used.

The broker provides retry count information for us via the x-delivery-count header, but only when consuming messages from a quorum queue. We have to fall back to tracking the data locally when classic queues are being used.

For this reason, and many others, I highly recommend converting to quorum queues as soon as possible! We have a migration guide and a tool to help you do this.

Regarding a fix for when running on classic queues, locally tracked information is never going to be as reliable as information provided by the broker, but we could increase the capacity of the collection. It would still be possible to hit this problem if there are enough failing messages in the queue regardless of how large the collection is, but it's clear that the current value of 100 is too low!

[bonieckimarcin]

In addition we have also noticed that switching PrefechMultiplier to 1 makes immediate retries to work properly

[bsmykowskiinf]

We also prepared a solution with minimal configuration with which we were able to reproduce the error.
To reproduce the issue, please follow instructions in readme file from the following repository:
https://github.com/bsmykowskiinf/RabbitRetriesTest

[bording]

@bonieckimarcin @bsmykowskiinf Thanks for including a repro project. That will make it much easier to investigate.

One question. I see you're using classic queues. Have you tried quorum queues? It would be good to know if it works properly with quorum queues or still fails in the same way.

[bonieckimarcin]

Upgrading to quorum is on our to do list. But looking at size and complexity of our project, it requires a lot of testing and we are not able to do that immediately. We still have to work with classic queues for some time.