garryyao / replay-aws-dlq

Replay AWS Dead Letter Queue messages on another queue

replay-aws-dlq

Re-drive AWS (dead letter) queue messages to target queue

Installation

$ npm install --save replay-aws-dlq

Usage

The following command will move all messages in [source] queue to [dest] queue where source queue is usually a dead-letter queue, and the destination queue is it's pairing queue.

replay-aws-dlq [source_queue_url] [dest_queue_url]

This module use AWS sdk beneath so you shall be able to use env variables to work across different accounts, e.g. the example below use AWS profile.

AWS_PROFILE=staging replay-aws-dlq  https://sqs.eu-central-1.amazonaws.com/123/my-queue-deadletter.fifo https://sqs.eu-central-1.amazonaws.com/123/my-queue.fifo

Special Note for FIFO queue

Please be aware if destination queue you're driving messages to, is FIFO queue that has enabled deduplication, and messages in DLQ were original come from it, you'll need to cool down for at least 5 mins internal before you can start driving messages back, otherwise you messages will end up not showing in the dest queue.

License

MIT © Garry Yao

replay-aws-dlq's Issues

License link goes to 404 in README

MIT © [Garry Yao]()

Can I recommend:

Licensed under [MIT](/LICENSE).

Or

MIT © [Garry Yao](https://github.com/garryyao)

moving fifo-based messages breaks parallel consumption

Currently, moving messages from a dead fifo queue to another fifo queue breaks parallel processing due to a non-unique message group ID. Is it not possible to pass the original message group ID?

replay-aws-dlq/lib/index.js

Line 41 in 9baa411

groupId: 're-drive',

Q: Do Amazon SQS FIFO queues support multiple consumers?

By design, Amazon SQS FIFO queues don't serve messages from the same message group to more than one consumer at a time. However, if your FIFO queue has multiple message groups, you can take advantage of parallel consumers, allowing Amazon SQS to serve messages from different message groups to different consumers.

as per https://aws.amazon.com/sqs/faqs/