threedotslabs / watermill-amazonsqs

Watermill - Amazon SNS/SQS PubSub Adapter

This package is still under development and NOT ready for production use.

Notes for SQS

A new SQS Publisher will create the queue if it does not exist.

Notes for SNS

A new SQS Publisher will create the topic if it does not exist. Be aware that without any subsciptions the messages published to that newly created topic would be lost. Make sure you always create the topic and subscriptions before you start an SNS publisher.

Unit Tests

You can run the unit tests by simply running the command: go test -cover -race ./...

Development and Testing

To try the in test mode (using goaws)

• start docker using: docker-compose up -d
• export AWS_ENDPOINT=http://localhost:4100/

Now you can run the application: go run cmd/main.go

watermill-amazonsqs's Issues

Refactor the utils.go and marshaller/unmarshaller into separate packages

Move those common components out of the amazonsqs package into a separate utils package?

proposal(sqs): automatic encoding of messages

Goal

Ensure payloads send using the publisher confirm as valid message types. Users of watermill should not be concerned with ensuring that their payload is valid for the publisher they're using (Apart from potential sizing issues).

See https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html detailing the supported characters for a payload

Changes

Check the message being published for any invalid characters, if it is found, base64 the message and add a metadata item with key _watermill_encoding & value base64.

The subscriber is auto updated to check for the _watermill_encoding metadata value, reversing the base64 encoding if it is found.

Help needed?

Hey @m110 ,
I have been enjoying your content and your course (so far 😉 )! At work, we are on the journey to an event-driven-architecture and are leveraging SQS in specific quite a bit. We would really like to test out watermill as the backbone for this architecture but this package seems a bit "underdeveloped" (no offense!). Do you have concrete plans to maintain and support it or is this more something the community should take care of? I am just curious if you are looking for help to maintain/develop this package to bring it to a stable release? Or do you suggest to better fork and maintain in private?

Greetings from Berlin!

Use aws-sdk-go-v2

The aws-sdk-go-v2 library has been available for quite a while and is pretty mature. Maybe it's a good idea to upgrade from aws-sdk-go: https://github.com/aws/aws-sdk-go-v2