n-vaidhya / transactional-outbox-main

Transactional-Outbox-Pattern

Transactional Outbox Pattern

An example of the transactional outbox pattern.

Project structure

The project consists of the following modules:

1. domain-events. Domain events shared between different services.
2. employee-service. Produces domain events reliably using the transactional outbox pattern and then forwards them to an SNS FIFO topic. Uses employees as an example.
3. consumer-service. Reads the published domain events by polling its own SQS FIFO queue. This could be any service interested in the events published by the employee-service.

Architecture

• Employee Service inserts, updates, or deletes employee entities and inserts generated domain events in the outbox table in a transaction.
• Uses Amazon SNS and Amazon SQS services to reliably forward events to multiple consumers.
• Employee Service polls the outbox table and forwards events to an SNS FIFO topic.
• Events related to the same employee are ordered. Events related to different employees may be consumed out of order.
• To maintain the order of events only one employee instance can read the database at the same time.
• SNS/SQS FIFO ensures there is no duplicate message delivery within a five-minute interval.
• Each consumer service has its own SQS FIFO.
• SQS FIFO subscribes to the employee SNS FIFO topic. A subscription filter is used to subscribe to specific event types.
• Consumer Service polls its queue and consumes the domain events.

Running locally

You can run the project locally using docker. The configuration provided matches the above architecture diagram. I used localstack, so no need for an AWS account to run the project.

You first need to compile and package all modules.

mvn clean package -Dmaven.test.skip=true

Then you can build and start services in docker.

docker-compose up --build

All services should be up and running.

Testing locally

Now that all services are running locally we can do some manual testing.

We can create an employee.

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{ "firstName": "Jake", "lastName": "Smith", "email": "[email protected]" }' \
  http://localhost:8001/employee

The consumer-service-1 logs that it received the domain event EmployeeCreated.

Now we can delete the employee using the UUID from the response.

curl --request DELETE http://localhost:8001/employee/8741749a-43f0-4463-a662-2ee693de749e

The consumer-service-2 logs that it received the domain event EmployeeDeleted.