aws-samples / aws-serverless-ecommerce-platform

Serverless Ecommerce Platform is a sample implementation of a serverless backend for an e-commerce website. This sample is not meant to be used as an e-commerce platform as-is, but as an inspiration on how to build event-driven serverless microservices on AWS.

AWS Serverless Ecommerce Platform

Status: Work-in-progress. Please create issues or pull requests if you have ideas for improvement.

The Serverless Ecommerce Platform is a sample implementation of a serverless backend for an e-commerce website. Functionalities are split across multiple micro-services that communicate either through asynchronous messages over Amazon EventBridge or over synchronous APIs.

This sample is not meant to be used as an e-commerce platform as-is, but as an inspiration on how to build event-driven serverless microservices on AWS. This makes lots of assumptions on the order flow that might not be suitable for most e-commerce platform and doesn't include many of the features that you might need for this.

Please note that you may incure AWS charges for deploying the ecommerce platform into your AWS account as not all services used are part of the free tier and you might exceed the free tier usage limit. To track costs in your AWS account, consider using AWS Cost Explorer and AWS Billing and Cost Management. You can also set up a billing alarm to get notified of unexpected charges.

Getting started

To install the necessary tools and deploy this in your own AWS account, see the getting started guide in the documentation section.

Architecture

High-level architecture

This is a high-level view of how the different microservices interact with each other. Each service folder contains anarchitecture diagram with more details for that specific service.

Technologies used

Communication/Messaging:

• AWS AppSync for interactions between users and the ecommerce platform.
• Amazon API Gateway for service-to-service synchronous communication (request/response).
• Amazon EventBridge for service-to-service asynchronous communication (emitting and reacting to events).

Authentication/Authorization:

• Amazon Cognito for managing and authenticating users, and providing JSON web tokens used by services.
• AWS Identity and Access Management for service-to-service authorization, either between microservices (e.g. authorize to call an Amazon API Gateway REST endpoint), or within a microservice (e.g. granting a Lambda function the permission to read from a DynamoDB table).

Compute:

• AWS Lambda as serverless compute either behind APIs or to react to asynchronous events.

Storage:

• Amazon DynamoDB as a scalable NoSQL database for persisting informations.

CI/CD:

• AWS CloudFormation with AWS Serverless Application Model for defining AWS resources as code in most services.
• AWS Cloud Development Kit (CDK) for defining AWS resources as code in the payment-3p service.
• Amazon CodeCommit as a repository to trigger the CI/CD pipeline.
• Amazon CodeBuild for building artifacts for microservices and running tests.
• Amazon CodePipeline for orchestrating the CI/CD pipeline to production.

Monitoring:

• Amazon CloudWatch for metrics, dashboards, log aggregation.
• AWS X-Ray for tracing across AWS services and across microservices.

Backend services

Services	Description
users	Provides user management, authentication and authorization.
products	Source of truth for products information.
orders	Manages order creation and status.
warehouse	Manages inventory and packaging orders.
delivery	Manages shipping and tracking packages.
delivery-pricing	Pricing calculator for deliveries.
payment	Manages payment collection and refunds.
payment-3p	Simulates a third party payment system.

Frontend service

Services	Description
frontend-api	User-facing API for interacting with the services.

Infrastructure services

Services	Description
pipeline	CI/CD pipeline for deploying the resources in production.
platform	Core platform resources for deploying backend services.

Shared resources

Name	Description
docs	Documentation application for all services.
shared	Shared resources accessible for all services, such as common CloudFormation templates and OpenAPI schemas.
tools	Tools used to build services.

Documentation

See the docs folder for the documentation.

Contributing

See the contributing and getting started documents to learn how to contribute to this project.

License

This library is licensed under the MIT-0 License. See the LICENSE file.

aws-serverless-ecommerce-platform's Issues

orders: check that PackageCreated has the same products as the order

The OnEvents function should check that a PackageCreated contains the same products as the order. If not, it should change them.

products: refactor integration test for ValidateFunction

The test case uses static data and setup_module() instead of a fixture. It also doesn't delete the DynamoDB item afterward.

tests(orders): test that delivery/warehouse events trigger an order event

Events like PackageCreated, DeliveryCompleted, etc. should trigger an OrderModified event.

tools: ensure that dependencies are deployed in dev environment

Right now, developers are responsible to make sure that they deploy stacks in the right order in the development environment. With this, it would automatically detect dependencies and deploy as needed.

Windows support

Could you please create a Windows version for building the project?

delivery: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the delivery service.

Add GitHub workflows for unit tests and lint

Add GitHub workflows that will automatically perform unit tests and lint against commits to master and pull requests.

frontend-api: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the frontend-api service.

tests(delivery): create integration tests for on_package_created

The on_package_created function does not have any integration tests at the moment.

products: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the products service.

orders: add CloudWatch alarms

Add CloudWatch alarms for the orders service.

users: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the users service.

payment: create the payment service

Create the payment service, which should be able to:

• verify payment tokens.
• associate the orderId and paymentToken on OrderCreated events.
• update the payment total on OrderModified events.
• process the payment on DeliveryCompleted events.
• cancel the pre-authorization on PackagingFailed and DeliveryFailed events.

payment-3p: create a fake 3rd party service for payments

This will create a service called "payment-3p", acting as a 3rd party payment service.

payment: add CloudWatch alarms

Add CloudWatch alarms for the payment service.

shared: inject helpers functions in Lambda environment

There are many functions that gets repeated in all Lambda functions. It'd be good to have a helpers.py file that gets injected in every Lambda function to prevent duplicating code.

warehouse: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the warehouse service.

bug: DLQ permissions

Right now, there is an issue to DLQ permissions which prevent service stacks with DLQs to deploy properly.

Error message from ecommerce-dev-product stack, TableUpdateFunctionDynamoDB resource:

The provided execution role does not have permissions to call SendMessage on SQS (Service: AWSLambda; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: XXX)

platform: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the platform service.

Add test coverage for unit tests

Unit tests should cover a certain percentage of the Lambda function codes. This should be checked when running tools/toolbox {service} tests-unit.

payment: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the payment service.

add DLQs for events

Some services have tables that trigger a Lambda functions upon change. It would be good to capture failed events to a dead letter queue.

• delivery: EventBridge
• orders: DynamoDB
• products: DynamoDB
• warehouse: EventBridge
• warehouse: DynamoDB

orders: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the orders service.

tools: split packaging for Pipeline per service

Right now, each service artifacts gets packaged centrally when using the pipeline. This would add a new function (or change some existing behaviour) to make this within the realm of the Makefile.

docs: update testing

Now that end-to-end tests are implemented, we need to update the docs/testing.md section and go more in-depth into each type of test.

add CloudWatch dashboards for all services

Add monitoring dashboards for the following services:

• frontend-api
• delivery
• delivery-pricing
• orders
• payment
• payment-3p
• pipeline
• platform
• products
• users
• warehouse

tools: generate parameter files

Right now, parameters have to be injected using parameter overrides. During the build phase, tools/toolbox should create parameter files for various environments (dev, tests, staging, prod).

pipeline: add limitations to deployment roles

Right now, deployment roles have full access.

Enforce source condition for "events:*"

With cfn-lint, we can add custom rules to verify permissions.

As EventBridge supports fine-grained permissions on the source type and detail type, we could enforce that messages come from the right source using.

bug: 'cdk synth' doesn't work for GitHub actions

The payment-3p service uses CDK, and run cdk synth as part of its build process. However, cdk synth in GitHub actions results in the following error:

ENOENT: no such file or directory, open '/home/runner/.aws/credentials'

Add security testing capabilities

The security command should be able to perform security analysis on a service, namely:

• Check for vulnerabilities in dependencies
• Check for vulnerabilities in function code
• Check for overly permissive IAM policies
• Check for infringement on rules, such as allowing to contact another service
• Check for infringement on EventBridge rules, such as creating a rule that listens to source outside what is authorized for that service

payment-3p build depends on local typescript installation

When running make all-payment-3p, the service fails to deploy properly as the typescript isn't transpiled. This is due to a missing typescript binary in my development environment. The build process shouldn't have dependencies on the local environment.

Inject service name from metadata in CloudFormation stack

The metadata.yaml file contains the service name. This can be used as a parameter to inject in the CloudFormation template.

Add service metadata information

The service metadata information would contain things such as:

• Permissions: which services are authorized to access resources from that service
• Dependencies: which services are needed for this service to work propely.

tools: add support for advanced deployment options

This would add support for the following deployment options:

• deploy all dependencies of specific service(s) (e.g. based on changes since a given commit)
• deploy to a new environment based on branch, run integration tests, then tear down that branch.

frontend: add CloudWatch alarms

Add CloudWatch alarms for the frontend service.

lint: run lint on test files

Right now, lint only runs on OpenAPI documents, CloudFormation templates and Lambda source code. It should also run on test files.

tests: add helpers and fixtures around common cases in unit tests

Common cases:

• Fixture API Gateway event
• DynamoDB requests
• Lambda requests

Add end-to-end tests

Right now, the system supports unit and integration tests. This would add support for end-to-end tests.

However, end-to-end tests span multiple services and therefore might not be stored in a service folder.

Add linting support

Add a linting functionality in tools/toolbox that could:

• Analyze CloudFormation templates
• Analyze Lambda function code
• Analyze OpenAPI documents

platform: add CloudWatch alarms

Add CloudWatch alarms for the platform service.

tools: flags for enabling features

Proposed change

This would add flags through CloudFormation parameters and conditions to allow enabling/disabling some resources and features based on the environment or user preferences. For example, enabling/disabling the listener SQS stack, CloudWatch dashboards, etc.

Right now, several stacks rely on an IsProd condition to check if prod features should be enabled.

With the proposed change, the environments.yaml file would add more parameters specific to these features. There would also be a new file (features.yaml) that would be masked in .gitignore where users could manually disable features that could incur costs. This file would override values from environments.yaml.

Right now, many features in CloudWatch have a fixed cost per resource per month, which could create an unpleasant surprise.

However, as this relies on CloudFormation parameters, which have a limit of 60 parameters per template, it's important to keep the number of feature flags low.

Example

For example, if we have a feature to enable monitoring features and we decide to use it only for production the environments.yaml file would look like this:

dev:
  parameters:
    LogLevel: DEBUG
    RetentionInDays: "7"

# ...

prod:
  parameters:
    LogLevel: INFO
    RetentionInDays: "30"
    MonitoringEnabled: false
  flags:
    can-test: false
    # This would probably be made redundant
    # is-prod: true

A CloudFormation template would include this (if they have monitoring features beyond the baseline):

Parameters:
  Environment:
    Type: String
    Default: dev
    Description: Environment name
  #...
  MonitoringEnabled:
    Type: String
    Default: false
    Description: Enable advanced monitoring features
  #...

Conditions:
  HasMonitoring: !Equals [!Ref MonitoringEnabled, true]

#...
Resources:
  #...
  Dashboard:
    Type: AWS::CloudWatch::Dashboard
    Condition: HasMonitoring
    Properties:
      #...

The features file would look like this:

features:
  MonitoringEnabled: false

orders: manage order status

This change adds a new field to orders named "status", which is already part of the Order schema, and a new field called "history" containing the changes for the order over time.

The "status" field could have the following values:

• NEW
• PACKAGED
• PACKAGING_FAILED
• DELIVERY_FAILED
• FULFILLED

This should add the following capabilities:

• New orders should have a status field with the value "NEW".
• Changing the status to PACKAGED following a PackageCreated event from the warehouse service.
• Changing the status to FULFILLED following a DeliveryCompleted event from the delivery service.
• Changing the status to PACKAGING_FAILED following a PackagingFailed event from the warehouse service.
• Changing the status to DELIVERY_FAILED following a DeliveryFailed event from the delivery service.

shared: split ecom helpers based on functionality

Right now, ecom.helpers provider helpers for both API Gateway integration and EventBridge. It would be nice to have ecom.apigateway with API Gateway helpers and ecom.eventbridge for EventBridge helpers.

payment-3p: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the payment-3p service.

Create a CI/CD pipeline for services

Add a CI/CD pipeline that can deploy each service.

The pipeline should be able to:

• perform linting
• build the service
• run unit tests
• deploy to tests
• run integration tests in tests
• deploy to staging
• run end-to-end tests in staging
• run security tests in staging
• deploy to prod

The staging and prod environments should be different AWS accounts. Therefore, the CI/CD pipeline should be able to deploy across accounts.

delivery-pricing: add CloudWatch dashboard

Ticket to track the creation of a CloudWatch dashboard for the delivery-pricing service.

payment-3p: provide a way to cancel payment requests

Right now, there are no API path to cancel a payment request. However, the payment service should contact the payment-3p service to cancel a payment request if the fulfilment of an order failed.

• payment-3p: add cancelPayment API
• payment: send a request to cancel the payment in the OnFailed function

tools: add support for parallel deployment of independent services

This feature would make deployment of services that are not dependent on each other parallel rather than in series, thus speeding up deployment of a development environment.

tools: detect changes between builds

Proposed change

This change would prevent building resources that are already built and thus lower the iteration time for development changes.

On the first make build for a given service, it will create a .checksum file in the build folder. This file will be a checksum of all the relevant files for the build command. There could be multiple checksum files for different build operations (e.g. CloudFormation, OpenAPI, Python code, etc.).

On subsequent runs, this would check the checksum file(s) and compare them to the checksums of the existing files. If there are no differences, this will skip the build step and consider it as successful.

Risks

If files outside of the service folder are changed (e.g. files in shared/src/ecom/), this could create discrepancies between the development environments and what would be built in the CI/CD pipeline. This could be overcome with the detection of changes within shared libraries.