Postman.gov.sg enables public officers to send templated messages to many recipients.
- Features
- API Usage
- Development
- Deployment
- Releasing
- Serverless
- Downtime procedure
- Infrastructure customizations
- Architecture
- Local node module
- How messages are sent
- Forking and configuring this product
- Contributions
- Passwordless login: login with your .gov.sg email
- Easily customize messages to reach a wide audience: create a message template, upload a file containing customization parameters and we will handle the customization for you
- Send emails: Just click send and Postman will send those messages out to your intended audience via email
- Send SMSes: Enter your twilio credentials, and Postman will send those messages via SMS. No integration with twilio is necessary
- View stats: Keep track of your campaign's progress as it's sending, and check back when it's done.
Just want to use the API? See api-usage.md for details.
Set up a postgresql@11 database, redis cache and localstack server. PostgreSQL and Redis can be started either natively or using Docker. We recommend using Docker.
export AWS_ENDPOINT=http://localhost:4566
export FILE_STORAGE_BUCKET_NAME=localstack-upload
export AWS_LOG_GROUP_NAME=postmangovsg-beanstalk-localstack
npm run dev:services
# Install postgres
brew install postgresql@11
brew services start postgresql@11
# Create the database
createdb postmangovsg_dev
# Check if you can connect to the database
psql -h localhost -p 5432 postmangovsg_dev
# Install redis
brew install redis
brew services start redis
# Check that redis is running
redis-cli ping
# Start localstack container
export AWS_ENDPOINT=http://localhost:4566
export FILE_STORAGE_BUCKET_NAME=localstack-upload
export AWS_LOG_GROUP_NAME=postmangovsg-beanstalk-localstack
npm run dev:localstack
brew tap lucagrulla/tap
brew install cw
# Tail logs on localstack
cw tail -r ap-northeast-1 -u $AWS_ENDPOINT -f $AWS_LOG_GROUP_NAME:`node --eval='console.log(require("os").hostname())'`
This project makes of detect-secrets to prevent secrets and credentials from being committed to the repository.
It runs as a pre-commit hook and it needs to be installed if you intend to make commits to the repo.
Note: The reason we're running detect-secrets
through detect-secrets:precommit
instead of using lint-staged
is because detect-secrets-hook
doesn't work well with the combination of output of staged files by lint-staged
and baseline supplied.
Run the following to install:
pip install detect-secrets==1.2.0
Upon blockage by detect-secrets-hook
, please take these steps:
- Go into each of the locations pointed out by
detect-secrets-hook
and remove accidentally added secrets - If some of these detections are false positive (please be super sure about this, when not sure check with teammates), update the secrets baseline by running `npm run detect-secrets:update
Example environment variables can be found in
Set the environment variables in a file named .env
in each folder. If you're a developer at OGP, ask your friendly colleague for their env variables. (Please help to ensure the .env-example
file stays up-to-date though!)
npm install
This step needs to be run if you have made a change to the database schema, or if you are setting up the project for the first time.
cd backend
npm run db:migrate # run all pending migrations
npm run db:seed # seed database with dummy data
If you run into errors at the db:migrate
step, this is likely because you have created a new model in TypeScript that has not been compiled to JavaScript. Run npm run build
to fix this error.
If you need to undo any database migrations:
cd backend
npm run db:undo # undo most recent migration
You can find more info on undoing migrations using Sequelize here.
If you wish to create a new migration, run:
cd backend
npx sequelize-cli migration:create --name migration-name-in-kebab-case
lingui is used for internationalization. Read this for more info.
cd frontend
npm run extract
npm run compile
npm run dev
You should find the
- React frontend at localhost:3000
- Express backend at localhost:4000
- Swagger docs at localhost:4000/docs
Alternatively, if you would like to develop locally against staging database and workers, ensure that you have set up the necessary variables in ./backend/.env
and run either:
npm run dev:connectstagingdb
(to make a read + write connection with the staging db); ornpm run dev:connectstagingdbreadonly
(to make a readonly connection with the staging db.).
Your frontend and backend will still be on localhost
but you will be able to use staging database and workers.
We use Github Actions to simplify our deployment process:
backend
is deployed on Elastic Beanstalkfrontend
is deployed on AWS Amplifyworker
is deployed on Elastic Container Serviceserverless
is deployed on AWS Lambda
- Checkout a new release branch from
develop
. - Bump the version of each of the subpackages (frontend, backend, worker).
- Stage changes for subpackage
package.json
andpackage-lock.json
. - Bump the version of the main package.
- You will need to use
--force
as the working directory is not clean.
- You will need to use
- Push both the version commit and tag to GitHub.
- Open a new pull request and consolidate changelist from the commits made since the previous release.
- Merge pull request into master to trigger Travis build and deploy. Note that we do not squash commits when merging into master.
- Manually trigger a custom build on Travis if there were changes made to the worker.
- Create a new pull request to merge release branch back into
develop
branch.
Example:
# Create a new release branch for a new minor version (e.g. v1.6.0 -> v1.7.0)
$ git checkout develop
$ git checkout -b release-v1.7.0
# Bump shared module version
$ cd ../ && cd shared/
$ npm version minor
# Bump frontend version
$ cd frontend/
$ npm version minor
# Bump backend version
$ cd ../ && cd backend/
$ npm version minor
# Bump worker version
$ cd ../ && cd worker/
$ npm version minor
# Stage changes for subpackage package.json and package-lock.json
$ cd ../
$ git add */package.json */package-lock.json
# Create a version commit for main package
$ npm version minor --force
# Push version commit and tag to GitHub
$ git push -u origin release-v1.7.0
$ git push origin v1.7.0
We make use of AWS lambda to handle the callbacks from Twilio, as well as updating email delivery status.
See serverless-docs for details
See downtime-procedure for steps on how to bring the app down in the event that we need to make database changes
[
{
"source": "</^[^.]+$|\\\\\\\\.(?!(css|gif|ico|jpg|js|png|txt|svg|woff|ttf)$)([^.]+$)/>",
"target": "/index.html",
"status": "200",
"condition": null
}
]
Create a cluster with four services. These names are currently hardcoded for deployment in .travis.yml
Cluster Name: postmangovsg-workers |
---|
Service Name | LaunchType | Platform version |
---|---|---|
staging-sending | FARGATE | 1.4.0 |
staging-logger | FARGATE | 1.4.0 |
prod-sending | FARGATE | 1.4.0 |
prod-logger | FARGATE | 1.4.0 |
See architecture for details
See local-module.md for details
See sending.md for details
Disclaimer of Liability. This product is pending Vulnerability Assessment and Penetration Testing (VAPT). You should conduct your own security assessment prior to using code provided in this repository. Open Government Products (OGP) makes no representations or warranties of any kind, expressed or implied about the completeness, accuracy, reliability, suitability or availability of this codebase. Any usage is at your own risk.
See configure/backend for details
See configure/frontend for details
See configure/worker for details
We currently have created two Grafana product metrics dashboards hosted on an EC2 instance. You can access them by connecting to the OGP VPN and following the URLs below. In order to SSH into the EC2 instance:
- Update the following env variables on
/backend/.env
:GRAFANA_KEY_FOLDER
(get the.pem
file from 1Password and store it in your~/.ssh/
folder) andGRAFANA_EC2_HOST_URL
(this should beec2-user@<EC2_instance_Public_IPv4_DNS>
, which you can find from 1Password too). - Connect to the OGP VPN first, then run
npm run grafana
.
The URLs of the Grafana dashboards are:
The production branch is master
, and the development branch is develop
.
If you have write access to this repository
- Check out your feature branch from
develop
- Make changes, and commit those changes
- Push these changes to Github
- Submit a pull request against
develop
, filling in the standard template
If you do not have write access to this repository
- Fork this repository
- Clone the forked repository to your machine
- Create a branch, make changes and commit those changes.
- Push these changes to Github
- Submit a pull request against
basefork/develop
(that's us!) - Describe the issue as thoroughly as possible, and with screenshots if applicable. A picture speaks a thousand words!
For more information, see CONTRIBUTING.md