Node - Koa - Typescript Project

The main purpose of this repository is to show a good end-to-end project setup and workflow for writing Node code in TypeScript using KOA and a NOSQL DB.

Koa is a new web framework designed by the team behind Express, which aims to be a smaller, more expressive, and more robust foundation for web applications and APIs. Through leveraging generators Koa allows you to ditch callbacks and greatly increase error-handling. Koa does not bundle any middleware within core, and provides an elegant suite of methods that make writing servers fast and enjoyable.

Through Github Actions CI, this boilerplate is deployed here! You can try to make requests to the different defined endpoints and see how it works. The following Authorization header will have to be set (already signed with the boilerplate's secret) to pass the JWT middleware:

HEADER (DEMO)

Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjEiLCJuYW1lIjoiSmF2aWVyIEF2aWxlcyIsImVtYWlsIjoiYXZpbGVzbG9wZXouamF2aWVyQGdtYWlsLmNvbSJ9.7oxEVGy4VEtaDQyLiuoDvzdO0AyrNrJ_s9NU3vko5-k

AVAILABLE ENDPOINTS DEMO SWAGGER DOCS DEMO

When running the project locally with watch-server, the swagger docs will be deployed at: http:localhost:3000/swagger-html, and the bearer token for authorization should be as follows:

HEADER (LOCALHOST BASED ON DEFAULT SECRET KEY 'your-secret-whatever')

Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjEiLCJuYW1lIjoiSmF2aWVyIEF2aWxlcyIsImVtYWlsIjoiYXZpbGVzbG9wZXouamF2aWVyQGdtYWlsLmNvbSJ9.rgOobROftUYSWphkdNfxoN2cgKiqNXd4Km4oz6Ex4ng

method	resource	description
`GET`	`/`	Simple hello world response
`GET`	`/users`	returns the collection of users present in the DB
`GET`	`/users/:id`	returns the specified id user
`POST`	`/users`	creates a user in the DB (object user to be includued in request's body)
`PUT`	`/users/:id`	updates an already created user in the DB (object user to be includued in request's body)
`DELETE`	`/users/:id`	deletes a user from the DB (JWT token user ID must be the same as the user you want to delete)

Pre-reqs

To build and run this app locally you will need a few things:

Install Node.js
Install MongoDB
Install VS Code

Features:

Nodemon - server auto-restarts when code changes
Koa v2
Swagger decorator (auto generated swagger docs)
Class-validator - Decorator based entities validation
Docker-compose ready to go
Postman (newman) integration tests
Locust load tests
Github actions - CI for building and testing the project
Cron jobs prepared

Included middleware:

koa-router
koa-bodyparser
Winston Logger
JWT auth koa-jwt
Helmet (security headers)
CORS

Getting Started

Clone the repository

git clone --depth=1 https://github.com/ivan2261/node-typescript-koa-starter.git <project_name>

Install dependencies

cd <project_name>
npm install

Configure your mongoDB server

# create the db directory
sudo mkdir -p /data/db
# give the db correct read/write permissions
sudo chmod 777 /data/db

# starting from macOS 10.15 even the admin cannot create directory at root
# so lets create the db diretory under the home directory.
mkdir -p ~/data/db
# user account has automatically read and write permissions for ~/data/db.

Start your mongoDB server (you'll probably want another command prompt)

mongod

# on macOS 10.15 or above the db directory is under home directory
mongod --dbpath ~/data/db

Run the project directly in TS

npm run watch-server

Build and run the project in JS

npm run build
npm run start

Run integration or load tests

npm run test:integration
npm run test:load

Docker (optional)

A docker-compose file has been added to the project with a postgreSQL (already setting user, pass and dbname as the ORM config is expecting) and an ADMINER image (easy web db client).

It is as easy as go to the project folder and execute the command 'docker-compose up' once you have Docker installed, and both the postgreSQL server and the Adminer client will be running in ports 5432 and 8080 respectively with all the config you need to start playing around.

If you use Docker natively, the host for the server which you will need to include in the ORM configuration file will be localhost, but if you were to run Docker in older Windows versions, you will be using Boot2Docker and probably your virtual machine will use your ip 192.168.99.100 as network adapter. This mean your database host will be the aforementioned ip and in case you want to access the web db client you will also need to go to http://19.168.99.100/8080

Entities validation

This project uses the library class-validator, a decorator-based entity validation, which is used directly in the entities files as follows:

export class User {
    @Length(10, 100) // length of string email must be between 10 and 100 characters
    @IsEmail() // the string must comply with an standard email format
    @IsNotEmpty() // the string can't be empty
    email: string;
}

Once the decorators have been set in the entity, you can validate from anywhere as follows:

const user = new User();
user.email = "avileslopez.javier@gmail"; // should not pass, needs the ending .com to be a valid email

validate(user).then(errors => { // errors is an array of validation errors
    if (errors.length > 0) {
        console.log("validation failed. errors: ", errors); // code will get here, printing an "IsEmail" error
    } else {
        console.log("validation succeed");
    }
});

For further documentation regarding validations see class-validator docs.

Environment variables

Create a .env file (or just rename the .example.env) containing all the env variables you want to set, dotenv library will take care of setting them. This project is using three variables at the moment:

PORT -> port where the server will be started on, Heroku will set this env variable automatically
NODE_ENV -> environment, development value will set the logger as debug level, also important for CI. In addition will determine if the ORM connects to the DB through SSL or not.
JWT_SECRET -> secret value, JWT tokens should be signed with this value
DATABASE_URL -> DB connection data in connection-string format.

Getting TypeScript

TypeScript itself is simple to add to any project with npm.

npm install -D typescript

If you're using VS Code then you're good to go! VS Code will detect and use the TypeScript version you have installed in your node_modules folder. For other editors, make sure you have the corresponding TypeScript plugin.

Project Structure

The most obvious difference in a TypeScript + Node project is the folder structure. TypeScript (.ts) files live in your src folder and after compilation are output as JavaScript (.js) in the dist folder.

The full folder structure of this app is explained below:

Note! Make sure you have already built the app using npm run build

Name	Description
dist	Contains the distributable (or output) from your TypeScript build. This is the code you ship
node_modules	Contains all your npm dependencies
src	Contains your source code that will be compiled to the dist dir
src/server.ts	Entry point to your KOA app
.github/workflows/test.yml	Github actions CI configuration
loadtests/locustfile.py	Locust load tests
integrationtests/node-koa-typescript.postman_collection.json	Postman integration test collection
.copyStaticAssets.ts	Build script that copies images, fonts, and JS libs to the dist folder
package.json	File that contains npm dependencies as well as build scripts
docker-compose.yml	Docker PostgreSQL and Adminer images in case you want to load the db from Docker
tsconfig.json	Config settings for compiling server code written in TypeScript
tslint.json	Config settings for TSLint code style checking
.example.env	Env variables file example to be renamed to .env
Dockerfile and dockerignore	The app is dockerized to be deployed from CI in a more standard way, not needed for dev

Configuring TypeScript compilation

TypeScript uses the file tsconfig.json to adjust project compile options. Let's dissect this project's tsconfig.json, starting with the compilerOptions which details how your project is compiled.

    "compilerOptions": {
        "module": "commonjs",
        "target": "es2017",
        "lib": ["es6"],
        "moduleResolution": "node",
        "sourceMap": true,
        "outDir": "dist",
        "baseUrl": ".",
        "experimentalDecorators": true,
        "emitDecoratorMetadata": true,  
        }
    },

`compilerOptions`	Description
`"module": "commonjs"`	The output module type (in your `.js` files). Node uses commonjs, so that is what we use
`"target": "es2017"`	The output language level. Node supports ES2017, so we can target that here
`"lib": ["es6"]`	Needed for TypeORM.
`"moduleResolution": "node"`	TypeScript attempts to mimic Node's module resolution strategy. Read more here
`"sourceMap": true`	We want source maps to be output along side our JavaScript.
`"outDir": "dist"`	Location to output `.js` files after compilation
`"baseUrl": "."`	Part of configuring module resolution.
`"experimentalDecorators": true`	Needed for TypeORM. Allows use of @Decorators
`"emitDecoratorMetadata": true`	Needed for TypeORM. Allows use of @Decorators

The rest of the file define the TypeScript project context. The project context is basically a set of options that determine which files are compiled when the compiler is invoked with a specific tsconfig.json. In this case, we use the following to define our project context:

    "include": [
        "src/**/*"
    ]

include takes an array of glob patterns of files to include in the compilation. This project is fairly simple and all of our .ts files are under the src folder. For more complex setups, you can include an exclude array of glob patterns that removes specific files from the set defined with include. There is also a files option which takes an array of individual file names which overrides both include and exclude.

Running the build

All the different build steps are orchestrated via npm scripts. Npm scripts basically allow us to call (and chain) terminal commands via npm. This is nice because most JavaScript tools have easy to use command line utilities allowing us to not need grunt or gulp to manage our builds. If you open package.json, you will see a scripts section with all the different scripts you can call. To call a script, simply run npm run <script-name> from the command line. You'll notice that npm scripts can call each other which makes it easy to compose complex builds out of simple individual build scripts. Below is a list of all the scripts this template has available:

Npm Script	Description
`start`	Does the same as 'npm run serve'. Can be invoked with `npm start`
`build`	Full build. Runs ALL build tasks (`build-ts`, `tslint`, `copy-static-assets`)
`serve`	Runs node on `dist/server/server.js` which is the apps entry point
`watch-server`	Nodemon, process restarts if crashes. Continuously watches `.ts` files and re-compiles to `.js`
`build-ts`	Compiles all source `.ts` files to `.js` files in the `dist` folder
`tslint`	Runs TSLint on project files
`copy-static-assets`	Calls script that copies JS libs, fonts, and images to dist directory
`test:integration`	Execute Postman integration tests collection using newman
`test:load`	Execute Locust load tests using a specific configuration

CI: Github Actions

Using Github Actions a pipeline is deploying the application in Heroku and running tests against it, checking the application is healthy deployed. The pipeline can be found at /.github/workflows/test.yml. This performs the following:

Build the project
- Install Node
- Install dependencies
- Build the project (transpile to JS)
- Run unit tests
Deploy to Heroku
- Install Docker cli
- Build the application container
- Install Heroku cli
- Login into Heroku
- Push Docker image to Heroku
- Trigger release in Heroku
Run integration tests
- Install Node
- Install Newman
- Run Postman collection using Newman against deployed app in Heroku
Run load tests
- Install Python
- Install Locust
- Run Locust load tests against deployed app in Heroku

TSLint

TSLint is a code linter which mainly helps catch minor code quality and style issues. TSLint is very similar to ESLint or JSLint but is built with TypeScript in mind.

TSLint rules

Like most linters, TSLint has a wide set of configurable rules as well as support for custom rule sets. All rules are configured through tslint.json. In this project, we are using a fairly basic set of rules with no additional custom rules. The settings are largely based off the TSLint settings that we use to develop TypeScript itself.

Running TSLint

Like the rest of our build steps, we use npm scripts to invoke TSLint. To run TSLint you can call the main build script or just the TSLint task.

npm run build   // runs full build including TSLint
npm run tslint  // runs only TSLint

Notice that TSLint is not a part of the main watch task. It can be annoying for TSLint to clutter the output window while in the middle of writing a function, so I elected to only run it only during the full build. If you are interesting in seeing TSLint feedback as soon as possible, I strongly recommend the TSLint extension in VS Code.

Register cron jobs

Cron dependency has been added to the project together with types. A cron.ts file has been created where a cron job is created using a cron expression configured in config.ts file.

import { CronJob } from 'cron';
import { config } from './config';

const cron = new CronJob(config.cronJobExpression, () => {
    console.log('Executing cron job once every hour');
});

export { cron };

From the server.ts, the cron job gets started:

import { cron } from './cron';
// Register cron job to do any action needed
cron.start();

Integrations and load tests

Integrations tests are a Postman collection with assertions, which gets executed using Newman from the CI (Github Actions). It can be found at /integrationtests/node-koa-typescript.postman_collection.json; it can be opened in Postman and get modified very easily.

Load tests are a locust file with assertions, which gets executed from the CI (Github Actions). It can be found at /loadtests/locustfile.py; It is written in python and can be executed locally against any host once python and locust are installed on your dev machine.

**NOTE: at the end of load tests, an endpoint to remove all created test users is called.

Logging

Winston is designed to be a simple and universal logging library with support for multiple transports.

A "logger" middleware passing a winstonInstance has been created. Current configuration of the logger can be found in the file "logger.ts". It will log 'error' level to an error.log file and 'debug' or 'info' level to the console.

// Logger middleware -> use winston as logger (logging.ts with config)
setupLogging(app);

Authentication - Security

The idea is to keep the API as clean as possible, therefore the auth will be done from the client using an auth provider such as Auth0. The client making requests to the API should include the JWT in the Authorization header as "Authorization: Bearer <jwt_token>". HS256 will be used as the secret will be known by both your api and your client and will be used to sign the token, so make sure you keep it hidden.

As can be found in the server.ts file, a JWT middleware has been added, passing the secret from an environment variable. The middleware will validate that every request to the routes below, MUST include a valid JWT signed with the same secret. The middleware will set automatically the payload information in ctx.state.user.

// JWT middleware -> below this line, routes are only reached if JWT token is valid, secret as env variable
app.use(jwt({ secret: config.jwtSecret }));

Go to the website https://jwt.io/ to create JWT tokens for testing/debugging purposes. Select algorithm HS256 and include the generated token in the Authorization header to pass through the jwt middleware.

Custom 401 handling -> if you don't want to expose koa-jwt errors to users:

app.use(function(ctx, next){
  return next().catch((err) => {
    if (401 == err.status) {
      ctx.status = 401;
      ctx.body = 'Protected resource, use Authorization header to get access\n';
    } else {
      throw err;
    }
  });
});

If you want to authenticate from the API, and you fancy the idea of an auth provider like Auth0, have a look at jsonwebtoken — JSON Web Token signing and verification

CORS

This boilerplate uses @koa/cors, a simple CORS middleware for koa. If you are not sure what this is about, click here.

// Enable CORS with default options
app.use(cors());

Have a look at Official @koa/cors docs in case you want to specify 'origin' or 'allowMethods' properties.

Helmet

This boilerplate uses koa-helmet, a wrapper for helmet to work with koa. It provides important security headers to make your app more secure by default.

Usage is the same as helmet. Helmet offers 11 security middleware functions (clickjacking, DNS prefetching, Security Policy...), everything is set by default here.

// Enable helmet with default options
app.use(helmet());

Have a look at Official koa-helmet docs in case you want to customize which security middlewares are enabled.

Dependencies

Dependencies are managed through package.json. In that file you'll find two sections:

dependencies

Package	Description
config	Loads config variables from .yml file.
koa	Node web framework.
koa-bodyparser	A bodyparser for koa.
koa-jwt	Middleware to validate JWT tokens.
koa-router	Router middleware for koa.
koa-helmet	Wrapper for helmet, important security headers to make app more secure
@koa/cors	Cross-Origin Resource Sharing(CORS) for koa
mongoose	MongoDB ODM.
reflect-metadata	Used by typeORM to implement decorators.
winston	Logging library.
class-validator	Decorator based entities validation.
koa-swagger-decorator	using decorator to automatically generate swagger doc for koa-router.
cron	Register cron jobs in node.

devDependencies

Package	Description
@types	Dependencies in this folder are `.d.ts` files used to provide types
nodemon	Utility that automatically restarts node process when it crashes
ts-node	Enables directly running TS files. Used to run `copy-static-assets.ts`
tslint	Linter (similar to ESLint) for TypeScript files
typescript	JavaScript compiler/type checker that boosts JavaScript productivity
shelljs	Portable Unix shell commands for Node.js

To install or update these dependencies you can use npm install or npm update.

ivan2261 / node-typescript-koa-rest Goto Github PK

node-typescript-koa-rest's Introduction