z-react / lad

:boy: Lad scaffolds a Koa webapp and API framework for Node.js

Lad scaffolds a Koa webapp and API framework for Node.js

_{A lad that fell in love with a lass • Built by @niftylettuce and contributors}

---

❤️ Love this project? Support @niftylettuce's FOSS on Patreon or PayPal 🦄

Table of Contents

• Features
  • Microservices
  • Front-end
  • Back-end
  • Translation
  • Email Engine
  • Error Handling
  • Performance
  • Security
• Get Started
  • Requirements
  • Install
  • Usage
  • Configuration
  • Tutorials
  • Community
• Architecture
• Principles
• Related
• Contributing
• Contributors
• Trademark Notice
• License

Features

Lad boasts dozens of features and is extremely configurable.

Microservices

These microservices are preconfigured for security, performance, and graceful reloading.

• Webapp server → web.js
• API server → api.js
• Job scheduler → bull.js
• Proxy server → proxy.js

Front-end

• Browser linting using eslint-plugin-compat and browserslist (see .browserslistrc for the default config)
• Pug template engine (you can easily use Moon, Vue, React, or Angular, though typically you aren't going to need it)
• Gulp (latest version 4.x)
• Sass
• PostCSS (with font-magician, import-url, font-grabber, base64, and cssnext pre-configured)
• Bootstrap
• Font Awesome
• SpinKit
• SweetAlert2
• Dense
• Waypoints
• LiveReload
• Frisbee
• …

Back-end

• Redis, sessions, and flash toast and modal SweetAlert2 messages (uses ioredis which has support for Cluster, Sentinel, and more)
• Koa-based webapp and API servers (uses HTTP/2 for production!)
• Pagination built-in (using ctx-paginate)
• RESTful API with BasicAuth and versioning
• Automated job scheduler with cron and human-readable syntax (backed by Mongoose and Bull)
• Passport-based authentication and group-based (Unix-like) permissioning
• Stripe-inspired error handling
• Mongoose and MongoDB with common database plugins
• Email template engine with Nodemailer and local rendering
• Proxy eliminates need for Nginx reverse-proxy or Apache virtual hosts
• Multilingual through built-in i18n translation support (see configuration)
• Automatic phrase translation with Google Translate
• Sitemap generator for simple SEO
• …

Translation

Finally a framework that solves i18n everywhere; complete with automatic translation.

• Translation constants built-in so you don't repeat yourself
• Webapp error messages and templates are translated
• Emails are translated
• API responses are translated
• Database errors are translated
• Authentication errors are translated
• …

Email Engine

Our beautiful email engine uses email-templates (which is also made by the creator of Lad)!

• Test your emails locally with automatic browser-rendering on the fly
• Automatically inlines CSS for cross-browser and cross-platform email client support
• Use Bootstrap in your email template designs
• Reuse your existing CSS and webapp styling
• Use any template engine (defaults to Pug)
• Render custom fonts in emails with code
• Add icons with Font Awesome with code
• Automatically avoid email client caching
• Include any image you want and it will be properly rendered
• Rids the need for awkward embedded image CID attachments
• …

Error Handling

We've spent a lot of time designing a beautiful error handler.

• Supports text/html, application/json, and text response types
• User-friendly responses
• HTML error lists
• …

See koa-better-error-handler for a complete reference.

Performance

• Compression and zero-bloat approach
• Stream-based file uploading
• Graceful reloading, shutdown, and reconnection handling
• Manifest asset revisioning
• Amazon S3 and CloudFront ready
• …

Security

• Database security plugins and helpers
• Automated tests and code coverage
• CORS, SameSite set to "lax" (an alternative to CSRF), CSRF (since not all browsers support SameSite yet) XSS, and rate limited protection
• Dotenv support for environment-based configurations
• App, user, and request-based logging
• SSL-ready (see instructions below)
• …

Get Started

We strictly support Mac and Ubuntu-based operating systems (Windows might work).

Requirements

Please ensure your operating system has the following software installed:

• Git - see GitHub's tutorial for installation

• Node.js (v8.9.0+) - use nvm to install it on any OS

  • After installing nvm you will need to run nvm install node
  • We also recommend you install yarn, which is an alternative to npm

• MongoDB (v3.x+):

  • Mac (via brew): brew install mongodb && brew services start mongo

  • Ubuntu:

    sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
    echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list
    sudo apt-get update
    sudo apt-get -y install mongodb-org

• Redis (v4.x+):

  • Mac (via brew): brew install redis && brew services start redis

  • Ubuntu:

    sudo add-apt-repository -y ppa:chris-lea/redis-server
    sudo apt-get update
    sudo apt-get -y install redis-server

Install

npm:

npm install -g lad

yarn:

yarn global add lad

Usage

Create a project

lad new-project
cd new-project

Development

To begin, try typing npm start (or yarn start) on command line. This will display to you all the scripts you can run.

The start script (among many others) uses nps and nps-utils under the hood. This helps to keep scripts very developer-friendly, and rids the need to write in JSON syntax.

This script accepts a <script> argument, whereas a script of all will spawn, watch, and re-compile all of the microservices mentioned above.

It will also open the browser for you to http://localhost:3000 automatically for testing the web front-end.

npm:

npm start all

yarn:

yarn start all

Debugging

• DEBUG - debug using debug output (widely adopted package in the community for debugging across all Node packages):

  DEBUG=* ...

• NODE_DEBUG - debug node internal modules:

  NODE_DEBUG=* ...

• MONGOOSE_DEBUG - debug Mongoose raw database operation output:

  MONGOOSE_DEBUG=true ...

• TRANSPORT_DEBUG - debug Nodemailer transport:

  TRANSPORT_DEBUG=true ...

• REDIS_MONITOR - debug Redis using MONITOR (uses @ladjs/redis and passes true for the monitor argument):

  REDIS_MONITOR=true ...

• REDIS_FRIENDLY_ERROR_STACK - debug Redis with friendly error stack messages (see showFriendlyErrorStack option of ioredis)

  REDIS_FRIENDLY_ERROR_STACK=true ...

Production

We strongly recommend using SemaphoreCI, PM2, and Digital Ocean for production deployment.

1. We've provided you with a preconfigured ecosystem.json deployment file. You will need to modify this file with your server's IP, hostname, and other metadata if needed.

2. Make sure that your project's assets are built with NODE_ENV=production flag, e.g. NODE_ENV=production npm run build (or with yarn as yarn build);this creates a build/rev-manifest.json file per koa-manifest-rev.

3. You can test this locally by installing PM2 globally with npm or yarn, and then running the following command:

   NODE_ENV=production pm2 start

4. See the Continuous Integration and Code Coverage and Tutorials sections below for instructions on how to setup continuous integration, code coverage, and deployment.

5. If you specify an environment variable value for AWS_CF_DOMAIN and NODE_ENV=production is set then your assets will need to be published to Amazon S3/Cloudfront. To do so run npm start publish-assets (or with yarn as yarn start publish-assets). This command automatically sets NODE_ENV=production for you as well via cross-env.

Tests

We use ava and nyc for testing and code coverage.

npm:

npm test

yarn:

yarn test

Configuration

Environment Variables

We have made configuration of your Lad project easy through a dotenv configuration package called @ladjs/env, per Twelve-Factor.

We use the following three packages to manage configuration:

• dotenv-extended - allows us to craft a .env definition (otherwise known as a "schema") in a file named .env.schema
• mustache - allows us to use the Mustache templating language in our .env and .env.defaults configuration files
• dotenv-parse-variables - automatically parses variable types from process.env (e.g. FOO=4 will set process.env.FOO = 4 with a Number variable type instead of a String)

Configuration is managed by the following, in order of priority:

1. Contents of the file at config/index.js (reads in process.env environment variables)
2. Contents of the files in directories under config/environments/ (sets defaults per environment, e.g. you can pass NODE_ENV=staging and it will load the file at config/environments/staging.js)
3. Environment variables used to override defaults or set required ones (e.g. NODE_ENV=production)
4. Environment configuration in .env
5. Environment configuration in .env.defaults

Precedence is taken by the environment configuration files, environment variables, then the .env file.

Basically dotenv won't set an environment variable if it already detects it was passed as an environment variable.

Take a look at the config folder contents and also at the defaults at .env.defaults.

SSL Configuration

To configure SSL for the web or API server simply set them in your .env file or pass them as environment variables.

Web server:

• WEB_PROTOCOL - you must set this to https
• WEB_SSL_KEY_PATH - file path to your SSL key file (e.g. /home/deploy/.ssl/web-key.pem)
• WEB_SSL_CERT_PATH - file path to your SSL certificate file (e.g. /home/deploy/.ssl/web-cert.pem)
• WEB_SSL_CA_PATH (optional) - file path to your SSL certificate authority file (e.g. /home/deploy/.ssl/web-ca-cert.pem)

API server:

• API_PROTOCOL - you must set this to https
• API_SSL_KEY_PATH - file path to your SSL key file (e.g. /home/deploy/.ssl/api-key.pem)
• API_SSL_CERT_PATH - file path to your SSL certificate file (e.g. /home/deploy/.ssl/api-cert.pem)
• API_SSL_CA_PATH (optional) - file path to your SSL certificate authority file (e.g. /home/deploy/.ssl/api-ca-cert.pem)

Outbound Email Configuration

By default in the development environment we simply render the email in your browser.

However in other environments such as production, you definitely want emails to be sent.

We built-in support for Postmark by default (though you can swap in your own transport provider in the jobs/email.js file):

1. Go to https://postmarkapp.com – Start Free Trial

2. Create a free trial account, then click Get Started, and proceed to create a "Server" and "Sender Signature"

3. Copy/paste the "Server API token" under "Credentials" in your .env file (example below)

   -POSTMARK_API_TOKEN=
   +POSTMARK_API_TOKEN=ac6657eb-2732-4cfd-915b-912b1b10beb1

4. Modify the SEND_EMAIL variable in .env from false to true

Favicon and Touch Icon Configuration

You can customize the favicon and touch icons – just generate a new set at https://realfavicongenerator.net and overwrite the existing in the assets folder.

Just make sure that any relative paths match up in the assets/browserconfig.xml and assets/manifest.json files.

Authentication Methods

We use Lad's auth package under the hood; so if you want to configure authentication providers you'll want to read more or contribute to @ladjs/auth.

Google Auth

In order to add Google sign-in to your app (so users can log in with their Google account):

1. Go to https://console.developers.google.com – Create a project (and fill out your project information – if you need a 120x120px default image, you can use this one with a CDN path of https://cdn.rawgit.com/ladjs/lad/82d38d64/media/lad-120x120.png

2. Under your newly created project, go to Credentials – Create credentials – OAuth client ID – Web application

3. Set "Authorized JavaScript origins" to http://yourdomain.com (replace with your domain) and also http://localhost:3000 (for local development)

4. Set "Authorized redirect URIs" to http://yourdomain.com/auth/google/ok (again, replace with your domain) and also http://localhost:3000/auth/google/ok (again, for local development)

5. Copy and paste the newly created key pair for respective properties in your .env file (example below)

   -GOOGLE_CLIENT_ID=
   +GOOGLE_CLIENT_ID=424623312719-73vn8vb4tmh8nht96q7vdbn3mc9pd63a.apps.googleusercontent.com
   -GOOGLE_CLIENT_SECRET=
   +GOOGLE_CLIENT_SECRET=Oys6WrHleTOksqXTbEY_yi07

6. In .env, make sure that AUTH_GOOGLE_ENABLED=true to enable this authentication method.

Translation Configuration

1. Go to https://console.developers.google.com
2. Enable the Google Translate API
3. Copy your API key and set it as the environment variable GOOGLE_TRANSLATE_KEY=******

Continuous Integration and Code Coverage

We strongly recommend that you use SemaphoreCI for continuous integration and Codecov for code coverage.

Here are the simple steps required to setup SemaphoreCI with Codecov:

1. Go to SemaphoreCI and sign up for a free account

2. Once your repository is pushed to GitHub, add it as a project on SemaphoreCI

3. Configure your project on SemaphoreCI with the following build settings:

   Replace npm with yarn if you're using yarn as your package manager

   • Language: JavaScript
   • Node.js version: 8.9.0+ (latest LTS)

     Note you can also add to Setup the script nvm install latest to install latest version if SemaphoreCI does not provide it from the drop-down

   • Setup: npm install
   • Job 1: npm run test-coverage
   • After job: npm run coverage

4. Go to Codecov and sign up for a free account

5. Add your project on Codecov and copy to your clipboard the token

6. Go to SemaphoreCI's Project Settings for your project and add CODECOV_TOKEN as an environment variable (with the contents from your clipboard)

7. Run a test build ("Rebuild last revision") on SemaphoreCI and check to make sure your code coverage report uploads properly on Codecov

8. Ensure your README.md file has the build status and code coverage badges rendered properly (you will need to use a different badge link from each provider if your GitHub repository is private)

Amazon S3 and CloudFront Asset Setup

In order for your assets to get properly served in a production environment, you'll need to configure AWS:

1. Go to https://console.aws.amazon.com/iam/home#security_credential ‐ Access Keys – Create New Access Key

2. Copy and paste the newly created key pair for respective properties in your .env file (example below)

   -AWS_IAM_KEY=
   +AWS_IAM_KEY=AKIAJMH22P6W674YFC7Q
   -AWS_IAM_SECRET=
   +AWS_IAM_SECRET=9MpR1FOXwPEtPlrlU5WbHjnz2KDcKWSUcB+C5CpS

3. Enable your API by clicking on Overview and then clicking the Enable button

4. Go to https://console.aws.amazon.com/s3/home – Create Bucket

5. Create a bucket and copy/paste its name for the property in .env (example below)

   -AWS_S3_BUCKET=
   +AWS_S3_BUCKET=lad-development

6. Go to https://console.aws.amazon.com/cloudfront/home – Create Distribution – Get Started

7. Set "Origin Domain Name" equal to your S3 bucket name (their autocomplete drop-down will help you find it)

8. Leave the remaining defaults as is (some fields might be blank, this is OK)

9. Copy/paste the newly created Distribution ID and Domain Name for respective properties in your .env file (example below)

   -AWS_CF_DI=
   +AWS_CF_DI=E2IBEULE9QOPVE
   -AWS_CF_DOMAIN=
   +AWS_CF_DOMAIN=d36aditw73gdrz.cloudfront.net

Tutorials

• Writing Your App
• Continous Integration and Deployment

Community

• Follow us on Twitter
• Join our Slack channel
• Subscribe to our Twitch channel
• Visit Koa's Community section.
• Join Mongoose's Slack channel

Architecture

The following bash output is the directory structure and organization of Lad:

tree template -I "build|node_modules|coverage|test"

template
├── LICENSE
├── README
├── api.js
├── app
│   ├── controllers
│   │   ├── api
│   │   │   ├── index.js
│   │   │   └── v1
│   │   │       ├── index.js
│   │   │       ├── log.js
│   │   │       └── users.js
│   │   ├── index.js
│   │   └── web
│   │       ├── admin
│   │       │   ├── index.js
│   │       │   └── users.js
│   │       ├── auth.js
│   │       ├── index.js
│   │       ├── my-account.js
│   │       └── support.js
│   ├── models
│   │   ├── index.js
│   │   ├── inquiry.js
│   │   └── user.js
│   └── views
│       ├── 404.pug
│       ├── 500.pug
│       ├── _breadcrumbs.pug
│       ├── _footer.pug
│       ├── _nav.pug
│       ├── _pagination.pug
│       ├── _register-or-login.pug
│       ├── about.pug
│       ├── admin
│       │   ├── index.pug
│       │   └── users
│       │       ├── index.pug
│       │       └── retrieve.pug
│       ├── dashboard
│       │   └── index.pug
│       ├── forgot-password.pug
│       ├── home.pug
│       ├── layout.pug
│       ├── my-account
│       │   ├── index.pug
│       │   └── security.pug
│       ├── privacy.pug
│       ├── register-or-login.pug
│       ├── reset-password.pug
│       ├── spinner
│       │   ├── 1.pug
│       │   ├── 10.pug
│       │   ├── 11.pug
│       │   ├── 2.pug
│       │   ├── 3.pug
│       │   ├── 4.pug
│       │   ├── 5.pug
│       │   ├── 6.pug
│       │   ├── 7.pug
│       │   ├── 8.pug
│       │   ├── 9.pug
│       │   └── spinner.pug
│       ├── support.pug
│       └── terms.pug
├── assets
│   ├── browserconfig.xml
│   ├── css
│   │   ├── _custom.scss
│   │   ├── _email.scss
│   │   ├── _hljs.scss
│   │   ├── _sticky-footer.scss
│   │   ├── _variables.scss
│   │   └── app.scss
│   ├── fonts
│   ├── img
│   │   ├── android-chrome-192x192.png
│   │   ├── android-chrome-384x384.png
│   │   ├── apple-touch-icon.png
│   │   ├── favicon-16x16.png
│   │   ├── favicon-32x32.png
│   │   ├── favicon.ico
│   │   ├── logo-square.svg
│   │   ├── mstile-150x150.png
│   │   ├── social.png
│   │   └── twitter.png
│   ├── js
│   │   ├── core.js
│   │   ├── logger.js
│   │   └── uncaught.js
│   ├── manifest.json
│   └── robots.txt
├── bull.js
├── config
│   ├── env.js
│   ├── index.js
│   ├── meta.js
│   ├── phrases.js
│   ├── polyfills.js
│   └── utilities.js
├── ecosystem.json
├── emails
│   ├── _content.pug
│   ├── _footer.pug
│   ├── _nav.pug
│   ├── inquiry
│   │   ├── html.pug
│   │   └── subject.pug
│   ├── layout.pug
│   ├── reset-password
│   │   ├── html.pug
│   │   └── subject.pug
│   └── welcome
│       ├── html.pug
│       └── subject.pug
├── env
├── gitignore
├── gulpfile.js
├── helpers
│   ├── i18n.js
│   ├── index.js
│   ├── logger.js
│   ├── passport.js
│   └── policies.js
├── index.js
├── locales
│   ├── en.json
│   ├── es.json
│   └── zh.json
├── nodemon.json
├── package-scripts.js
├── package.json
├── proxy.js
├── queues
│   ├── email.js
│   ├── index.js
│   └── mandarin.js
├── routes
│   ├── api
│   │   ├── index.js
│   │   └── v1
│   │       └── index.js
│   ├── index.js
│   └── web
│       ├── admin.js
│       ├── auth.js
│       ├── index.js
│       └── my-account.js
├── template
├── web.js
└── yarn.lock

31 directories, 119 files

Principles

Lad is designed according to these principles:

1. Always be developer-friendly
2. Adhere to MVC, Unix, KISS, DRY, YAGNI and Twelve Factor
3. Target the scrappy, bootstrapped, and ramen-profitable hacker

Related

• lipo - Free image manipulation API service built on top of Sharp
• cabin - Logging and analytics solution for Node.js, Lad, Koa, and Express
• lass - Scaffold a modern boilerplate for Node.js

Contributing

Interesting in contributing to this project or testing early releases?

1. Follow all of the above Requirements

2. You will need to fork and clone this repository locally

3. After forking, follow these steps:

   cd lad
   yarn install
   cd template
   yarn install
   yarn start

If you'd like to preview changes to the README.md file, you can use docute.

yarn global add docute-cli
cd lad
docute ./

Then visit http://localhost:8080 in your browser.

Contributors

Name	Website
Nick Baugh	http://niftylettuce.com

Trademark Notice

Lad, Lass, Cabin, Lipo, and their respective logos are trademarks of Niftylettuce LLC. These trademarks may not be reproduced, distributed, transmitted, or otherwise used, except with the prior written permission of Niftylettuce LLC. If you are seeking permission to use these trademarks, then please contact us.

License

MIT © Nick Baugh