simonmcc / cfgmgmtcmp-2019-iac-ci-cd

cfgmgmtcmp-2019-iac-ci-cd

Real World AWS Packer & Terraform and Inspec Pipeline

This is a working Jenkinsfile Multibranch Pipeline for building AWS AMI's images & deploying EC2 instances based on the AMI with terraform (in both a per-branch test stack, destroyed by the pipeline, and maintaining production from the master branch).

The pipeline uses 2 docker images to run jobs, most of the work is done in hashicorp-pipeline, which contains packer, terraform, aws cli & some other needed binaries (jq, perl), the other docker image used is chef/inspec, which is used to test the stack built during non-master pipeline executions.

Features

• terraform fmt -check=true -diff=true used to check terraform config files
• packer validate used to validate packer config files
• base and application AMI's built and tagged with SHA1 of the base/base.json and app/app.json, to help prevent unnecessary AMI builds
• Automatically configures terraform remote state (S3 & DynamoDB)
• terraform workspaces used to store per-branch terraform state
• parameterized terraform build using per-branch tfvars
• chef/inspec used to validate AWS resources
• terraform plan with stash of plan, human approval, terraform apply workflow for master/production changes

What next to use this in your own project

• Add demo-aws-creds credential set (uses AWS Credentials plugin)
• Update terraform to actually deploy an EC2 instance (for demo purposes, this only builds a VPC, so zero cost & quick cycle time for pipeline experimentation, trivial to add an EC2 instance)
• Use AWS Auto Scaling group to replace running EC2 instances with EC2 instances built from the master branch app AMI
• Use AMI tags to ensure app AMI matches the branch (Use the SHA1 of app/app.json to search for the AMI in terraform, requires some tfvars templating/injection)
• Extend chef/inspec stage to test AWS EC2
• Add a Selenium or other web test stage

If you want a simple dockerized Jenkins setup to host this, I used Jenkins 201, there's also example Job DSL here.

Further info

Some of the scripts in this pipeline (build.sh & common.sh) are based on an incomplete packer/AMI example from CircleCI.

• http://cavaliercoder.com/blog/finding-the-latest-centos-ami.html

aws ec2 deregister-image --image-id ami-4fa54026

HMM