Coder Social home page Coder Social logo

slalombuild / terraform-atmos-accelerator Goto Github PK

View Code? Open in Web Editor NEW
38.0 3.0 8.0 628 KB

An opinionated, multi-cloud, multi-region, best-practice accelerator for Terraform.

License: MIT License

HCL 100.00% Shell 0.01%
accelerator atmos hcl terraform iac platform-engineering

terraform-atmos-accelerator's Introduction

Terraform Accelerator
Terraform (Atmos) Accelerator

An opinionated, multi-cloud, multi-region, best-practice accelerator for Terraform


Originally developed internally by the Platform Engineers at Slalom Build, based on our experience building enterprise-grade Terraform for customers. We decided to open source this accelerator to empower other engineers, encourage contributions and work more collaboratively with customers.

๐ŸŽฏ Project goals

  1. Stop re-inventing the wheel! Create a repository that can be used to quickly provision resources in AWS, Azure and GCP. The approach must be -

    1. scalable - scaling from one small group of resources in one region to many resources, in multiple accounts, in multiple regions should not be difficult or overly complex
    2. maintainable - the structure should be logical, code should be DRY and straightforward, no hacks
    3. extensible - the patterns used should enable others to a) easily extend existing code, and b) create their own implementations
    4. well-tested - no explanation required hopefully!
    5. not just our best practice - follow Terraform best practices as closely as possible. See naming conventions and code structure
  2. Standardize creation of the following basic resources (and more as the need is discovered), with the understanding that they will be further customized for every use case (reference):

    Service type AWS Google Cloud Azure
    Virtual networking VPC Virtual Private Cloud Azure Virtual Network
    DNS Route 53 Cloud DNS Azure DNS
    Virtual machines EC2 Compute Engine Azure Virtual Machines
    Autoscaling AutoScaling Groups Compute Engine AutoScaler Azure Autoscale, Azure Virtual Machine Scale Sets
    Load balancing Application Load Balancing Cloud Load Balancing Azure Load Balancer
    Auditing CloudTrail Cloud Audit Logs Azure Audit Logs
    Monitoring CloudWatch Cloud Monitoring Azure Monitor
    Policy enforcement AWS Config Compute Resource Policy Azure Policy
    Parallel task execution SQS, SNS Cloud Tasks Azure Service Bus, Azure Storage Queues
    Terraform state storage S3 & DynamoDB Cloud Datastore Azure Storage
  3. Create integration tests for every module we create and use

  4. Create a robust CI/CD pipeline for end-to-end deployment and testing of infrastructure code

  5. Build a community of maintainers and contributors for as long as the accelerator remains useful

  6. Manage vulnerabilities to improve the overall security posture of solutions based on the accelerator

๐Ÿค” When to use the terraform accelerator (or not)

This accelerator is best suited for:

  • Managing multiple environments on a medium/large/enterprise scale
  • Multi-region cloud deployments
  • Hybrid (multi-cloud) deployments
  • Terraform automation and collaboration, to enable team autonomy and enable multiple teams to own infrastructure
  • Brown-field development (bring your own landing zones)

When you should pass on this accelerator:

  • If you need to use Terraform Cloud as a back-end (atmos doesn't support it yet)
  • If your terraform is very simple and likely won't grow in complexity

๐Ÿงฌ Project structure

.
โ”œโ”€โ”€ .github                         # GitHub workflows and repo config
โ”œโ”€โ”€ .vscode                         # VSCode settings
โ”œโ”€โ”€ components                     
โ”‚ย ย  โ””โ”€โ”€ terraform                   # Terraform components
โ”‚ย ย      โ”œโ”€โ”€ aws                     
โ”‚ย ย      โ”œโ”€โ”€ azure                   
โ”‚ย ย      โ””โ”€โ”€ gcp                     
โ”œโ”€โ”€ config                          # Atmos configuration files
โ”‚ย ย  โ”œโ”€โ”€ aws                         
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ atmos.yaml
โ”‚ย ย  โ”œโ”€โ”€ azure
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ atmos.yaml
โ”‚ย ย  โ””โ”€โ”€ gcp
โ”‚ย ย      โ””โ”€โ”€ atmos.yaml
โ”œโ”€โ”€ docs                            # Project documentation
โ”œโ”€โ”€ examples                        # Example configs and stacks
โ”‚   โ”œโ”€โ”€ config
โ”‚   โ””โ”€โ”€ stacks
โ””โ”€โ”€ stacks                          # Atmos stack configurations
 ย ย  โ”œโ”€โ”€ aws                         # AWS stack definitions
 ย ย  โ”‚ย ย  โ”œโ”€โ”€ automation
 ย ย  โ”‚ย ย  โ”œโ”€โ”€ catalog                 
 ย ย  โ”‚ย ย  โ”œโ”€โ”€ dev
 ย ย  โ”‚ย ย  โ”œโ”€โ”€ prod
 ย ย  โ”‚ย ย  โ””โ”€โ”€ staging
 ย ย  โ”œโ”€โ”€ azure                       # Azure stack definitions
 ย ย  โ”‚ย ย  โ””โ”€โ”€ dev
 ย ย  โ””โ”€โ”€ gcp                         # GCP stack definitions
  ย      โ””โ”€โ”€ dev

๐Ÿ’ฌ Talk to us!

We host a community chat every Tuesday at 11am (Vancouver/Pacific Time), where we talk about the accelerator, recent and upcoming changes, feature requests, etc. Please join us https://bit.ly/terraform-accelerator-chat

๐Ÿซถ Contributing

Contributions are always welcome! Fork this repository, make your changes, submit a PR. You know how it works.

๐Ÿ› Support

Raise an issue via Github. Or if you're part of the Slalom Build team, contact us internally via our #terraform-accelerator channel.


License

Copyright ยฉ 2023 Slalom LLC.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

terraform-atmos-accelerator's People

Contributors

davenicoll avatar jamengual avatar renovate[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

terraform-atmos-accelerator's Issues

๐Ÿ› [BUG] - `atmos describe stacks` errors for AWS and GCP stacks

Description

When running atmos describe stacks for either the AWS or GCP stacks, I get

The component 'asg/test-component' in the stack 'dev/us-west-2' inherits from 'asg/test-component/defaults' (using 'metadata.inherits'), but 'asg/test-component/defaults' is not defined in any of the config files for the stack 'dev/us-west-2โ€™

The stack configs are messy and need fixing up.

Reproduction steps

cd config/aws
atmos describe stacks

cd config/gcp
atmos describe stacks

Logs

No response

Cloud

AWS, GCP

Tool

atmos

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

  • WARN: RepoCacheS3.read() - failure

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

terraform
components/terraform/aws/acm/main.tf
  • cloudposse/acm-request-certificate/aws 0.18.0
components/terraform/aws/acm/providers.tf
components/terraform/aws/acm/versions.tf
  • aws ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/alb/main.tf
  • cloudposse/alb/aws 1.11.1
components/terraform/aws/alb/providers.tf
components/terraform/aws/alb/versions.tf
  • aws < 6.0.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/apigateway/main.tf
  • cloudposse/api-gateway/aws 0.6.0
  • cloudposse/api-gateway/aws 0.6.0
components/terraform/aws/apigateway/providers.tf
components/terraform/aws/apigateway/versions.tf
  • aws ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/asg/main.tf
  • cloudposse/ec2-autoscale-group/aws 0.39.0
components/terraform/aws/asg/providers.tf
components/terraform/aws/asg/versions.tf
  • aws >= 4.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/automation-roles/main.tf
  • cloudposse/iam-role/aws 0.19.0
  • cloudposse/iam-role/aws 0.19.0
  • cloudposse/iam-role/aws 0.19.0
  • cloudposse/iam-role/aws 0.19.0
  • cloudposse/tfstate-backend/aws 1.4.1
components/terraform/aws/automation-roles/providers.tf
components/terraform/aws/automation-roles/versions.tf
  • aws >= 4.0
  • hashicorp/terraform >= 1.2.9
components/terraform/aws/cloudtrail/main.tf
  • cloudposse/cloudtrail/aws 0.23.0
  • cloudposse/cloudtrail-s3-bucket/aws 0.26.3
components/terraform/aws/cloudtrail/providers.tf
components/terraform/aws/cloudtrail/versions.tf
  • aws >= 2.0
  • local >= 1.2
  • null >= 2.0
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/cloudwatch/main.tf
  • cloudposse/cloudwatch-events/aws 0.7.0
  • cloudposse/sns-topic/aws 0.21.0
components/terraform/aws/cloudwatch/providers.tf
components/terraform/aws/cloudwatch/versions.tf
  • aws >= 2.0
  • local >= 1.2
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/config/main.tf
  • cloudposse/config/aws 1.5.2
  • cloudposse/config-storage/aws 1.0.2
components/terraform/aws/config/providers.tf
components/terraform/aws/config/versions.tf
  • aws >= 2
  • local >= 1.2
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/dns-delegated/acm.tf
  • cloudposse/acm-request-certificate/aws 0.18.0
components/terraform/aws/dns-delegated/main.tf
  • cloudposse/utils/aws 1.4.0
components/terraform/aws/dns-delegated/providers.tf
components/terraform/aws/dns-delegated/versions.tf
  • aws >= 4.9.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/dns-primary/acm.tf
  • cloudposse/acm-request-certificate/aws 0.18.0
components/terraform/aws/dns-primary/providers.tf
components/terraform/aws/dns-primary/versions.tf
  • aws >= 4.9.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/dynamodb/main.tf
  • cloudposse/dynamodb/aws v0.35.1
components/terraform/aws/dynamodb/providers.tf
components/terraform/aws/dynamodb/versions.tf
  • aws >= 4.0.0
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/ec2/main.tf
  • cloudposse/ec2-instance/aws 1.4.0
components/terraform/aws/ec2/providers.tf
components/terraform/aws/ec2/versions.tf
  • aws >= 2.0
  • null >= 2.0
  • hashicorp/terraform >= 1.0
components/terraform/aws/ecs-service/main.tf
  • cloudposse/label/null 0.25.0
  • cloudposse/alb-ingress/aws 0.28.0
  • cloudposse/ecs-container-definition/aws 0.61.1
  • cloudposse/ecs-alb-service-task/aws 0.74.0
  • cloudposse/ecs-cloudwatch-autoscaling/aws 0.7.3
  • cloudposse/label/null 0.25.0
  • cloudposse/cloudwatch-logs/aws 0.6.8
  • cloudposse/label/null 0.25.0
components/terraform/aws/ecs-service/providers.tf
components/terraform/aws/ecs-service/versions.tf
  • aws ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/ecs/main.tf
  • cloudposse/alb/aws 1.11.1
  • cloudposse/label/null 0.25.0
components/terraform/aws/ecs/providers.tf
components/terraform/aws/ecs/versions.tf
  • aws < 6.0.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/global-accelerator/main.tf
  • cloudposse/global-accelerator/aws 0.6.0
  • cloudposse/global-accelerator/aws 0.6.0
  • cloudposse/s3-bucket/aws 4.2.0
components/terraform/aws/global-accelerator/providers.tf
components/terraform/aws/global-accelerator/versions.tf
  • aws >= 2.0
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/guardduty/main.tf
  • cloudposse/guardduty/aws 0.5.0
components/terraform/aws/guardduty/providers.tf
components/terraform/aws/guardduty/versions.tf
  • aws >= 3.0
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/kms/main.tf
  • cloudposse/kms-key/aws 0.12.2
components/terraform/aws/kms/providers.tf
components/terraform/aws/kms/versions.tf
  • aws >=3.64.0
  • hashicorp/terraform >= 0.13
components/terraform/aws/lambda/main.tf
  • cloudposse/lambda-function/aws 0.5.3
  • cloudposse/s3-bucket/aws 4.2.0
components/terraform/aws/lambda/providers.tf
components/terraform/aws/lambda/versions.tf
  • aws >= 4.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/macie/main.tf
  • cloudposse/macie/aws 0.1.3
components/terraform/aws/macie/providers.tf
components/terraform/aws/macie/versions.tf
  • aws >= 3
  • hashicorp/terraform >= 0.15.0
components/terraform/aws/s3/main.tf
  • cloudposse/s3-bucket/aws 4.2.0
components/terraform/aws/s3/providers.tf
components/terraform/aws/s3/versions.tf
  • aws < 6.0.0
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/securityhub/main.tf
  • cloudposse/security-hub/aws 0.12.1
components/terraform/aws/securityhub/providers.tf
components/terraform/aws/securityhub/versions.tf
  • aws >= 2
  • local >= 1.2
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/sns/main.tf
  • cloudposse/sns-topic/aws 0.21.0
components/terraform/aws/sns/providers.tf
components/terraform/aws/sns/versions.tf
  • aws >= 3.0
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/sqs/providers.tf
components/terraform/aws/sqs/versions.tf
  • aws >= 3.63
  • hashicorp/terraform >= 0.13.1
components/terraform/aws/tfstate-backend/main.tf
  • cloudposse/tfstate-backend/aws 1.4.1
components/terraform/aws/tfstate-backend/providers.tf
components/terraform/aws/tfstate-backend/versions.tf
  • aws ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/aws/vpc/flow-logs.tf
  • cloudposse/vpc-flow-logs-s3-bucket/aws 1.1.0
components/terraform/aws/vpc/main.tf
  • cloudposse/dynamic-subnets/aws 2.4.2
  • cloudposse/vpc/aws 2.2.0
components/terraform/aws/vpc/providers.tf
components/terraform/aws/vpc/versions.tf
  • aws < 6.0.0
  • hashicorp/terraform >= 0.13.0
components/terraform/aws/vpc/vpc-endpoints.tf
  • cloudposse/label/null 0.25.0
  • cloudposse/label/null 0.25.0
  • cloudposse/label/null 0.25.0
  • cloudposse/label/null 0.25.0
  • cloudposse/label/null 0.25.0
  • cloudposse/label/null 0.25.0
  • cloudposse/label/null 0.25.0
  • cloudposse/label/null 0.25.0
  • cloudposse/label/null 0.25.0
  • cloudposse/vpc/aws v2.2.0
components/terraform/aws/waf/main.tf
  • cloudposse/waf/aws 1.4.0
components/terraform/aws/waf/providers.tf
components/terraform/aws/waf/versions.tf
  • aws >= 2.0
  • hashicorp/terraform >= 0.13.0
components/terraform/azure/aks/main.tf
  • Azure/aks/azurerm 7.5.0
  • Azure/naming/azurerm 0.4.1
components/terraform/azure/aks/providers.tf
components/terraform/azure/aks/versions.tf
  • azurerm >=3.51.0, < 4.0
  • http > 3.2.0, < 4.0
  • random 3.6.1
  • hashicorp/terraform >=1.3
components/terraform/azure/example_providers.tf
  • azurerm 3.100.0
  • hashicorp/terraform >1.3
components/terraform/azure/keyvault/main.tf
  • Azure/naming/azurerm 0.4.1
components/terraform/azure/keyvault/versions.tf
  • azurerm ~> 3.39
  • hashicorp/terraform >= 1.3
components/terraform/azure/tfstate-backend/naming.tf
  • Azure/naming/azurerm 0.4.1
components/terraform/azure/tfstate-backend/providers.tf
components/terraform/azure/tfstate-backend/versions.tf
  • azurerm >=3.63.0, < 4.0
  • http > 3.2.0, < 4.0
  • random 3.6.1
  • hashicorp/terraform >=1.3
components/terraform/gcp/cloud-function/main.tf
  • GoogleCloudPlatform/cloud-functions/google 0.4.1
components/terraform/gcp/cloud-function/providers.tf
components/terraform/gcp/cloud-function/versions.tf
  • google < 5.26
  • hashicorp/terraform >= 1.0.0
components/terraform/gcp/dns/main.tf
  • terraform-google-modules/cloud-dns/google 5.2.0
components/terraform/gcp/dns/providers.tf
components/terraform/gcp/dns/versions.tf
  • google ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/gcp/folder/providers.tf
components/terraform/gcp/folder/versions.tf
  • google ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/gcp/gcs-bucket/providers.tf
components/terraform/gcp/gcs-bucket/versions.tf
  • google ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/gcp/kms/main.tf
  • terraform-google-modules/kms/google ~> 2.3.0
components/terraform/gcp/kms/providers.tf
components/terraform/gcp/kms/versions.tf
  • google ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/gcp/net-ilb-l4/main.tf
  • terraform-google-modules/lb/google ~> 4.1.0
components/terraform/gcp/net-ilb-l4/providers.tf
components/terraform/gcp/net-ilb-l4/versions.tf
  • google ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/gcp/net-ilb-l7/main.tf
  • GoogleCloudPlatform/lb-http/google 10.2.0
components/terraform/gcp/net-ilb-l7/providers.tf
components/terraform/gcp/net-ilb-l7/versions.tf
  • google ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/gcp/net-vpc/versions.tf
  • google ~> 5.0
  • hashicorp/terraform >= 1.0.0
components/terraform/gcp/network/main.tf
  • terraform-google-modules/network/google ~> 9.1.0
components/terraform/gcp/network/nat.tf
  • terraform-google-modules/cloud-nat/google ~> 5.0.0
  • terraform-google-modules/cloud-router/google ~> 6.0.0
components/terraform/gcp/network/providers.tf
components/terraform/gcp/network/versions.tf
  • google < 5.26
  • hashicorp/terraform >=1.3
components/terraform/gcp/project/providers.tf
components/terraform/gcp/project/versions.tf
  • google ~> 5.0
  • hashicorp/terraform >= 1.0.0

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.