terramate-io / terramate

Terramate CLI is an open-source Infrastructure as Code (IaC) Orchestration and Code Generation tool for Terraform, OpenTofu and Terragrunt.

Terramate

🚀 Getting Started | 📖 Documentation | 💻 Playground

What is Terramate CLI?

Terramate CLI is an open-source Infrastructure as Code (IaC) code generation and orchestration tool for Terraform, OpenTofu and Terragrunt. With Terramate, you can:

(a) Simplify complex codebases to make your work more productive and reliable. Break down large state files with stacks to reduce run-time and blast radius and keep them DRY with native code generation.

(b) Automate and orchestrate Terraform, OpenTofu and Terragrunt in any CI/CD using Pull Request automation, GitOps blueprints, and workflow tooling with zero-config orchestration and change detection.

(c) Provide the foundation to later leverage Terramate Cloud, a management layer that gives you visibility, control, and actionable insights into your infra.

Installation

With brew:

brew install terramate

With Go:

go install github.com/terramate-io/terramate/cmd/...@latest

For other installation methods, please see the documentation.

Getting Started

The fastest way to get started with Terramate is our getting started guide.

Why simplify complex codebases?

As Terraform and OpenTofu offer no standard pattern on how to organize code efficiently, projects quickly sprawl out of control. The consequences are code complexity, long-running pipelines, large blast radius, drift, and poor governance.

Terramate CLI enables a new approach to eliminating the root cause of these consequences: using a modular and efficient approach called Terramate Stacks. A stack is a combination of infrastructure code, state and configuration, which can be nested to split state further.

Organizationally, a stack is a way to establish an ownership model. By consistently sizing stacks into isolated units that your team members can quickly understand, assuming ownership of your infra becomes a lot simpler. Terramate CLI supports Terraform, OpenTofu and Terragrunt, and can be run locally as well as in your existing VCS (Github, Gitlab) CI/CD.

Terramate CLI is designed to work especially well with Terramate Cloud, which provides an IaC management layer and offers a generous free tier.

Features

• Orchestration: Run any command and configurable workflows in stacks with unlimited concurrency.
• Change Detection: Only execute stacks that contain changes. Allows to detect changes in referenced Terraform and OpenTofu modules as well as Terragrunt dependencies.
• Code Generation: Generate code such HCL, JSON and YAML to keep your stacks DRY (Don't repeat yourself). Comes with support for global variables and functions.
• Automation Blueprints: Pre-configured GitOps workflows for GitHub, GitLab, BitBucket and Atlantis to enable Pull Automation with plan previews in your existing CI/CD.
• Drift Management: Detect and reconcile drift with scheduled workflows.
• Observability, Visibility and Insights: Provides actionable insights and observability into your stacks, deployments, and resources.

Terramate Cloud

Terramate Cloud provides you with the best IaC management experience possible. It helps you orchestrate your stacks in your existing CI/CD platform, such as GitHub Actions, GitLab and Bitbucket Pipelines, so you don't need to buy yet another CI/CD vendor.

It also provides a powerful dashboard that adds observability and insights, deployment metrics (e.g., DORA), notifications, drift management, asset management, and more for your stacks to provide the developer experience and infrastructure to build, scale and observe all your infrastructure managed with IaC.

Join the Community

• Join our Discord
• Contact us via email at [email protected]
• Follow us on X
• Follow us on LinkedIn

Additional Resources

• Documentation
• Playground
• Getting started guide
• Terramate Blog

Reporting Bugs, Requesting Features, or Contributing to Terramate

Want to report a bug or request a feature? Open an issue

Interested in contributing to Terramate? Check out our Contribution Guide

License

See the LICENSE file for licensing information.

Terramate

Terramate is a CNCF and Linux Foundation silver member.

terramate's Issues

[FEATURE] Indicate source config file for generate_hcl and generate_file

Is your feature request related to a problem? Please describe.
currently ( version 0.2.16) the file generated by generate_hcl and generate_file have the following record:
// TERRAMATE: GENERATED AUTOMATICALLY DO NOT EDIT
It may be beneficial during troubleshooting to know what source config was used

Describe the solution you'd like
the output changed to:

// TERRAMATE: GENERATED AUTOMATICALLY DO NOT EDIT
//source config: /<path to>/<config file>

Describe alternatives you've considered
none I can think of.

[FEATURE] Introduce "stack move" command

Is your feature request related to a problem? Please describe.
to manage dependencies efficiently, it may be useful to introduce a "stack move" command.

Describe the solution you'd like
tm stack move <path to source stack| uniquely identifiable tags> <path to destination>
as part of the move:

1. validate stack's dependencies
2. update the 'before/after' in all other stacks according to this stack's new location in an interactive session.
3. the move, as well as config updates, should be an atomic operation.

The "move" can also serve as a "rename" with the same functionality.

Describe alternatives you've considered
potential alternative

[FEATURE] Git worktree

Is your feature request related to a problem? Please describe.
apologies for the question.
But does terramate work/support git's worktree ?

Describe the solution you'd like
worktree is used to checkout a branch, completed all the work, and deploy from this branch.

Describe alternatives you've considered
checking out a branch into the same directory is error-prone since I have a few branches at the same time.

Additional context
none at the moment.

feat: add basic backend configuration code generation

As defined here: https://github.com/mineiros-io/terrastack/blob/main/docs/backend-config.md
Focused on just the basic usage/feature of generating code inside multiple stacks, but no metadata/globals yet (hence "basic").

[BUG] `... ' modifier is lost during generate_hcl, how to pass it to generated HCL

Describe the bug

I'm trying to use "generate_hcl" and ... modifier is lost. Which is expected if the HCL is evaluated but I'm unsure how to pass it down to generated HCL?

I need to use this because I've a nested for and I'd like to flatten and merge the final result.

Do I have to use generate_file in this case?

Ideally, I'd love to do the whole preprocessing in globals block and pass the final output to generated HCL but merge and other functions are not working there.

To Reproduce

Steps to reproduce the behavior:

1. Given context:

generate_hcl "_terramate_deployments.gen.tf" {
  content {
    locals {
      app_env_deployments = global.app_env_deployments
      app_envs            = global.app_envs

      deployments = merge([
          for app_env in local.app_envs: {
            for name in local.app_env_deployments:
              "${name}-${app_env}" => {
                env  = app_env
                name = name
              }
          }
      ]...)
    }
}

3. Run command: terramate generate
4. See error

Expected behavior

... is present in generated output

Log Output

Environment (please complete the following information):

• OS: [e.g. Linux] Linux
• OS Version [e.g. Ubuntu 20.04] Ubuntu 22.04
• Git Version [e.g. 2.35.1] (git --version): 2.30
• Terramate Version [e.g. 0.1.0] (terramate --version): 0.2.14

Additional context

none

[BUG] Failure to Evaluate 'Generate HCL' Configurations - Problem with Maps?

Description

I upgraded from 0.2.9 to 0.2.13. Terramate suddenly seems unhappy with some of our 'generate_hcl' content blocks, but as far as I can tell they're perfectly valid. It's possible, though, that there's something we're doing incorrectly.

I have a few templates that are throwing the same error. I've noticed it always appears to struggle with a map.
They all say the same thing. As a very rough guess with little-to-no information, Terramate is no longer evaluating maps properly. But again, that's mostly speculation on my part.

Steps to Reproduce

Example Terramate template.

generate_hcl "_dns_vpc_associations.tf" {
  content {
    module "dns_vpc_associations_strateos" {
      source    = "../../../modules/aws-dns-vpc-associations"
      providers = {
        aws.vpc         = aws.primary
        aws.hosted_zone = aws.primary
      }
      vpc_associations = {
        private_default_aws_vpn = {
          vpc_id  = global.default_aws_vpn_client.vpc_id
          zone_id = module.dns.private_zone_id
        }
        private_internal_default_aws_vpn = {
          vpc_id  = global.default_aws_vpn_client.vpc_id
          zone_id = module.dns.private_internal_zone_id
        }
      }
    }
  }
}

Upon terramate generate command, I receive an error:

- /path/to/file/default_dns_vpc_associations.tm.hcl:5,19-8,8: evaluating content block: generate_hcl "_dns_vpc_associations.tf": partial evaluation failed: /path/to/file/default_dns_vpc_associations.tm.hcl:6,9-16: If this expression is intended to be a reference, wrap it in parentheses. If it's instead intended as a literal name containing periods, wrap it in quotes to create a string literal.

The error then disappears if I change my template as such:

generate_hcl "_dns_vpc_associations.tf" {
  content {
    module "dns_vpc_associations_strateos" {
      source    = "../../../modules/aws-dns-vpc-associations"
      providers = {
        (aws.vpc)         = aws.primary
        (aws.hosted_zone) = aws.primary
      }
      vpc_associations = {
        private_default_aws_vpn = {
          vpc_id  = global.default_aws_vpn_client.vpc_id
          zone_id = module.dns.private_zone_id
        }
        private_internal_default_aws_vpn = {
          vpc_id  = global.default_aws_vpn_client.vpc_id
          zone_id = module.dns.private_internal_zone_id
        }
      }
    }
  }
}

Another example:

generate_hcl "_main.tf" {
  lets {
    default_node_group_name = tm_try(global.eks.node_groups.default_group_name, "${global.env}-prim-default")
  }
  content {
    module "eks_cluster" {
      source = global.module_sources.eks.source
      version = global.module_sources.eks.version

      providers = { aws = aws.primary }
      eks_managed_node_groups = merge({ let.default_node_group_name = local.node_groups.default }, local.node_groups.additional)
      # More Terraform that is not relevant here...
    }
  }
}

I see the exact same error. I can fix it by changing this line and adding parentheses around let.default_node_group_name:

eks_managed_node_groups = merge({ (let.default_node_group_name) = local.node_groups.default }, local.node_groups.additional)

Expected Behavior

I expected Terramate to properly parse my generate_hcl configuration and generate the Terraform code as previous versions of Terramate did.

Environment

• OS.: Linux
• OS Version: Ubuntu 22.04
• Git Version: 2.34.1
• Terramate Version: 0.2.13

[FEATURE] Json stack list output

Is your feature request related to a problem? Please describe.

It would be very useful if terramate could output a generic (and good coverage) JSON output of stacks, their metadata, their modules etc. Then this output can be output to a json file and further transformed any way the operator wants. (First use for me right now is transforming it using jsonnet into an atlantis.yaml file).

Describe the solution you'd like

Json output, such as:

[
  {
    "desc": "Terraform Base Roles for Int account",
    "hostpath": "/home/luser/dev/repo/terraform/infra/awesomest-infra/int/terraform-base-int",
    "id": "",
    "modules": [
      {
        "hostpath": "/home/luser/dev/repo/terraform/modules/awesomest-infra/terraform-account",
        "stackrelpath": "../../../../modules/awesomest-infra/terraform-account",
        "relpath": "../../../../modules/awesomest-infra/terraform-account",
        "modules": [
          {
            "hostpath": "/home/luser/dev/repo/terraform/modules/awesomest-infra/infra-lookup",
            "stackrelpath": "../../../../modules/awesomest-infra/infra-lookup",
            "relpath": "../infra-lookup"
          }
        ]
      },
      {
        "hostpath": "/home/luser/dev/repo/terraform/modules/awesomest-infra/infra-lookup",
        "stackrelpath": "../../../../modules/awesomest-infra/infra-lookup",
        "relpath": "../infra-lookup"
      }
    ],
    "name": "terraform-base-int",
    "path": "/terraform/infra/awesomest-infra/int/terraform-base-int",
    "pathbase": "terraform-base-int",
    "relpath": "terraform/infra/awesomest-infra/int/terraform-base-int",
    "relpathtoroot": "../../../../..",
    "watch": [
      "/terraform/some/watch/config.cfg",
    ],
    "relwatch": [
      "../../../../some/watch/config.cfg",
    ],
  }
]

Additional context

Given a jsonnet file like the following:

atlantis.jsonnet:

/*
  Source file for atlantis.yaml
  Generate with `make atlantis.yaml`.
*/

local terramateStacks = import '.terramate-stacks.json';

local terramateStack(stack) = {
  dir: stack.relpath,
  autoplan: {
    when_modified:
      (  // Need grouping parentheses here because of the 'else [] +'
        if std.objectHas(stack, 'modules')
        then [
          m.stackrelpath
          for m in stack.modules
        ] else []
      ) +
      (
        if std.objectHas(stack, 'relwatch')
        then
          stack.relwatch
        else []
      ),
  },
  apply_requirements: [
    'mergeable',
    'approved',
  ],
};

local atlantis = {
  version: 3,
  projects: [
    terramateStack(stack)
    for stack in terramateStacks
  ],
};

std.manifestYamlDoc(atlantis, quote_keys=false)

could parse output above into an atlantis.yaml:

# ...
  autoplan:
    when_modified:
    - "../../../../modules/dice-infra/terraform-account"
    - "../../../../modules/dice-infra/infra-lookup"
  dir: "terraform/infra/dice-infra/int/terraform-base-int"
# ...

feat: fail early if user's environment do not match expectations.

At the moment terrastack requires (by default) a repository with an origin remote set with a main branch. We should fail with an explanation instead of reporting weird git errors.

Create Stack object

From @katcipis comment:

Hmm it does seem cool to have a single Stack object that knows how to do single stack operations, like running a command on that stack, etc, and then just having something that manages a list of stacks... kinda drafty but thinking on:

stacks, err := terrastack.LoadStacks(projectRoot)
stacks.List() // returns []terrastack.Stack
stacks.Changed() // Same as list, but filtering only changed stacks
stacks[0].Run() // running something on a stack
stacks[0].Do() //or doing something else with the stack

// maybe the individual stack also knows if it has changes on it, so we could have something like:
stacks[0].Changed()
// which would be used on the overall list all changed stacks logic.

The overall idea is for the manager to be less of an actual manager (having too much logic/details on stacks) and more like a StackSet (or just Stacks).

Originally posted by @katcipis in #8 (comment)

[FEATURE] Support `condition` in `import` blocks

Describe the solution you'd like

Would like to see conditional imports. Example:

import {
  condition = global.environment == "staging"
  source = "/environments/staging/config.tm.hcl"
}

This could be leveraged to pull in specific globals based on some metadata etc.

[BUG] Terramate Run Change Detection Fails, but List Change Detection Succeeds

Describe the bug
When trying to use Terramate (v0.1.16) to run a command over stacks with automatic change detection the command errors out with the following message:

➜  terramate-exploration git:(feature/inital) ✗ terramate run --changed --git-change-base HEAD^ -- ls
2022-07-23T12:22:38Z FTL computing selected stacks error="listing changed stacks error: getting revision \"\": failed to exec: /usr/bin/git rev-parse  : stderr=\"fatal: ambiguous argument '': unknown revision or path not in the working tree.\\nUse '--' to separate paths from revisions, like this:\\n'git <command> [<revision>...] -- [<file>...]'\\n\"" action=runOnStacks() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration

However, when running the Terramate list command the automatic change detection works as expected:

➜  terramate-exploration git:(feature/inital) ✗ terramate list --changed --git-change-base HEAD^
terraform/stacks/access/boundary
terraform/stacks/infrastructure/kubernetes/cluster
terraform/stacks/infrastructure/kubernetes/ecosystem/karpenter

To Reproduce

I've never been able to get terramate run to work with automatic change detection, so I don't think there are really any steps to reproduce (other than setting up my exact environment for testing). I started a simple project, made some changes then tested it and given that terramate list --changed works I feel that the issue lies in the run command and not the environment?

Expected behavior
I expect the terramate run command to automatically determine the changed stacks in the same fashion that terramate list does.

Log Output

➜  terramate-exploration git:(feature/inital) ✗ terramate --log-level trace run --changed --git-change-base HEAD^ -- ls
2022-07-23T12:23:22Z TRC Running in directory action=newCli() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create new git wrapper. action=lookupProject() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Parse Terramate config. action=TryLoadRootConfig() path=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Parsing configuration files action=ParseDir() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC listing files action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC looking for Terramate files action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC ignoring dotfile action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration entryName=.git
2022-07-23T12:23:22Z TRC ignoring dir action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration entryName=terraform
2022-07-23T12:23:22Z TRC Found Terramate file action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration entryName=terramate.tm.hcl
2022-07-23T12:23:22Z TRC Reading config file. action=parser.AddDir() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration file=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration/terramate.tm.hcl
2022-07-23T12:23:22Z TRC file added action=parser.AddDir() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Range over blocks. action=filterBlocksByType()
2022-07-23T12:23:22Z TRC checking for top-level attributes. action=parseTerramateSchema() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Range over unmerged blocks. action=parseTerramateSchema() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Range over terramate block attributes. action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Parsing  attribute 'required_version'. action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Parse terramate sub blocks action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Found config block. action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Parse root config. action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Range over block attributes. action=parseRootConfig()
2022-07-23T12:23:22Z TRC Type is 'git' action=parseRootConfig()
2022-07-23T12:23:22Z TRC Parse git config. action=parseRootConfig()
2022-07-23T12:23:22Z TRC Range over block attributes. action=parseGitConfig()
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=check_remote
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=check_uncommitted
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=check_untracked
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=default_branch
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=default_remote
2022-07-23T12:23:22Z TRC Type is 'run' action=parseRootConfig()
2022-07-23T12:23:22Z TRC Parse run config. action=parseRootConfig()
2022-07-23T12:23:22Z TRC Checking run.env block action=parseRunConfig()
2022-07-23T12:23:22Z DBG Create new git wrapper providing config. action=newGit()
2022-07-23T12:23:22Z TRC Construct new config. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Apply defaults. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Config program path was null. action=applyDefaults() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Look for path 'git'. action=applyDefaults() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Validate git config. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get path program path information. action=validate() path=/usr/bin/git workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get git version. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z DBG Get git version. action=Version() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create cmd to execute action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Append arguments action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Running git command action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC git command executed with success action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get root of git repo. action=lookupProject() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create cmd to execute action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Append arguments action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Running git command action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC git command executed with success action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get absolute path of git directory. action=lookupProject() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Evaluate symbolic links. action=lookupProject() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Load root config. action=lookupProject() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Parsing configuration files action=ParseDir() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC listing files action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC looking for Terramate files action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC ignoring dotfile action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration entryName=.git
2022-07-23T12:23:22Z TRC ignoring dir action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration entryName=terraform
2022-07-23T12:23:22Z TRC Found Terramate file action=listTerramateFiles() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration entryName=terramate.tm.hcl
2022-07-23T12:23:22Z TRC Reading config file. action=parser.AddDir() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration file=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration/terramate.tm.hcl
2022-07-23T12:23:22Z TRC file added action=parser.AddDir() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Range over blocks. action=filterBlocksByType()
2022-07-23T12:23:22Z TRC checking for top-level attributes. action=parseTerramateSchema() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Range over unmerged blocks. action=parseTerramateSchema() dir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Range over terramate block attributes. action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Parsing  attribute 'required_version'. action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Parse terramate sub blocks action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Found config block. action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Parse root config. action=parseTerramateBlock
2022-07-23T12:23:22Z TRC Range over block attributes. action=parseRootConfig()
2022-07-23T12:23:22Z TRC Type is 'git' action=parseRootConfig()
2022-07-23T12:23:22Z TRC Parse git config. action=parseRootConfig()
2022-07-23T12:23:22Z TRC Range over block attributes. action=parseGitConfig()
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=check_remote
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=check_uncommitted
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=check_untracked
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=default_branch
2022-07-23T12:23:22Z TRC setting attribute on config action=parseGitConfig() attribute=default_remote
2022-07-23T12:23:22Z TRC Type is 'run' action=parseRootConfig()
2022-07-23T12:23:22Z TRC Parse run config. action=parseRootConfig()
2022-07-23T12:23:22Z TRC Checking run.env block action=parseRunConfig()
2022-07-23T12:23:22Z TRC Set defaults from parsed command line arguments. action=newCli() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z DBG Set defaults. action=setDefaults() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC checking if terramate version satisfies project constraint action=cli.checkVersion() root=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC parsing version constraint constraint="~> 0.1.11, < 0.2" version=0.1.16
2022-07-23T12:23:22Z TRC parsing terramate version constraint="~> 0.1.11, < 0.2" version=0.1.16
2022-07-23T12:23:22Z TRC checking version constraint constraint="~> 0.1.11, < 0.2" version=0.1.16
2022-07-23T12:23:22Z DBG Handle command. action=run() cmd="run <cmd>" workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create new terramate manager. action=computeSelectedStacks() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get list of stacks. action=computeSelectedStacks() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC `Changed` flag was set. List changed stacks. action=listStacks() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create git wrapper on project root. action=ListChanged()
2022-07-23T12:23:22Z TRC Construct new config. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Apply defaults. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Config program path was null. action=applyDefaults() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Look for path 'git'. action=applyDefaults() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Validate git config. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get path program path information. action=validate() path=/usr/bin/git workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get git version. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z DBG Get git version. action=Version() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create cmd to execute action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Append arguments action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Running git command action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC git command executed with success action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Check if path is git repo. action=ListChanged()
2022-07-23T12:23:22Z TRC Create cmd to execute action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Append arguments action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Running git command action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC git command executed with success action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z DBG Get list of untracked files. action=checkRepoIsClean()
2022-07-23T12:23:22Z DBG List untracked files. action=ListUntracked() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create cmd to execute action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Append arguments action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Running git command action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC git command executed with success action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Remove empty lines. action=removeEmptyLines()
2022-07-23T12:23:22Z DBG Get list of uncommitted files in dir. action=checkRepoIsClean()
2022-07-23T12:23:22Z DBG List uncommitted files. action=ListUncommitted() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create cmd to execute action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Append arguments action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Running git command action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC git command executed with success action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Remove empty lines. action=removeEmptyLines()
2022-07-23T12:23:22Z DBG List changed files. action=ListChanged()
2022-07-23T12:23:22Z TRC Get dir info. action=listChangedFiles() path=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Check if path is dir. action=listChangedFiles() path=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create git wrapper with dir. action=listChangedFiles() path=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Construct new config. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Apply defaults. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Config program path was null. action=applyDefaults() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Look for path 'git'. action=applyDefaults() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Validate git config. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get path program path information. action=validate() path=/usr/bin/git workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get git version. action=WithConfig() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z DBG Get git version. action=Version() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create cmd to execute action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Append arguments action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Running git command action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC git command executed with success action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Get commit id of git base ref. action=listChangedFiles() path=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Create cmd to execute action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Append arguments action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z TRC Running git command action=Git.exec() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration
2022-07-23T12:23:22Z FTL computing selected stacks error="listing changed stacks error: getting revision \"\": failed to exec: /usr/bin/git rev-parse  : stderr=\"fatal: ambiguous argument '': unknown revision or path not in the working tree.\\nUse '--' to separate paths from revisions, like this:\\n'git <command> [<revision>...] -- [<file>...]'\\n\"" action=runOnStacks() workingDir=/home/daxisaurus/gitlab.com/daxisaurus/terramate-exploration

Environment:

• OS: Ubuntu on WSL2
• OS Version: Ubuntu 20.04
• Git Version: Tried with both 2.25.1 and 2.37.1
• Terramate Version: 0.1.16

Additional context
Given the error message I suspect there's an issue determining the revision to pass to the git rev-parse command that Terramate currently doesn't handle.

[BUG] Fails to Identify Global Value in HEREDOC

Describe the bug
I believe Terramate fails to resolve global values correctly when it's part of a HEREDOC. Fortunately, it's not difficult to work around in the meantime.

To Reproduce

Just as a quick example, I have a stack configuration that includes a global value env...

globals {
  env = "my_environment"
}

Create a main.tm.hcl file for a stack.

generate_hcl "_main.tf" {
  content {
    module "webapp_elastic_search" {
      source                   = "${terramate.stack.path.to_root}/modules/aws-elasticsearch"
      domain_name              = "${global.env}-web"
      enabled                  = true
      cloudwatch_log_enabled   = false
      create_service_link_role = false
      vpc_id                   = data.terraform_remote_state.vpc.outputs.primary_vpc_id
      vpc_options_subnet_ids   = [data.terraform_remote_state.vpc.outputs.primary_vpc_public_subnets[0]]
      vpc_allowed_cidr_blocks  = [data.terraform_remote_state.vpc.outputs.primary_vpc_cidr]
      access_policies          = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "es:*",
      "Principal": {
        "AWS": "*"
    },
      "Effect": "Allow",
      "Resource": "arn:aws:es:us-west-2:${data.aws_caller_identity.current.account_id}:domain/${global.env}-web/*"
    }
  ]
}
POLICY
    }
  }
}

When the file is generated, the HEREDOC looks like this with ${global.env} still there.

// TERRAMATE: GENERATED AUTOMATICALLY DO NOT EDIT

module "webapp_elastic_search" {
  access_policies          = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "es:*",
      "Principal": {
        "AWS": "*"
    },
      "Effect": "Allow",
      "Resource": "arn:aws:es:us-west-2:${data.aws_caller_identity.current.account_id}:domain/${global.env}-web/*"
    }
  ]
}
POLICY
  cloudwatch_log_enabled   = false
  create_service_link_role = false
  domain_name              = "my_environment-web"
  enabled                  = true
  source                   = "../../../modules/aws-elasticsearch"
  vpc_allowed_cidr_blocks = [
    data.terraform_remote_state.vpc.outputs.primary_vpc_cidr,
  ]
  vpc_id = data.terraform_remote_state.vpc.outputs.primary_vpc_id
  vpc_options_subnet_ids = [
    data.terraform_remote_state.vpc.outputs.primary_vpc_public_subnets[0],
  ]
}

Expected behavior
I expected the ${global.env} text to instead be my_environment inside of the HEREDOC. Unless there's a feature I'm not aware of or misunderstand.

Log Output
Terramate generation is successful, so no helpful logs.

Environment (please complete the following information):

• OS: Linux
• OS Version: Ubuntu 21.04
• Git Version: 2.34.1
• Terramate Version: 0.2.8

Additional context
None

bug: running commands on stacks from outside dir breaks

Using:

terrastack run -b <full path> <command>

From outside the stack base dir breaks, while inside the stack base dir it works, here is a small script that reproduces the problem:

#!/bin/bash

set -o errexit
set -o nounset

stack_absolute_path=$(mktemp -d)

cd "${stack_absolute_path}"

mkdir stack1
touch stack1/terrastack

mkdir stack2
touch stack2/terrastack

# Running with absolute path basedir, inside stack, works:
terrastack run -b "${stack_absolute_path}" ls

# Running with absolute path basedir, outside stack, breaks:
cd ..
terrastack run -b "${stack_absolute_path}" ls

Running this script on a empty dir results in:

Running on all stacks:
[stack1] running /bin/ls
terrastack

[stack2] running /bin/ls
terrastack

Running on all stacks:
[stack1] running /bin/ls
2021/11/05 19:41:38 warn: failed to execute command: chdir stack1: no such file or directory

[stack2] running /bin/ls
2021/11/05 19:41:38 warn: failed to execute command: chdir stack2: no such file or directory

2021/11/05 19:41:38 warn: some (2) commands failed

Version:

terrastack version
0.0.1

Which I don't think is precise since we didn't release terrastack yet =P, so here is the git reference used to build (latest at the time I'm writing this): 489c57e2a40c6a0965424a762b4c8f78b4dc838a.

Kudos to @i4ki for finding the bug, I just reproduced/reported it.

[FEATURE] conditional terraform providers

Is your feature request related to a problem? Please describe.
Currently, if you want to add a new provider to a certain stack in your project, you need to update global.tm.hcl as follows:

generate_hcl "_providers.gen.tf" {
  condition = global.generate_aws_provider || global.generate_mysql_provider

  content {
    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = "~> 4.14"
        }
        # add new provider mysql
        mysql = {
          source  = "petoju/mysql"
          version = "~> 3.0.29"
        }
      }
    }
  }
...

And after this every stack in the project will be updated, which is not what I would like to do at all.

Describe the solution you'd like
Something like conditional tm_merge on multiple content blocks with terraform.required_providers would really help.
Conditions work in tm_dynamic right now only.
Maybe some sugar like this:

  content {
    terraform {
      condition = tm_try(global.generate_aws_provider, true)
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = "~> 4.14"
        }
    }

    terraform {
      condition = tm_try(global.generate_mysql_provider, false)
      required_providers {
        mysql = {
          source  = "petoju/mysql"
          version = "~> 3.0.29"
        }
      }
    }
  }

[FEATURE] Do not insist that root config is at root of git repo

Is your feature request related to a problem? Please describe.
With release 0.1.13 a check was added that the root config should be at the top level of the hosting git repo. I would rather that (at least for configurations that do not use the git integration) the root config can be at the root of the terramate directory within the repo, which may not be the top level. eg. repo_root/
repo_root/terraform-modules
repo_root/some-other-stuff
repo_root/terramate/terramate.tm.hcl (which contains the config block)

Describe the solution you'd like
A clear and concise description of what you want to happen.
Revert to previous behaviour for deployments that do not use git-integration. eg. set a config value that says, git_integration=false, and if a file with that config setting is encountered at a lower level in the repo it is used for the config.

Describe alternatives you've considered
For now I have put the config block at the root of the repo, but it messes up the semantics of the directory structure.

Additional context
Add any other context or screenshots about the feature request here.

[FEATURE] Support tags in `terramate generate`

Describe the bug
If you run terramate generate --help it shows you that the --tags option is available. However, if you make use of this, Terramate will still generate the code for all stacks.

To Reproduce
In a Terramate configuration with multiple tagged stacks, run terramate generate --tags="< valid tags here>".

Expected behavior
I would have expected Terramate to only generate code for the stacks which match the tag filter.

Log Output
Add logs from Terramate to help debug your problem.

Environment (please complete the following information):

• OS: Linux
• OS Version: Arch Linux
• Git Version: 2.39.2
• Terramate Version: 0.2.16

tests: start e2e cli testing

The idea is to setup the base for easily adding new tests on terrastack at the CLI level, so we can test issues with wrong flags integrated with the full behavior (hence e2e). My initial idea is to start with a single test for this bug: #25

The scenario is fairly complex and I would like us to have something that makes complex test scenarios reasonably easy to add.

bug: module change detection reporting wrong stacks.

If a module changes, terrastack list --changed is showing the stacks dependent as changed but if other stack depends on other modules (not changed) they're also been reported as changed. If a stack does not have modules, it's not been reported as changed, that's why the tests didn't catch this, this test case was missing.

Below is a bash script to reproduce the issue:

#!/usr/bin/env bash

mkdir -p bug

cd bug
git init >/dev/null

mkdir -p modules/{1,2} stacks/stack-{1,2,3}

cat <<EOF >> modules/1/main.tf
# module 1
EOF

cat <<EOF >> modules/2/main.tf
# module 2
EOF

cat <<EOF >> stacks/stack-1/main.tf

module "mod1" {
    source = "../../modules/1"
}
EOF

cat <<EOF >> stacks/stack-2/main.tf

module "mod1" {
    source = "../../modules/2"
}
EOF

cat <<EOF >> stacks/stack-3/main.tf

# no module
EOF

terrastack init stacks/stack-{1,2,3}

git add .           >/dev/null
git commit -m "all" >/dev/null

terrastack list --changed # must show no changes

git checkout -b change-the-module-1 2>/dev/null

echo '# changed' >> modules/1/main.tf

git add modules/1/main.tf           >/dev/null
git commit -m "module 1 changed"    >/dev/null

terrastack list --changed
# will output:
#   stacks/stack-1
#   stacks/stack-2

It creates 3 stacks where stack-1 depend on modules/1 and stack-2 depend on modules/2 and stack-3 has no module dependency. By changing modules/1 it shows stack-1 and stack-2 incorrectly as changed.

Output:

$ ./bug.sh
stacks/stack-1
stacks/stack-2

[BUG] Terramate cannot parse overrides the same as terraform does

Describe the bug

Terramate doesn't parse override files the same as Terraform and errors on for example module override blocks without source specifications.

To Reproduce

Steps to reproduce the behavior:

1. Place a module block inside a vpc.tf file:

   module "vpc" {
     source = "terraform-aws-modules/vpc/aws"
   
     name = "my-vpc"
     cidr = "10.0.0.0/16"
   
     azs             = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
     private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
     public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
   
     enable_nat_gateway = true
     enable_vpn_gateway = true
   
     tags = {
       Terraform = "true"
       Environment = "dev"
     }
   }

2. Add a vpc_override.tf file, and override the module block. This module block is without a source = argument:

   module "vpc" {
     name = "my-vpc-override"
   }

3. Run terramate list --changed and see example error:

   2022-11-07T16:30:49+01:00 ERR listing changed stacks error: checking module changes: applying operation to file "foo_override.tf": parsing modules: terraform schema error: module must have a "source" attribute file=/home/dev/dev/tf-stack/foo_override.tf:5,22-8,2
   

Adding a "duplicated" source line on all override blocks avoids this issue.

Expected behavior
Same parsing functionality as Terraform. Overides to work.

Environment (please complete the following information):

• Terramate Version 0.1.40

[FEATURE] Add support for folder in generate_hcl block

Hi,

When dealing with multiple generated files from different modules, it could interesting to isolate generated files into some folders instead of simply use a "_" to order files.

Currently, the char "/" is not allowed in the file name making it impossible.

Many thanks

bug: remote origin check fails with references with "/"

We are getting this error on some scenarios involving change detection and git checks:

terramate list -c
2021/12/09 21:57:09 checking if remote "origin" exists: unexpected remote reference "refs/remotes/origin/name/etc"

feat: fail if local origin/main revision is behind remote origin/main

We should check for the remote origin/main revision and fail if it differs from the local one.
We can use git ls-remote for that or other solution.

[QUESTION] Is it possible to dynamically generate stacks based on used/all regions?

Hi,
thank you for sharing the Terramate project.

I'm curious, is it possible to use Terramate to dynamically generate stacks based on regions data gathered from AWS by using aws_region data source.

My case is to instantiate some resources for all/used regions.

Switch to HCL

The terrastack HCL configuration language will have the following properties:

1. .tsk extension (or .tsk.hcl until we have IDE plugins).

As each HCL application (terrastack, terraform, consul, etc) extends HCL with its own types, functions, and object semantics, it's wise to differentiate its file extension from plain HCL so other tools don't try to mistakenly load them.

2. A stack configuration is the union of loading all .tsk files in the stack directory.

This is needed to easily support all use cases.
Eg.: Some files will be auto-generated (ts init, ts generate, etc) so we can generate them without messing user's configurations.
Eg.: Complex stack configurations can be split in several files for readability.
eg.: For future features, we can separate stack config from stack variables (tf var/data).

3. The command terrastack init will generate a version.tsk with the tool configuration but user can change the file name (terrastack init --file terrastack.tsk).

The init will generate a single file for terrastack tool configuration, this file is better to keep separated from stack configuration so the tooling can easily change it (upgrade versions, default config, etc).

$ terrastack init
$ terrastack config list-path=relative git-default-branch=main

Will generate the version.tsk below:

# version.tsk

terrastack {
    required_version = "~> 0.1"
    
    config = {
        git-default-branch = "main"
        list-path = "relative"
    }
}

This is also more inline with TF best practice of having a version.tf file.

feat: add backend configuration support to metadata

Once we have basic metadata in place: #54
We can then add support to it on backend configuration.

refactor: extract logic embedded on main for testing

The idea is to enable easier e2e testing of the cli, including flags/output, by extracting logic from main, having something like this:

https://github.com/tailscale/tailscale/blob/d6dde5a1aca795be5165e6add00ba814c68ef7dc/cmd/tailscale/tailscale.go#L23

There is also go's approach of testing using the binary itself (more blackbox):

https://github.com/golang/go/blob/c49627e81b05f23f97544fc6bfae3347296b4a06/src/cmd/go/go_test.go#L86

But after some discussion with @i4ki we decided to start with the extracting logic from main approach.

[QUESTION] Is conditional generation possible?

Can I conditional generate hcl depending on some global variable?

Use case: I need to add assume_role role to aws provider only if role name exists in globals

Code example:

globals {
  region = "us-east-1"
  aws_role_arn = ""
}

generate_hcl "backend.tf" {
  content {
    provider "aws" {
      region = "${global.region}"
      dynamic "assume_role" {
        for_each = global.aws_role_arn != "" ? [true] : []

        content {
          role_arn = global.aws_role_arn
        }
      }
    }
  }
}

Current code will correctly handle such case but it will provide ugly generated code.

Generated code:

provider "aws" {
  region = "us-east-1"
  dynamic "assume_role" {
    for_each = "" != "" ? [true] : []
    content {
      role_arn = ""
    }
  }
}

[FEATURE] Print stack on which Terraform commands are ran

Is your feature request related to a problem? Please describe.

Maybe I missed something but could not find it by myself. Is there a way to print the stack name or id upon which Terraform commands are ran?

I am running Terramate on dozens of stacks and the Terraform plan output can be tedious to analyse for a human. Currently, I retrieve the execution order using terramate experimental run-order and then I run terramate run terraform plan. But from here, I have to mentally link the output of each plan with the execution order.

I notice that all these information are already accessible running the command terramate experimental metadata but I do not find a way to retrieve them durring Terramate execution. I am thinking of something alongside (for example) when running terramate run terraform plan:

-- ${terramate.stack.id} --
No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

-- ${terramate.stack.id} --
No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Describe the solution you'd like

Here are some idea

• Have a way to access stack’s metadata during a stack execution.
• Print stack metadata before the Terraform command being executed

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[BUG] Codegeneration provides inccorrect code

Describe the bug
A clear and concise description of what the bug is.

I have next string in my generate_hcl function call
terraform${tm_try(global.state_bucket_key, terramate.path)}/terraform.tfstate

When I generate stack in path /live/production/us-east-1/infrastructure/route53-association

if provide strange value

Correct value terraform/live/production/us-east-1/infrastructure/route53-association/terraform.tfstate
Current value terraform/live/production/us-east-1/infrastructure/route53-associationciationorm.tfstate

state_bucket_key variable is undefined

Environment (please complete the following information):

• OS: [e.g. Linux] MacOS (MBP M1)
• OS Version [e.g. Ubuntu 20.04]
• Git Version [e.g. 2.35.1] (git --version)
• Terramate Version [e.g. 0.1.0] (terramate --version) 0.1.3

Additional context
Add any other context about the problem here.

[BUG] Calculating maps from constructed labeled globals creates random results.

Describe the bug
A clear and concise description of what the bug is.

To Reproduce

Steps to reproduce the behavior:

1. Given the following code in a stack

stack {
  name        = "providers"
  description = "providers"
  id          = "1e9936ee-5181-4cb4-b0cd-9385b8653226"
}

globals {
  _available_providers = {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.14"
    }
    vault = {
      source  = "hashicorp/vault"
      version = "~> 3.10"
    }
    postgresql = {
      source  = "cyrilgdn/postgresql"
      version = "~> 1.18.0"
    }
    mysql = {
      source  = "petoju/mysql"
      version = "~> 3.0.29"
    }
  }

  required_providers = {for k, v in global._available_providers : k => v if tm_try(global.use_providers[k], false)}
}

globals "use_providers" {
  aws   = true
}

globals "use_providers" {
  mysql = true
}

3. Run command terramate experimental eval global.required_providers

4. See error of randomly generating content of required_providers global.

Expected behavior

consitently generate the following code:

	required_providers = {
	  aws = {
	    source  = "hashicorp/aws"
	    version = "~> 4.14"
	  }
	  mysql = {
	    source  = "petoju/mysql"
	    version = "~> 3.0.29"
	  }
	}

Environment (please complete the following information):

• OS: Linux
• Terramate Version 0.2.13

Additional context

• as detected in #864
• setting globals { use_providers = { ... } will not create random results.

[BUG] Terramate panics when using ternary expression with `tm_hcl_expression()` function

Describe the bug

Terramate seems to panic when running generate with a ternary expression mixed with the tm_hcl_expression() function.

Example:

globals {
  database = global.module_rds_enabled ? tm_hcl_expression("module.rds.db_instance_database_name") : tm_hcl_expression("module.rds_aurora.cluster_database_name")
}

To Reproduce

Use the next configuration

terramate {
  required_version = "~> 0.1.42"

  config {
    git {
      check_untracked   = false
      check_uncommitted = false
      check_remote      = false
    }
  }
}

globals {
  module_rds_enabled = true

  database = global.module_rds_enabled ? tm_hcl_expression("module.rds.db_instance_database_name") : tm_hcl_expression("module.rds_aurora.cluster_database_name")
}

generate_hcl "_test.tf" {
  content {
    database = global.database
  }
}

stack {
  name = "test-stack"
}

and run terramate generate

Expected behavior

The above expression should return database = module.rds.db_instance_database_name when global.module_rds_enabled = true or database = module.rds_aurora.cluster_database_name when global.module_rds_enabled = false

Log Output

Logs from panic:

panic: cannot produce tokens for customdecode.ExpressionVal(&hclsyntax.ScopeTraversalExpr{Traversal:hcl.Traversal{hcl.TraverseRoot{isTraverser:hcl.isTraverser{}, Name:"module", SrcRange:hcl.Range{Filename:"<generated-hcl>\x00module.rds_aurora.cluster_database_name", Start:hcl.Pos{Line:1, Column:1, Byte:0}, End:hcl.Pos{Line:1, Column:7, Byte:6}}}, hcl.TraverseAttr{isTraverser:hcl.isTraverser{}, Name:"rds_aurora", SrcRange:hcl.Range{Filename:"<generated-hcl>\x00module.rds_aurora.cluster_database_name", Start:hcl.Pos{Line:1, Column:7, Byte:6}, End:hcl.Pos{Line:1, Column:18, Byte:17}}}, hcl.TraverseAttr{isTraverser:hcl.isTraverser{}, Name:"cluster_database_name", SrcRange:hcl.Range{Filename:"<generated-hcl>\x00module.rds_aurora.cluster_database_name", Start:hcl.Pos{Line:1, Column:18, Byte:17}, End:hcl.Pos{Line:1, Column:40, Byte:39}}}}, SrcRange:hcl.Range{Filename:"<generated-hcl>\x00module.rds_aurora.cluster_database_name", Start:hcl.Pos{Line:1, Column:1, Byte:0}, End:hcl.Pos{Line:1, Column:40, Byte:39}}})

goroutine 1 [running]:
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a628?, 0xc00019d140?}}, {0x14aba00?, 0xc00085f170?}}, {0xc000af6c00, 0x14, 0x20})
        github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:298 +0x120b
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a820?, 0xc00085f250?}}, {0x14dac80?, 0xc000609b30?}}, {0xc000c4b680, 0x10, 0x10})
        github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:284 +0xf93
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a820?, 0xc00085f260?}}, {0x14dac80?, 0xc000609b90?}}, {0xc000c4b680, 0xc, 0x10})
        github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:284 +0xf93
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a820?, 0xc00085f280?}}, {0x14dac80?, 0xc000609bf0?}}, {0xc000025ae0, 0x4, 0x4})
        github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:284 +0xf93
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a820?, 0xc000ac4970?}}, {0x14dac80?, 0xc000bb1aa0?}}, {0x0, 0x0, 0x0})
        github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:284 +0xf93
github.com/hashicorp/hcl/v2/hclwrite.TokensForValue({{{0x164a820?, 0xc000ac4970?}}, {0x14dac80?, 0xc000bb1aa0?}})
        github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:24 +0x35
github.com/mineiros-io/terramate/hcl/eval.(*engine).evalVar(0xc000a00768)
        github.com/mineiros-io/terramate/hcl/eval/partial.go:901 +0x41a
github.com/mineiros-io/terramate/hcl/eval.(*engine).evalIdent(0xc000a00768)
        github.com/mineiros-io/terramate/hcl/eval/partial.go:609 +0x136
github.com/mineiros-io/terramate/hcl/eval.(*engine).evalExpr(0xc000a00768)
        github.com/mineiros-io/terramate/hcl/eval/partial.go:354 +0x1278
github.com/mineiros-io/terramate/hcl/eval.(*engine).Eval(0xc000a00768)
        github.com/mineiros-io/terramate/hcl/eval/partial.go:115 +0xee
github.com/mineiros-io/terramate/hcl/eval.(*Context).PartialEval(0xc0000153c8, {0x164a548?, 0xc0000ad0e0?})
        github.com/mineiros-io/terramate/hcl/eval/eval.go:126 +0x230
github.com/mineiros-io/terramate/generate/genhcl.copyBody(0xc000502090?, 0xc000426dc0, {0x164a5f0, 0xc0000153d0})
        github.com/mineiros-io/terramate/generate/genhcl/genhcl.go:310 +0x84b
github.com/mineiros-io/terramate/generate/genhcl.appendBlock(0xc000a012d0?, 0xc0001506c0, {0x164a5f0, 0xc0000153d0})
        github.com/mineiros-io/terramate/generate/genhcl/genhcl.go:338 +0xa8
github.com/mineiros-io/terramate/generate/genhcl.copyBody(0x164a740?, 0xc000426f20, {0x164a5f0, 0xc0000153d0})
        github.com/mineiros-io/terramate/generate/genhcl/genhcl.go:322 +0xa9a
github.com/mineiros-io/terramate/generate/genhcl.Load(0xc000304780?, {{0xc000037770, 0x46}, {0xc0006ed500, 0x6d, 0x6d}}, {0x164c798?, 0xc000741dc0}, 0x1?)
        github.com/mineiros-io/terramate/generate/genhcl/genhcl.go:226 +0xad4
github.com/mineiros-io/terramate/generate.loadStackCodeCfgs(0xc00055e948?, {{0xc000037770, 0x46}, {0xc0006ed500, 0x6d, 0x6d}}, 0x164c798?, 0xc000741dc0?)
        github.com/mineiros-io/terramate/generate/generate.go:963 +0xe7
github.com/mineiros-io/terramate/generate.Do.func1({{0xc000037770, 0x46}, {0xc0006ed500, 0x6d, 0x6d}}, 0xc000741dc0, 0xc0007f5e00?)
        github.com/mineiros-io/terramate/generate/generate.go:159 +0x46e
github.com/mineiros-io/terramate/generate.forEachStack(0xc000304790?, {0xc000320840, 0x56}, 0xc000a036a0)
        github.com/mineiros-io/terramate/generate/generate.go:686 +0xacd
github.com/mineiros-io/terramate/generate.Do(0xc0006fe098, {0xc000320840?, 0x1f4?})
        github.com/mineiros-io/terramate/generate/generate.go:138 +0x85
github.com/mineiros-io/terramate/cmd/terramate/cli.(*cli).generate(0xc0006fe000, {0xc000320840?, 0x0?})
        github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:600 +0x4a
github.com/mineiros-io/terramate/cmd/terramate/cli.(*cli).run(0xc0006fe000)
        github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:392 +0x4d8
github.com/mineiros-io/terramate/cmd/terramate/cli.Exec({0xc0001ac010, 0x1, 0x1}, {0x1647ca0, 0xc0001a0000}, {0x1647cc0, 0xc0001a0008}, {0x1647cc0, 0xc0001a0010})
        github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:202 +0xdd
main.main()
        github.com/mineiros-io/terramate/cmd/terramate/main.go:30 +0x78

Environment (please complete the following information):

• OS: Mac OSX
• OS Version: 12.6 (21G115)
• Git Version: git version 2.37.1 (Apple Git-137.1)
• Terramate Version: 0.1.42

[FEATURE] Allow globs on import

It would be nice to use globs to import modules.

Since my current setup counts with many files on each "stack" that I have, I could easily wrap the file contents into a generate_hcl block and just import everything (as is) to my stacks instead of adding many import blocks.

example:

modules/
└── my_mod/
    ├── data.tm
    ├── iam.tm
    ├── locals.tm
    └── main.tm

import {
  source = "/modules/my_mod/*.tm"
}

I know that I can wrap everything into a single file or just add many import blocks, so it's a "nice to have" feature!

[FEATURE] Support the metadata variables in a stack definition

Is your feature request related to a problem? Please describe.
If you rename the folder structure in a stack while using the orchestration / dependency management features of terramate, you have to manually update any references.

Describe the solution you'd like
Support the metadata variables in a stack definition, particularly for orchestration (before, after, etc.). This would allow for easy renaming / moving of stack folder structure without having to remember to update config in the stack definition.

Describe alternatives you've considered
I have substituted by directly referencing the name of the path.

Additional context
See bug #692

[BUG] Terramate do not wait to its backgroung processes when a stop signal is sent

Describe the bug

Terraform is still running after Terramate has stopped completely due to a stop signal (control-c).

To Reproduce

Steps to reproduce the behavior:

1. Given a very simple project with several stacks.
2. Run command 'terramate run terraform apply' and send a stop signal with control-c while Terraform is aplying changes.
3. See error

Expected behavior

Terramate exit and all its processes.

Log Output

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

^C
Interrupt received.
Please wait for Terraform to exit or data loss may occur.
Gracefully shutting down...

Stopping operation...

~/workspace/playing-with-terramate$ ^C
~/workspace/playing-with-terramate$ ╷
│ Error: execution halted
│
│

Environment (please complete the following information):

• OS: Ubuntu
• OS Version: Ubuntu 20.04
• Git Version: 2.25.1
• Terramate Version: 0.1.7

Additional context

[BUG] Issue with metadata interpretation in stack configuration

Describe the bug
I am attempting to define orchestration / dependency relationships between stacks. I defined it via the after statement in the stack config and used the metadata such as ${terramate.stack.path.to_root}. Terramate fails to evaluate stating there is no variable named "terramate"

To Reproduce

Steps to reproduce the behavior:

1. Define a stack as

 stack {
  name = "test vm"
  after = [
    "${terramate.root.path.fs.absolute}/stacks/resources/vm-admin-keypair",
    #"../../resources/vm-admin-keypair",
    "${terramate.stack.path.to_root}/resources/vm-security-group",
    #"../../resources/vm-security-group",
  ]
 }

Note, I tried two different approaches above, but both fail with the same error.

2. Run command terramate list
3. See error

terramate list
2022-11-03T11:50:18-05:00 ERR terramate schema error: loading from /redacted/path/to/stacks: loading from /redacted/path/to/stacks/staging: loading from /redacted/path/to/stacks/staging/redacted: eval expression: failed to evaluate "after" attribute: There is no variable named "terramate". file=/redacted/path/to/stacks/staging/redacted/stack.tm.hcl:4,8-17
2022-11-03T11:50:18-05:00 ERR terramate schema error: loading from /redacted/path/to/stacks: loading from /redacted/path/to/stacks/staging: loading from /redacted/path/to/stacks/staging/redacted: eval expression: failed to evaluate "after" attribute: There is no variable named "terramate". file=/redacted/path/to/stacks/staging/redacted/stack.tm.hcl:6,8-17
2022-11-03T11:50:18-05:00 FTL looking up project root

Expected behavior
A clear and concise description of what you expected to happen.

I expected the variable interpolation to work here as it does in multiple other configuration elements within Terramate. Of course, it will work if I uncomment the lines above with the ../.. and comment out the lines with the Terramate metadata elements. This is what I have done for now.

Log Output
Add logs from Terramate to help debug your problem.

Environment (please complete the following information):

• OS: MacOS
• OS Version: 12.6.1
• Git Version: 2.37.1
• Terramate Version: 0.1.40

[QUESTION] Should "terramate list" work in lowest directory in example repo?

[I originally reported this question in the examples repo, but maybe its make more sent to log it as a question here...]

Hello. I'm new to terramate and trying to learn through experimenting with the terramate-example-code-generation, but running into some problems.

It seems I can only run terramate list or terramate generate at certain directory levels within a project.

(I'm using terramate version 0.1.18)

If I try to run in the lowest directory level: ./stacks/prod/cloud-runs/app1, I get errors. It's not until I walk up the directory two levels that terramate list works.

An example is below. Noticed the command succeeds after walking up two dirs.

Is this expected, or maybe a bug? There's nothing obvious (at least to me) in the terramate doc that suggests what might be causing this behavior.

Thanks in advance for any help/guidance, as well as sharing terramate. It looks like a very compelling well thought-out tool.

❯ pwd
/path/to/terramate-example-code-generation/stacks/prod/cloud-runs/app1
❯ terramate list
2022-08-03T18:00:32-04:00 FTL failed to lookup project root error="/path/to/terramate-example-code-generation/stacks/prod/cloud-runs/import_cloud_run.tm.hcl:14,12-49: import error: failed to create sub parser: failed to stat context basedir "/path/to/terramate-example-code-generation/stacks/prod/cloud-runs/modules/cloud-run": stat /path/to/terramate-example-code-generation/stacks/prod/cloud-runs/modules/cloud-run: no such file or directory" action=newCli() workingDir=/path/to/terramate-example-code-generation/stacks/prod/cloud-runs/app1
❯ cd ..
❯ pwd
/path/to/terramate-example-code-generation/stacks/prod/cloud-runs
❯ terramate list
2022-08-03T18:00:41-04:00 FTL failed to lookup project root error="/path/to/terramate-example-code-generation/stacks/prod/cloud-runs/import_cloud_run.tm.hcl:14,12-49: import error: failed to create sub parser: failed to stat context basedir "/path/to/terramate-example-code-generation/stacks/prod/cloud-runs/modules/cloud-run": stat /path/to/terramate-example-code-generation/stacks/prod/cloud-runs/modules/cloud-run: no such file or directory" action=newCli() workingDir=/path/to/terramate-example-code-generation/stacks/prod/cloud-runs
❯ cd ..
❯ pwd
/path/to/terramate-example-code-generation/stacks/prod
❯ terramate list
cloud-runs/app1
cloud-runs/app2
service-accounts/cloud-run

[FEATURE] Homebrew installation option for MacOS and Linux

Is your feature request related to a problem? Please describe.
Nope.

Describe the solution you'd like
Provide an option to install Terramate via Homebrew Tap (https://docs.brew.sh/Taps) or Homebrew Core (https://github.com/Homebrew/homebrew-core), including compatibility with Linux (https://docs.brew.sh/Homebrew-on-Linux).

Describe alternatives you've considered
Native installation with Golang lacks plenty of features supported by Homebrew and/or package managers.

Additional context
Would be good to have such an option. Thank you.

feat: add terrastack.(path|name) metadata

As specified in: https://github.com/mineiros-io/terrastack/blob/main/docs/metadata.md
It will be more useful as we implement more features that can use it, like globals/re-usable backend configuration.

[Question] Separated git repos, and using DRY in stacks

Hey guys !

I'm currently in the process of evaluating Terramate to see if it would fit with my infrastructure needs.

I've been looking at the mineiros-io/terramate-example-code-generation example repo, and the example structure is :

├── modules
│   ├── cloud-run
│   └── service-account
└── stacks
    ├── prod                 # Repeated
    │   ├── cloud-runs
    │   │   ├── app1
    │   │   ├── app2
    │   └── service-accounts
    │       └── cloud-run
    └── staging              # Repeated
        ├── cloud-runs
        │   ├── app1
        │   ├── app2
        └── service-accounts
            └── cloud-run

In order to keep a precise versioning of each modules, I would like to store the modules in a separate Git repository instead of storing them in the same repo as the stack folder.

Since this kind of setup is supported by Terraform, I'm assuming it's also supported when using Terramate, but can someone confirm ?

Also, after reading the docs and trying to follow the DRY approach in the modules folder, I don't really see the reason why I should not follow the DRY approach also in the stacks folder too ?

I think that it could make sense to configure the CI/CD pipepline to detect the branch main or development and simply load different environment variables in order to target the different environments upon PR.

Example of CI/CD triggers on the stacks repo :

• push to development branch -> deploy to Dev environment
• merge to main branch -> deploy to Staging environment
• release a new version -> deploy to Prod environment

So I am wondering what it would take to build the following setup :

. # My GitHub Org
├── modules        # Separate git repo, to be able to use version/tags when calling the modules in the stacks
│   ├── cloud-run
│   └── service-account
└── stacks         # Separate git repo, also DRY. Env vars are dictating which Environment to target
    ├── cloud-runs
    │   ├── app1
    │   ├── app2
    └── service-accounts
        └── cloud-run

Is there anything specific that is preventing Terramate from being able to work in a setup where the stack are not under an environment structure, and that the environments variables are the things dictates where the changes are being deployed to ?

Thanks

Thomas

[QUESTION] How to pass output from one stack to another stack?

Is your question related to a problem? Please describe.
One of our common use cases is to pass outputs created by one stack as input into variables of a dependent stack. E.g. creating a load balancer in one stack and using the name (or id) of the load balancer in another stack to define the target groups and listeners.

Describe the solution you'd like
Being able to reference the output variable name of the providing stack for the input variable.

Describe alternatives you've considered
Trying to generate the data resource using global variables and internal knowledge from the providing stack resource naming convention - which is flaky.

Additional context
Being able to separate resources in smaller parts allows us to iterate faster and test more independently.

[FEATURE] Switch off the git safeguard checks

Is your feature request related to a problem? Please describe.
Have terramate project in a git repo, but do not wish to make use of the git integration at this point in time.
So, have simply config {} at top level. However, terramate insists on checking for untracked files etc. It would be good
if that can be switched off.

Describe the solution you'd like
config setting to not fuss about git repo if you are not using git integration features

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

plan run-order command ambiguous output

Hi, This is my First Issue for mineiros i didn't see any contribution guide line so i will do my best to supply all information needed.

When running terramate plan run-order the output are only modules names making it hard to know which module actually is going to be execute when re using the same module names in the stack, Just like in the terramate-example-code-generation repository.

Here are the example outputs:

terramate stacks list

stacks/prod/cloud-runs/app1
stacks/prod/cloud-runs/app2
stacks/staging/cloud-runs/app1
stacks/staging/cloud-runs/app2

terramate plan run-order

app1
app2
app1
app2

I would expect to have the relative path in the run order output - or as a metdata along side the model name. So i would know if prod is before staging.

terramate version = 0.0.11

[BUG] bad config when run the example code

Describe the bug
Run sample code exception

2022-06-28T00:20:33+08:00 FTL listing stacks error="listing stacks error: failed to exec: /usr/local/bin/git version : stderr=\"error: could not expand include path '~/.gitcinclude'\\nfatal: bad config line 46 in file /usr/local/git/etc/gitconfig\\n\"" action=printStacks()

To Reproduce

Steps to reproduce the behavior:

1. git clone https://github.com/mineiros-io/terramate-example-code-generation
2. cd terramate-example-code-generation && terramate list
3. See error

Environment (please complete the following information):

• macOs Monterey 12.1
• OS Version 12.1
• Git Version 2.31.0
• Terramate Version 0.1.9

[QUESTION] How to save plan output

If running 'terramate run terraform plan' from a directory with several stacks beneath it is there a way to save the plan output to file(s).

[FEATURE] Regex for watcher

Is your feature request related to a problem? Please describe.
When you need to watch changes from more than 10 files it gets somewhat hard to remember to add new files to stack.watch.

Describe the solution you'd like
Currently there is that:

stack {
  ...
  watch = [
    "customtemplates/template1.tftpl",
    "customtemplates/template2.tftpl",
    "customtemplates/template3.tftpl",
    "jsons/file1.json",
    "jsons/file2.json",
    "jsons/file3.json"
  ]
}

Could be more readable and easily manageable this way:

stack {
  ...
  watch = [
    "customtemplates/*.tftpl",
    "jsons/*.json"
  ]
}

Describe alternatives you've considered
For now I'm adding all filepaths that needs to be watched manually for each file.

Additional context
Thanks.

[FEATURE] - Add in doc a comparison with pro and cons with other tools

Describe the solution you'd like
I would be able to fully understand key concepts differences between terramate and other well known tools like terragrunt.

Additional context
Because terramate approach seems interesting I would understand how you inject vars to terraform with different scopes and what should be versionned on git in stacks folders.

[BUG] terramate run terraform init etc. crashes

Describe the bug
'terramate generate' works great, and creates the tf files in a stack. And these files work with direct terraform invocation (ie. terraform init, terraform plan etc.) But if I try 'terramate run terraform init' etc. I get a runtime error:

erramate run terraform plan
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x13a43db]

goroutine 1 [running]:
github.com/mineiros-io/terramate/git.(*Git).Remotes(0x0)
github.com/mineiros-io/terramate/git/git.go:325 +0x21b
github.com/mineiros-io/terramate/cmd/terramate/cli.project.checkDefaultRemote({{0xc0000ae700, 0x44}, {0xc0000ae700, 0x6a}, 0x0, {{0xc0000ae700, 0x44}, 0xc0001e0df8, 0x0}, {0x0, ...}, ...})
github.com/mineiros-io/terramate/cmd/terramate/cli/project.go:184 +0x292
github.com/mineiros-io/terramate/cmd/terramate/cli.(*cli).checkGit(0xc000292b60)
github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:363 +0x2d9
github.com/mineiros-io/terramate/cmd/terramate/cli.(*cli).runOnStacks(0xc000292b60)
github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:878 +0x2f0
github.com/mineiros-io/terramate/cmd/terramate/cli.(*cli).run(0xc000292b60)
github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:336 +0x514
github.com/mineiros-io/terramate/cmd/terramate/cli.Exec({0xc0000b0010, 0x3, 0x3}, {0x15abee0, 0xc0000a2000}, {0x15abf00, 0xc0000a2008}, {0x15abf00, 0xc0000a2010})
github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:139 +0xb7
main.main()
github.com/mineiros-io/terramate/cmd/terramate/main.go:24 +0x7a
To Reproduce

I am calling a module via a source of a relative path that leads to a directory outside of the terramate root, so maybe that is a
factor

Steps to reproduce the behavior:
Just running 'terramate run terraform ' with terraform version 1.1.7 (or 1.2.2) and terramate 0.1.5

Expected behavior
Expect the terraform command to be executed

Log Output
Add logs from Terramate to help debug your problem.

Environment (please complete the following information):

• OS: OSX
• OS Version Big Sur 11.6.5
• Git Version N/A
• • Terramate Version 0.1.5

Additional context
Add any other context about the problem here.

[BUG] Label conflict in conditional code generation when different conditions is used

Describe the bug
When we conditionally generate code we should allow same names when only one condition is valid

To Reproduce

Try to run terraform generate with next code snippet

generate_hcl "providers.gen.tf" {
  condition = global.aws_role_arn != ""

  content {
    provider "aws" {
      region = "${global.region}"
      assume_role {
        role_arn = global.aws_role_arn
      }
    }
  }
}

generate_hcl "providers.gen.tf" {
  condition = global.aws_role_arn == ""

  content {
    provider "aws" {
      region = "${global.region}"
    }
  }
}

Expected behavior
Code should be generated without errors because only one condition will be valid at the same time

Log Output
Add logs from Terramate to help debug your problem.

- stack /some-private-stack
        error: label conflict detected: loading generate_hcl: /global.tm.hcl has blocks with same label "providers.gen.tf"

Hint: '+', '~' and '-' means the file was created, changed and deleted, respectively.

Environment (please complete the following information):

• OS: [e.g. Linux] MacOS (MBP M1)
• OS Version [e.g. Ubuntu 20.04]
• Git Version [e.g. 2.35.1] (git --version) git version 2.32.0 (Apple Git-132)
• Terramate Version [e.g. 0.1.0] (terramate --version) 0.1.4

Additional context
Add any other context about the problem here.

[BUG] Missing step in Project Setup section of README.md

Describe the bug Issue
Before the command git push --set-upstream origin main is run you need to enter the command git branch -M main or you will get an error error: src refspec main does not match any

To Reproduce

Steps to reproduce the behavior:

1. Follow the README.md steps to play with Terramate

Environment (please complete the following information):

• OS: GNU/Linux
• OS Version: Ubuntu 20.04.4 LTS
• Git Version: 2.25.1
• Terramate Version: 0.1.1

[QUESTION] How is the project root determined?

I have terramate buried fairly low down a directory tree (not a git repo). ie. /.../..../..../terramate/stacks. In the terramate directory I have terramate.tm.hcl with
terramate {
required_version = "0.1.5"
}
in it.
But if I run terramate list it reports 'FTL project root not found action=newCli() workingDir=/.
So, is there something I need to do to mark the stack root?