hashicorp / terraform-exec

Terraform CLI commands via Go.

terraform-exec

A Go module for constructing and running Terraform CLI commands. Structured return values use the data types defined in terraform-json.

The Terraform Plugin SDK is the canonical Go interface for Terraform plugins using the gRPC protocol. This library is intended for use in Go programs that make use of Terraform's other interface, the CLI. Importing this library is preferable to importing github.com/hashicorp/terraform/command, because the latter is not intended for use outside Terraform Core.

While terraform-exec is already widely used, please note that this module is not yet at v1.0.0, and that therefore breaking changes may occur in minor releases.

We strictly follow semantic versioning.

Go compatibility

This library is built in Go, and uses the support policy of Go as its support policy. The two latest major releases of Go are supported by terraform-exec.

Currently, that means Go 1.18 or later must be used.

Usage

The Terraform struct must be initialised with NewTerraform(workingDir, execPath).

Top-level Terraform commands each have their own function, which will return either error or (T, error), where T is a terraform-json type.

Example

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/hashicorp/go-version"
	"github.com/hashicorp/hc-install/product"
	"github.com/hashicorp/hc-install/releases"
	"github.com/hashicorp/terraform-exec/tfexec"
)

func main() {
	installer := &releases.ExactVersion{
		Product: product.Terraform,
		Version: version.Must(version.NewVersion("1.0.6")),
	}

	execPath, err := installer.Install(context.Background())
	if err != nil {
		log.Fatalf("error installing Terraform: %s", err)
	}

	workingDir := "/path/to/working/dir"
	tf, err := tfexec.NewTerraform(workingDir, execPath)
	if err != nil {
		log.Fatalf("error running NewTerraform: %s", err)
	}

	err = tf.Init(context.Background(), tfexec.Upgrade(true))
	if err != nil {
		log.Fatalf("error running Init: %s", err)
	}

	state, err := tf.Show(context.Background())
	if err != nil {
		log.Fatalf("error running Show: %s", err)
	}

	fmt.Println(state.FormatVersion) // "0.1"
}

Testing Terraform binaries

The terraform-exec test suite contains end-to-end tests which run realistic workflows against a real Terraform binary using tfexec.Terraform{}.

To run these tests with a local Terraform binary, set the environment variable TFEXEC_E2ETEST_TERRAFORM_PATH to its path and run:

go test -timeout=20m ./tfexec/internal/e2etest

For more information on terraform-exec's test suite, please see Contributing below.

Contributing

Please see CONTRIBUTING.md.

terraform-exec's Issues

Simplify ability to get state JSON from state affecting subcommands

Here is the upstream issue to simplify this interaction: hashicorp/terraform#25467

Ideally we would be able to do something like terraform apply -state-out-json foo.json and just read that file with the terraform-json module. We could then use this in the methods to return the state (ie. Apply(...) (*tfjson.State, error))

CI and release process

CI should run tests on all PRs.

Release process should update version(s) in code (e.g. tinstall/version.go), format CHANGELOG, tag, and push.

Add error types for state/planfile version incompatibility

There is a class of CLI errors of the form:

            Error: Invalid plan file
            
            Failed to read plan from plan file: plan file was created by Terraform
            0.12.29, but this is 0.13.0; plan files cannot be transferred between
            different Terraform versions.

Handle all these cases with well-known error types:

• Plan file created by different version
• State file created by later version
• State file created by 0.12 without explicit backend configuration and with a name other than "terraform.tfstate", and accessed in 0.13 hashicorp/terraform#25920

The only "format_version" presently known by terraform-json and this library is 0.1. Add at least an e2e test that will flag if another version is encountered. There's no need to error at runtime unless and until the Terraform CLI does so.

lock-timeout option not available as of Terraform 0.15

Hi there,

In the README example, we have this line:
err = tf.Init(context.Background(), tfexec.Upgrade(true), tfexec.LockTimeout("60s"))

However, when running code similar to the example, I'm getting the following error message:

Expected no error, but got: -lock, -lock-timeout, -verify-plugins, and -get-plugins options are no longer available as of Terraform 0.15: unexpected version 0.15.3 (min: -, max: 0.15.0)

If I remove the timeout option, it is working fine.
You may want to correct the example or add explanations on how to install a particular version of Terraform?

Cheers

Nic

Additional well known error tracking issue

Possible candidates, split out to new issue if addressing any of these:

• config syntax
• provider side errors
• TF CLI version constraint error
  • #66
• state file version mismatch error (other state errors? ie connectivity?)
• provider panic
• Provider installation error

Build tfinstall binaries using goreleaser

Add it to the CircleCI build

Differentiate Go panics from other exit code 2 usage

Exit code 2 is also used for Go panics, so in Plan we need to properly differentiate between a core panic and exit for the detailed exit code. Unsure how to simulate panics though, maybe we should go back through fixed ones to see if we can simulate one in a shipped release for testing.

Improve golden planfile test maintainability

The golden plan file used for Plan tests is specific to OS and Terraform version. Make this easier to maintain across development environments.

Missing upgrade013.go

Hello

I see it is upgrade012.go but there is no upgrade013.go to update from 0.12 to 0.13.
How can I manage the update from 0.12 to 0.13?

Is there any workaround I'm missing?

Thanks!

Don't set Pdeathsig when running inside aws lambda

When running tfexec inside a aws lambda any command dies with a cryptic error:
fork/exec /var/task/terraform: operation not permitted: PathError.
The issue is in tfexec/cmd_linux.go, setting Pdeathsig is not allowed in the lambda runtime and the execution is blocked.
I understand running terraform inside a lambda is not a recommended pattern but I find it really handy for modules that don't take too long to apply (15 mins max runtime).
The fix itself is simple enough and doesn't affect non-lambda linux runtimes: check for lambda runtime before setting SysProcAttr

	if _, ok := os.LookupEnv("LAMBDA_TASK_ROOT"); !ok {
		cmd.SysProcAttr = &syscall.SysProcAttr{
			// kill children if parent is dead
			Pdeathsig: syscall.SIGKILL,
			// set process group ID
			Setpgid: true,
		}
	}

Would you be interested in a PR to address this?

Run tests against Terraform Core master on CircleCI cron/scheduled job

Use TFEXEC_E2ETEST_VERSIONS as added in #51

Parse terraform version -json output

From 0.13 onwards, terraform version accepts a -json flag. See hashicorp/terraform#25252.

It might be worth trying terraform version -json first, before falling back to the legacy version output parsing logic added in #7.

Support TF_LOG_{CORE,PROVIDER}

Core hashicorp/terraform#26685 adds support for two new environment variables, TF_LOG_CORE and TF_LOG_PROVIDER, for independent control of the log level of each subsystem.

Support this in terraform-exec.

Enforce version constraints for command options

For each option, check what Terraform version it was added in, and add a tf.compatible check enforcing the minimum version. We might want a better framework for doing this than adding logic in each command, but it might not be worth doing that because option compatibility is per command, not per option, and often influences default flag behaviour (see the changes in #120 for an example).

Add convenience method to Plan and return JSON as single func

One option is supporting the exit code:
https://www.terraform.io/docs/commands/plan.html#detailed-exitcode

The Plan signature could be changed to: Plan(...) (changesPresent bool, err error) or something similar to surface the exit code data.

Another option is just returning the actual JSON plan always from the Plan method. In programmatic usage you probably will want it frequently anyway.

We could optionally do both of these in the return values.

Reflect new validation JSON output options

Investigate to see what is changing in hashicorp/terraform#28057 and reflect it in the Validate method, which in proxy probably means reflecting the changes in terraform-json.

Provide better errors caused by context cancellation

#91 was closed partially because we were worried about implications in terms of backwards incompatibility.

I do however believe that the library should provide decent error messages - i.e. not context deadline exceeded by default, so I'm hoping this can be revisited before v1 as that gives us chance to make some breaking changes.

Feature Request: workspace support

Feature Description

tfexec doesn't support a workspace-based environment, and I want support it.

Use Case(s)

I want to run Plan and Apply in a workspace environment. I have confirmed that the following code does not currently work.

package main

import (
	"context"
	"fmt"
	"github.com/hashicorp/terraform-exec/tfexec"
)

func main() {
	var dst = "/Users/takuro.saito/work/core-infra/terraform/aws/tools/slack-command/"
	var ctx context.Context
	ctx = context.Background()

	tf, err := tfexec.NewTerraform(dst, "")
	if err != nil {
		panic(err)
	}

	fmt.Println("init")
	err = tf.Init(ctx)
	if err != nil {
		panic(err)
	}
}

❯ go run main.go
init
panic: 
Error: Failed to select workspace: input not a valid number




goroutine 1 [running]:
main.main()
        /var/tmp/path/to/main.go:23 +0x133
exit status 2

Add ability to install latest from Terraform git repo

Ensure we propagate CHECKPOINT_DISABLE always

Even if not propagating env, we may want to always propagate this env var, or at least make the propagation an explicit opt out. If people intend to disable a phone home, we should err on the side of honoring it.

Custom Slack notification template

.circleci/SlackTemplate.json was added in #116. Setting this up without massive duplication in the CircleCI config file will require some wizardry: see CircleCI-Public/slack-orb#219 and https://github.com/CircleCI-Public/slack-orb/wiki/Dynamic-Templates.

Support passing User-Agent

Tools executing Terraform (such as Terraform Language Server) would benefit from measuring impact/usage and one way of doing it would be by appending an identifier to User-Agent by passing TF_APPEND_USER_AGENT which then enables filtering data from whatever upstream backend requests were sent to.

https://github.com/hashicorp/terraform-plugin-test/issues/22

I'm not sure if this should be a first-class config option, but I'm creating this issue to discuss whether it should.

Split out tfinstall into separate hcinstall module

https://github.com/hashicorp/hc-install generalises tfinstall to all HashiCorp products with binaries on releases.hashicorp.com. It's in an experimental state with an API overhaul to improve usability.

Remember to include changes from #111.

Support terraform state mv

Support terraform state mv:

Usage: terraform state mv [options] SOURCE DESTINATION

 This command will move an item matched by the address given to the
 destination address. This command can also move to a destination address
 in a completely different state file.

 This can be used for simple resource renaming, moving items to and from
 a module, moving entire modules, and more. And because this command can also
 move data to a completely new state, it can also be used for refactoring
 one configuration into multiple separately managed Terraform configurations.

 This command will output a backup copy of the state prior to saving any
 changes. The backup cannot be disabled. Due to the destructive nature
 of this command, backups are required.

 If you're moving an item to a different state file, a backup will be created
 for each state file.

Options:

  -dry-run            If set, prints out what would've been moved but doesn't
                      actually move anything.

  -backup=PATH        Path where Terraform should write the backup for the original
                      state. This can't be disabled. If not set, Terraform
                      will write it to the same path as the statefile with
                      a ".backup" extension.

  -backup-out=PATH    Path where Terraform should write the backup for the destination
                      state. This can't be disabled. If not set, Terraform
                      will write it to the same path as the destination state
                      file with a backup extension. This only needs
                      to be specified if -state-out is set to a different path
                      than -state.

  -lock=true          Lock the state files when locking is supported.

  -lock-timeout=0s    Duration to retry a state lock.

  -state=PATH         Path to the source state file. Defaults to the configured
                      backend, or "terraform.tfstate"

  -state-out=PATH     Path to the destination state file to write to. If this
                      isn't specified, the source state file will be used. This
                      can be a new or existing path.

Support Terraform 0.14.0 and later -chdir Global Option

Reference: https://github.com/hashicorp/terraform/blob/v0.14/CHANGELOG.md
Reference: hashicorp/terraform#26087
Reference: hashicorp/terraform#25558
Reference: hashicorp/terraform#18030

Terraform commands have historical incompatibilities when executed outside the configuration directory, for example automatic variables files are not properly read. Terraform 0.14.0 (all pre-release versions) introduces a new -chdir global option, e.g. terraform -chdir /path/to/config <command> syntax. Later Terraform versions will deprecate the older terraform <command> /path/to/config as an argument.

workspace delete

Hi
Today the tf-exec only support
workspace list and new
i would like to make it possible to do workspace delete as well

Thanks

tfinstall CLI utility

The tfinstall package can also provide a tfinstall binary for quickly installing Terraform, e.g. as a CI build step. Probably best as an internal/cmd package so it's not confused with the library's public API.

Example CLI:

tfinstall [-latest-version] [-version=version] OUTDIR

Terraform version 0.15 panics on tf.Init

Description

Hello👋 Terraform recently upgraded to 0.15, and ever since my program started failing when calling tf.Exec() with the following error:

-lock, -lock-timeout, -verify-plugins, and -get-plugins options are no longer available as of Terraform 0.15: unexpected version 0.15.0 (min: -, max: 0.15.0)

Reproduce

The issue can be reproduced with the sample Go code:

package main

import (
	"context"
	"fmt"
	"io/ioutil"
	"os"

	"github.com/hashicorp/terraform-exec/tfexec"
	"github.com/hashicorp/terraform-exec/tfinstall"
)

func main() {
	tmpDir, err := ioutil.TempDir("", "tfinstall")
	if err != nil {
		panic(err)
	}
	defer os.RemoveAll(tmpDir)

	execPath, err := tfinstall.Find(context.Background(), tfinstall.LatestVersion(tmpDir, false))
	if err != nil {
		panic(err)
	}

	workingDir := "/Users/pflorent/Project/arewefastyet_fork/infra/terraform"
	tf, err := tfexec.NewTerraform(workingDir, execPath)
	if err != nil {
		panic(err)
	}

	err = tf.Init(context.Background(), tfexec.Upgrade(true), tfexec.LockTimeout("60s"))
	if err != nil {
		panic(err)
	}

	state, err := tf.Show(context.Background())
	if err != nil {
		panic(err)
	}

	fmt.Println(state.FormatVersion) // "0.1"
}

The temporary fix I have found is to initialize tf using tfinstall.ExactVersion() with a version < 0.15, instead of tfinstall.LatestVersion().

Specs

Running on MacOS with go1.16.

Add ability to watch lock file for outside re-initialization

This is mostly for use within the context of the language server. We need to be able to watch files for changes and know that the schema is potentially out of date.

The reason I think this makes sense in this package is that the behavior varies per TF version and would be useful to incorporate in to the schema functionality already built in so we don't have to build version compatibility logic in to multiple packages / projects.

Supply duration args as time.Duration not string

See https://github.com/hashicorp/terraform/blob/f3bed4039fa423554fc8b698aac36a5b832980a6/command/workspace_new.go#L31

Supply backend config as vars a "rawFlags" n/v pairs similar to CLI

See https://github.com/hashicorp/terraform/blob/d2e999ba1f2a4a12f29e65f4ede95882f55e15f0/command/meta_config.go#L387-L392

Investigate / test tfenv usage or other script like executable wrappers

This is mostly relevant to the LS, but something we should investigate.

Known wrappers

See #6 (comment) for more details.

tfenv

terraform on path is now a shell script: yes
Terraform version may depend on working directory: yes

terraform-switcher

terraform on path is now a shell script: no
Terraform version may depend on working directory: yes

homebrew-terraforms

terraform on path is now a shell script: no
Terraform version may depend on working directory: no

asdf

terraform on path is now a shell script: unknown
Terraform version may depend on working directory: unknown

Consider including stderr in error from Init

One of the nightly E2E tests had failed with the following output:

=== RUN   TestDestroy/basic-0.12.30
    util_test.go:110: [INFO] running Terraform command: /var/folders/6y/gy9gggt14379c_k39vwb50lc0000gn/T/tfinstall342488260/v-0.12.30/terraform init -no-color -force-copy -input=false -lock-timeout=0s -backend=true -get=true -upgrade=false -lock=true -get-plugins=true -verify-plugins=true
    destroy_test.go:16: error running Init in test directory: exit status 2
        2021/04/05 00:22:07 [DEBUG] Using modified User-Agent: Terraform/0.12.30 tfexec-e2etest HashiCorp-terraform-exec/0.13.1

Here exit status 2 is not enough context for further debugging - which is "fine" if it's just a nightly test (although annoying), but it may not be that great for a consumer leveraging Init() and needing to debug similar error.

which makes me wonder if it's worth either:

1. Including stderr in the error at all times (for commands that otherwise don't expose stderr), or
2. Revisiting the idea of custom error types, which was initially binned due to worries about breaking compatibility when exploring ways of resolving #107

Tfexec test helper package for import

As of #51, we run tfexec's e2e tests against Terraform master. We should also provide a package for consumers to import, perhaps tftester, which allows the tfexec test suite to be used to test the following projects (and others):

• Terraform Core
• terraform-ls
• terraform-plugin-go

Integrating this into the CI for these projects would allow them to test changes against the behaviour of the Terraform CLI before they are merged.

Occasional test failures due to timeout on registry request

The fix for this may well be in Core or the registry, but filing here for now.

Tests runs occasionally fail with a Failed to query available provider packages error like the following:

=== RUN   TestStateMv/basic_with_state-0.14.4
    util_test.go:111: [INFO] running Terraform command: /tmp/tfinstall187681955/v-0.14.4/terraform init -no-color -force-copy -input=false -lock-timeout=0s -backend=true -get=true -upgrade=false -lock=true -get-plugins=true -verify-plugins=true
    state_mv_test.go:23: error running Init in test directory: exit status 1
        
        Error: Failed to query available provider packages
        
        Could not retrieve the list of available versions for provider hashicorp/null:
        could not connect to registry.terraform.io: Failed to request discovery
        document: Get "https://registry.terraform.io/.well-known/terraform.json":
        net/http: request canceled while waiting for connection (Client.Timeout
        exceeded while awaiting headers)
        
--- FAIL: TestStateMv (13.12s)
    --- SKIP: TestStateMv/basic_with_state-0.11.14 (0.23s)
    --- SKIP: TestStateMv/basic_with_state-0.12.30 (0.47s)
    --- PASS: TestStateMv/basic_with_state-0.13.6 (1.12s)
    --- FAIL: TestStateMv/basic_with_state-0.14.4 (11.31s)

Example failing test run: https://app.circleci.com/pipelines/github/hashicorp/terraform-exec/661/workflows/bdc5a99c-63da-4f12-ad66-52df573f600c/jobs/9510

The tests pass when rerun.

More examples needed, but suspect this can happen during any test that runs terraform init.

verifySumsSignature returns unexpected error: openpgp: signature made by unknown entity

Description

Since Terraform release 0.15.1 and their GPG key rotation (hashicorp/terraform#28505) I get the following error when running tfinstall.Find using the documentation's code and my own code:

unexpected error: openpgp: signature made by unknown entity

This error is generated by the following chunk of code.

terraform-exec/tfinstall/download.go

Lines 98 to 100 in c4b8475

	// verifySumsSignature downloads SHA256SUMS and SHA256SUMS.sig and verifies
	// the signature using the HashiCorp public key.
	func verifySumsSignature(sumsPath, sumsSigPath string) error {

The checksum URLs I have during the execution are:

variable	value
sumsURL	https://releases.hashicorp.com/terraform/0.15.1/terraform_0.15.1_SHA256SUMS
sumsSigURL	https://releases.hashicorp.com/terraform/0.15.1/terraform_0.15.1_SHA256SUMS.sig

Specs

Running on MacOS with go1.16.

Plan/Apply `-json` support missing

Plans and applies now support streaming json with the -json flag. Adding support for the flag may not be enough--including streamed data structures from stdout would be preferred.

Add darwin test runner

Add ability to constrain Latest for tfinstall

For example, you may want to get the latest 0.12 version.

End-to-end test framework

Test the non-*Cmd versions of the tfexec functions, preferably with as many test cases as possible.

Borrow test cases from Core and terraform-json.

Ideally perform some sort of matrix testing across multiple Terraform versions and OS environments (needs #11).

Support uncoloured output across all commands

There was an attempt to implement this in the language server recently

https://github.com/hashicorp/terraform-ls/pull/208/files

but we didn't realize that some commands don't support the flag the same way others do - specifically terraform providers schema seems to need a special treatment. This then lead to reverting of that change in a subsequent bugfix release:

hashicorp/terraform-ls#226
hashicorp/terraform-ls#227

Support multiple -var-file flags

-var-file can be used multiple times

Plan/apply `-replace` and `-refresh=false` options are missing

There's a couple of new plan features that could use support added to this lib:
https://www.terraform.io/docs/cli/commands/plan.html#planning-options

Parallelise tests in CI to reduce duration

See #119 (comment).

Investigate whether parallel execution can make e2e tests run faster on CI.

Test provider version output

Currently we do not test logic which parses provider versions:

terraform-exec/tfexec/internal/e2etest/version_test.go

Line 29 in e0c9ea1

// TODO: test/assert provider info

Add ability to cache binaries between e2etest run

Downloading binaries for every test run is slow.

The easiest method is probably allowing a hard coded install directory that it can look for prior to downloading.

rename tfexec.StateShow to tfexec.Show

StateShow() actually runs terraform show -json, which is distinct from terraform state show. Output from the latter is intended for human consumption, so we will not be implementing it.

https://www.terraform.io/docs/commands/show.html
https://www.terraform.io/docs/commands/state/show.html

Investigate and fix race conditions

Version: v0.13.0 (22121c5627c8729080f6de7e829a6a2b4f772703)

I also managed to reproduce some races under 0.12.0, but it was hard to tell whether these are the same due to some other (non-race) test failures adding to the noise.

There is a few tests which fail under go test -race:

--- FAIL: TestApply (180.40s)
    --- FAIL: TestApply/basic-0.11.14 (29.45s)
--- FAIL: TestTFVersionMismatch (3.29s)
    --- FAIL: TestTFVersionMismatch/tf99-0.11.14 (0.95s)
--- FAIL: TestContext_sleepTimeoutExpired (190.26s)
    --- FAIL: TestContext_sleepTimeoutExpired/sleep-0.12.30 (62.82s)
--- FAIL: TestValidate (7.81s)
    --- FAIL: TestValidate/invalid-0.12.30 (0.53s)

Full output is available in this gist: https://gist.github.com/radeksimko/9f9cd6a451204c12cc3e312dd0c34836 or you can reproduce it yourself via go test ./... -race.

use runtime.BuildInfo for version

Instead of a constant in tfinstall/version.go, we could use https://golang.org/pkg/runtime/debug/#BuildInfo.

Use go-version.Version.Core() to match prerelease versions

For the purposes of testing, we want versions such as 0.15.0-dev to be counted as 0.15.0. As of go-version v1.3.0, this can be done by calling .Core() on the version read from the environment to obtain a new version without the prerelease. Implement this instead of matching prerelease versions ad hoc like we do now.