How can I power off/on Aiven resources using Terraform? about terraform-provider-aiven

[rikonen]

Controlling the powered state is currently not supported because certain service state information becomes unavailable and/or unchangeable for powered off services. We will probably look into this at some point but making it work consistently in all cases is quite difficult.

Could you elaborate what kind of use case you have?

[akomlik]

In out dev environment we bring up expensive large resources (kafka, pg) along with big AWS docker swarm only temporarily. When not in use we destroy swarm in AWS and shutdown Kafka and PG at Aiven
This preserves all the data accumulated when we bring it up next time.

We now switch to Terragrunt to control individual Terraform projects responsible for these pieces of infrastructure (Kafka, PG, SQL, Swarm, InlfuxDB) to allow us bring up/down entire development environment with a single command.

Thinking more on this I realized the way "terraform destroy" works it needs to destroy the resource not just change its powerstate.

So to make it work in our use case you can add new property similar to termination_protection.
Call it "power_off_on_destroy". When this property set to "true" "terraform destroy" will perform poweroff instead of actual destroy of service.
Next "terraform apply" on same service (same properties for name, cloud, etc) will discover existing powered off service and turn it on instead of creating new one.

[rikonen]

Some attribute that powers off services instead of deleting them when the Terraform environment is destroyed would seem like a reasonable extension. However, if you use Terraform to also control your databases, as you should if you're using it otherwise, the databases would be deleted separately so even if the service itself is preserved the data would be gone. The issue with gone databases could be worked around by separately removing them from Terraform state (terraform state rm aiven_database.mydb etc.) before doing terraform destroy so the teardown part could be made to work somewhat reasonably.

Bringing the services back up with Terraform alone is something that would require somewhat too hacky behavior. Automatically importing services when they are being created does not sound like something we'd want to do. There's too high risk that the system ends up automatically doing something one didn't expect to happen. You could just do terraform import aiven_service.myservice myproject/myservice etc. for the service related resources explicitly before running terraform apply so that would take care of getting the earlier service back to Terraform without hacks. However, the service would then still be powered off and wouldn't really work. The power on / power off functionality is difficult to make properly in all cases as mentioned earlier. Since powering on a service manually doesn't require more than

curl -XPUT -H "Authorization: aivenv1 myapitoken" -H "Content-Type: application/json" -d '{"powered": true}' "https://api.aiven.io/v1beta/project/myproject/service/myservice"

you could do that before starting to apply the Terraform configuration. Now then if you do manual step to power on the services you might as well do manual step to power off the services without having the slightly hacky power_off_on_destroy attribute by doing exactly the same request as above but with value false. So your steps would be something like:

Power on:

curl ... '{"powered": true}' ...
terraform import aiven_service.mypg myproject/mypg
terraform import aiven_database.mydb myproject/mypg/mydb
terraform apply

Powef off:

terraform state rm aiven_database.mydb
terraform state rm aiven_service.mypg
curl ... '{"powered": false}' ...
terraform destroy

[akomlik]

Great suggestions and examples, thanks Rauli!
I'm still in early stages of converting all pieces of infrastructure under TF control.
At this time we use scripts that create folders for different components (VPC, backup, InlfuxDB, swarm) with single main.tf file. Then the script runs apply or destroy inside it.

Our desire is to get away from scripting model and express as much of our infrastructure as code.
I quickly realized I could not do this with Terraform alone so found Terragrunt solves many deficiencies.
I see that Terragrunt has before/after hooks feature to do exactly this sort of thing you suggest.
https://github.com/gruntwork-io/terragrunt#before-and-after-hooks

[rikonen]

I'm closing this for now as the "terraform destroy without deleting stateful services" and "terraform apply with auto import of services" features are not directly implementable in a reasonable way and workaround exists. Please let us know if you still think there are some changes that should be done to the Aiven Terraform provider itself or if there's some other input you'd require.

[akomlik]

Thanks. Once I start using this I'll have more input.