googlecloudplatform / terraform-google-alloy-db

Creates an Alloy DB instance

terraform-google-alloy-db

This module handles Google Cloud Platform AlloyDB cluster creation and configuration with Automated Backup Policy, Primary node instance and Read Node Pools. The resource/resources that this module will create are:

• Creates an AlloyDB Cluster with/without automated backup policy
• Creates a Primary Instance
• Creates a pool of Read Instances

You can also create Cross Region Replica using this module. See example in cross_region_replica

Compatibility

This module is meant for use with Terraform 1.3+ and tested using Terraform 1.3+. If you find incompatibilities using Terraform >=1.3, please open an issue.

Version

Current version is 2.X. Upgrade guides:

• 0.1 -> 0.2
• 0.2 -> 1.0
• 1.X -> 2.0
• 2.X -> 3.0

Usage

• Functional examples are included in the examples directory.
• If you want to create a cluster with failover replicas and manage complete lifecycle (including failover and switchover) of primary and DR clusters using this module, follow the example in simple_example folder.
• If you are planning to create cluster/instance with private service connect follow example in private_service_connect.

Basic usage of this module is as follows:

module "alloy-db" {
  source               = "GoogleCloudPlatform/alloy-db/google"
  version              = "~> 3.0"

  cluster_id           = "alloydb-cluster"
  cluster_location     = "us-central1"
  project_id           = <"PROJECT_ID">
  cluster_labels       = {}
  cluster_display_name = ""
  cluster_initial_user = {
    user     = "<USER_NAME>",
    password = "<PASSWORD>"
  }
  network_self_link = "projects/${project_id}/global/networks/${network_name}"

  automated_backup_policy = {
    location      = "us-central1"
    backup_window = "1800s",
    enabled       = true,
    weekly_schedule = {
      days_of_week = ["FRIDAY"],
      start_times  = ["2:00:00:00", ]
    }
    quantity_based_retention_count = 1,
    time_based_retention_count     = null,
    labels = {
      test = "alloydb-cluster"
    },
  }
  primary_instance = null

  read_pool_instance = null

}

• Usage of this module for creating a AlloyDB Cluster with a primary instance and a read replica instance

module "alloy-db" {
  source               = "GoogleCloudPlatform/alloy-db/google"
  version              = "~> 3.0"
  project_id           = <PROJECT_ID>
  cluster_id           = "alloydb-cluster-with-primary-instance"
  cluster_location     = "us-central1"
  cluster_labels       = {}
  cluster_display_name = ""
  cluster_initial_user = {
    user     = "<USER_NAME>",
    password = "<PASSWORD>"
  }
  network_self_link = "projects/${project_id}/global/networks/${network_name}"

  automated_backup_policy = null

  primary_instance = {
    instance_id       = "primary-instance",
    instance_type     = "PRIMARY",
    machine_cpu_count = 2,
    database_flags    = {},
    display_name      = "alloydb-primary-instance"
  }

  read_pool_instance = [
    {
      instance_id        = "cluster-1-rr-1"
      display_name       = "cluster-1-rr-1"
      require_connectors = false
      ssl_mode           = "ALLOW_UNENCRYPTED_AND_ENCRYPTED"
    }
  ]

}

Inputs

Name	Description	Type	Default	Required
allocated_ip_range	The name of the allocated IP range for the private IP AlloyDB cluster. For example: google-managed-services-default. If set, the instance IPs for this cluster will be created in the allocated range	string	null	no
automated_backup_policy	The automated backup policy for this cluster. If no policy is provided then the default policy will be used. The default policy takes one backup a day, has a backup window of 1 hour, and retains backups for 14 days	object({ location = optional(string) backup_window = optional(string) enabled = optional(bool) weekly_schedule = optional(object({ days_of_week = optional(list(string)) start_times = list(string) })), quantity_based_retention_count = optional(number) time_based_retention_count = optional(string) labels = optional(map(string)) backup_encryption_key_name = optional(string) })	null	no
cluster_display_name	Human readable display name for the Alloy DB Cluster	string	null	no
cluster_encryption_key_name	The fully-qualified resource name of the KMS key for cluster encryption. Each Cloud KMS key is regionalized and has the following format: projects/[PROJECT]/locations/[REGION]/keyRings/[RING]/cryptoKeys/[KEY_NAME]	string	null	no
cluster_id	The ID of the alloydb cluster	string	n/a	yes
cluster_initial_user	Alloy DB Cluster Initial User Credentials	object({ user = optional(string), password = string })	null	no
cluster_labels	User-defined labels for the alloydb cluster	map(string)	{}	no
cluster_location	Location where AlloyDb cluster will be deployed	string	n/a	yes
cluster_type	The type of cluster. If not set, defaults to PRIMARY. Default value is PRIMARY. Possible values are: PRIMARY, SECONDARY	string	"PRIMARY"	no
continuous_backup_enable	Whether continuous backup recovery is enabled. If not set, defaults to true	bool	true	no
continuous_backup_encryption_key_name	The fully-qualified resource name of the KMS key. Cloud KMS key should be in same region as Cluster and has the following format: projects/[PROJECT]/locations/[REGION]/keyRings/[RING]/cryptoKeys/[KEY_NAME]	string	null	no
continuous_backup_recovery_window_days	The numbers of days that are eligible to restore from using PITR (point-in-time-recovery). Defaults to 14 days. The value must be between 1 and 35	number	14	no
database_version	The database engine major version. This is an optional field and it's populated at the Cluster creation time. This field cannot be changed after cluster creation. Possible valus: POSTGRES_14, POSTGRES_15	string	null	no
deletion_policy	Policy to determine if the cluster should be deleted forcefully. Deleting a cluster forcefully, deletes the cluster and all its associated instances within the cluster	string	null	no
network_self_link	Network ID where the AlloyDb cluster will be deployed. If network_self_link is set then psc_enabled should be set to false	string	null	no
primary_cluster_name	Primary cluster name. Required for creating cross region secondary cluster. Not needed for primary cluster	string	null	no
primary_instance	Primary cluster configuration that supports read and write operations.	object({ instance_id = string, display_name = optional(string), database_flags = optional(map(string)) labels = optional(map(string)) annotations = optional(map(string)) gce_zone = optional(string) availability_type = optional(string) machine_cpu_count = optional(number, 2) ssl_mode = optional(string) require_connectors = optional(bool) query_insights_config = optional(object({ query_string_length = optional(number) record_application_tags = optional(bool) record_client_address = optional(bool) query_plans_per_minute = optional(number) })) enable_public_ip = optional(bool, false) cidr_range = optional(list(string)) })	n/a	yes
project_id	The ID of the project in which to provision resources.	string	n/a	yes
psc_allowed_consumer_projects	List of consumer projects that are allowed to create PSC endpoints to service-attachments to this instance. These should be specified as project numbers only.	list(string)	[]	no
psc_enabled	Create an instance that allows connections from Private Service Connect endpoints to the instance. If psc_enabled is set to true, then network_self_link should be set to null	bool	false	no
read_pool_instance	List of Read Pool Instances to be created	list(object({ instance_id = string display_name = string node_count = optional(number, 1) database_flags = optional(map(string)) availability_type = optional(string) gce_zone = optional(string) machine_cpu_count = optional(number, 2) ssl_mode = optional(string) require_connectors = optional(bool) query_insights_config = optional(object({ query_string_length = optional(number) record_application_tags = optional(bool) record_client_address = optional(bool) query_plans_per_minute = optional(number) })) enable_public_ip = optional(bool, false) cidr_range = optional(list(string)) }))	[]	no

Outputs

Name	Description
cluster	Cluster created
cluster_id	ID of the Alloy DB Cluster created
cluster_name	ID of the Alloy DB Cluster created
primary_instance	Primary instance created
primary_instance_id	ID of the primary instance created
primary_psc_attachment_link	The private service connect (psc) attachment created for primary instance
primary_psc_dns_name	The DNS name of the instance for PSC connectivity created for primary instance
read_instance_ids	IDs of the read instances created
read_psc_attachment_links	The private service connect (psc) attachment created read replica instances
read_psc_dns_names	The DNS names of the instances for PSC connectivity created for replica instances
replica_instances	Replica instances created

Requirements

These sections describe requirements for using this module.

Software

The following dependencies must be available:

• Terraform v1.3
• Terraform Provider for GCP plugin >= v5.32+

Service Account

A service account with the following roles must be used to provision the resources of this module:

• Cloud AlloyDB Admin: roles/alloydb.admin

APIs

A project with the following APIs enabled must be used to host the resources of this module:

• alloydb.googleapis.com

Contributing

Refer to the contribution guidelines for information on contributing to this module.

Security Disclosures

Please see our security disclosure process.

terraform-google-alloy-db's Issues

Add CMEK option in this module

Add CMEK option in this module for cluster and automayer backup.

Add support for query_insights_config in primary and read instances

Currently the module does not support configuration for query insights, so any manual enabling of query insights on a primary or read instance results in a terraform plan that wants to revert those settings.

example except from plan with a primary instance that has had query insights enabled

~ resource "google_alloydb_instance" "primary" {
    ...
      - query_insights_config {
          - query_plans_per_minute  = 5 -> null
          - query_string_length     = 1024 -> null
          - record_application_tags = false -> null
          - record_client_address   = false -> null
        }
    ...
    }

Default Username/Password

If you omit the optional cluster_initial_user what are the default credentials? 🤔 seeing as how it's optional I didn't add it and now I'm having issues authenticating. Any idea? Thanks! Great work btw

Database flags in the secondary cluster getting removed by Terraform

Hi Team,

We're using this module to provision a primary and a secondary AlloyDB cluster. Everything works well but we're facing a weird issue with DB flags in the secondary cluster. These flags are replicated into secondary from the primary but a subsequent Terraform apply is trying to remove them from the secondary cluster. I believe this is expected from the Terraform's side because DB flags were never part of the its state for the secondary cluster.

Here is the plan

// us-west1 is our secondary cluster

# module.alloydb["us-west1"].module.this.google_alloydb_instance.primary will be updated in-place
  ~ resource "google_alloydb_instance" "primary" {
      ~ database_flags        = {
          - "alloydb.enable_pgaudit"     = "on" -> null
          - "alloydb.iam_authentication" = "on" -> null
          - "log_connections"            = "on" -> null
          - "log_disconnections"         = "on" -> null
          - "log_replication_commands"   = "on" -> null
          - "log_statement"              = "all" -> null
          - "log_timezone"               = "CET" -> null
          - "pgaudit.log"                = "ddl,write" -> null
        }
        # (20 unchanged attributes hidden)
        # (3 unchanged blocks hidden)
    }

TF version - Terraform v1.9.0
Module version - ~> 2.0

Are we doing something wrong here? Any help or pointers will be highly appreciated. Thanks in advance.

Add support for private service connect

there is no output named by "private_ip_address" of alloydb instance to fetch

Hi,
I was using this alloy-db module but seems like there is no output variable configured for private IP address of alloydb instance.
May I know any way to fetch it out in the same terraform configuration

Not being able to define the "database_version"

Currently it is not possible to define the database_version for the AlloyDB cluster.
By default POSTGRES_14 is used, however there are cases when POSTGRES_15 may be needed. Additionally to be future-proof it makes sense to add the database_version variable to set the version.

`network_self_link` isn't a self link

The network_self_link attribute actually will not accept a network's self_link. If a self link is provided, an error like the below is seen:

Error: Error creating Cluster: googleapi: Error 499: malformed network path: "https://www.googleapis.com/compute/v1/projects/myproject123/global/networks/mynetworkname"

The usage example in the main README.md composes the value as below:

  network_self_link = "projects/${project_id}/global/networks/${network_name}"

I believe that format is known as a resource's id. In my experience, the convention is to pass self_links around, so ideally the module internals would be changed to support accepting an actual self_link (perhaps falling back to accepting an id if possible).

Add support for continuous backup

Add support for continuous backup. TPG version 4.77

Support for SSL enforcement modes for AlloyDB

AlloyDB now provides an ability to configure various SSL enforcement modes.
Public docs: instance-ssl

Terraform AlloyDB instance resource is already supporting SSL. Document

Currently we have to manually enable it from web interface.
Please can we have this issue resolved ?

Read_pool examples have invalid CPU configuration

Following this read_pool_instance example leads to an invalid configuration.

Specifically, machine_cpu_count = 1 is invalid, and leads to this error in terraform apply:

module.alloy-db.google_alloydb_instance.read_pool["read-instance-1"]: Creating...
╷
│ Error: Error creating Instance: googleapi: Error 400: The request was invalid: Invalid machine (CPU) config: cpu_count must be one of [2 4 8 16 32 64 96 128]: got 1
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.BadRequest",
│     "fieldViolations": [
│       {
│         "field": "machineConfig"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.RequestInfo",
│     "requestId": "b201b76e23f334d7"
│   }
│ ]
│ 
│   with module.alloy-db.google_alloydb_instance.read_pool["read-instance-1"],
│   on .terraform/modules/alloy-db/main.tf line 177, in resource "google_alloydb_instance" "read_pool":
│  177: resource "google_alloydb_instance" "read_pool" {

The multiple read_pool example will also have this error.

Suggestion:

• fix examples with valid configuration
• add validation in the module to check for valid configurations

Permanent 'Note: Objects have changed outside of Terraform'

When using the module, the primary instance is always showing as 'Objects have changed outside of Terraform' due to the update_time attribute on the instance.

Versions:
google provider v5.26.0
Terraform v1.8.4

module "alloydb" {
  source  = "GoogleCloudPlatform/alloy-db/google"
  version = "~> 2.0"

  project_id       = var.project_id
  cluster_id       = var.product
  cluster_location = "europe-west2"

  network_self_link = data.google_compute_network.default.id

  database_version = "POSTGRES_15"

  primary_instance = {
    instance_id = "primary"
    database_flags = {
      "alloydb.iam_authentication" = "on"
    }
    ssl_mode          = "ENCRYPTED_ONLY"
    machine_cpu_count = var.alloydb_cpu_count
    availability_type = var.alloydb_availability_type
    gce_zone          = var.alloydb_gce_zone
  }

  depends_on = [google_service_networking_connection.vpc_connection, google_compute_global_address.alloydb_private_ip_alloc]
  
}

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # module.chatbot_product.module.alloydb.google_alloydb_instance.primary has changed
  ~ resource "google_alloydb_instance" "primary" {
        id                    = "projects/my-project/locations/europe-west2/clusters/my-cluster/instances/primary"
        name                  = "projects/my-project/locations/europe-west2/clusters/my-cluster/instances/primary"
      ~ update_time           = "2024-05-30T14:05:05.6364540[19]Z" -> "2024-06-03T09:46:08.658064797Z"
        # (15 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

I can't seem to find any indication as to what is causing the update or what the update was. We can't ignore the attribute because the module wraps the creation of the resource.

PSC example tries to change network_config every time

I'm using the PSC example, run terraform apply, and it applies perfectly. PSC attachment is created and I can access the cluster using the internal IP.

When I run terraform plan again, it tries to change the cluster network_config parameter:

Terraform will perform the following actions:

  # module.alloydb_main["REDACTED"].google_alloydb_cluster.default will be updated in-place
  ~ resource "google_alloydb_cluster" "default" {
        id                     = "projects/REDACTED/locations/europe-west1/clusters/REDACTED"
        name                   = "projects/REDACTED/locations/europe-west1/clusters/REDACTED"
        # (18 unchanged attributes hidden)

      + network_config {}

        # (3 unchanged blocks hidden)
    }

let me know if I can do something to help, I'm still trying to find the root cause

IP Address?

Any possible way to get the ip_address of the primary instance? 🤔
I know the outputs are not part of the module but just wanted to see if that was possible

When will this be published?

Not an issue but a question. I would like to start using this module but I don't see it published in Terraform Modules.
Thanks

Read CPU Count

I was trying to pass in a variable to control CPU count on read instance but doesn't work. It appears the resource doesn't take in a machine_cpu_count like the primary resource

resource "google_alloydb_instance" "read_pool" {
  for_each      = local.read_pool_instance
  cluster       = google_alloydb_cluster.default.name
  instance_id   = each.key
  instance_type = each.value.instance_type

  read_pool_config {
    node_count = each.value.node_count
  }

  database_flags = each.value.database_flags

  depends_on = [google_alloydb_instance.primary]
}

[FEATURE REQUEST] Be able to create public read instances

Description

With the current version of the module, we can't create public read instances.

I've created a PR to add all the needed options: #90

Can anyone review this and release a new version please ?

No available releases match the given constraints

Hi,

I'm having trouble using v2.0.0 release. We are using this module on terragrunt latest with terraform 1.6.6

󰀵  tf --version
Terraform v1.6.6
on darwin_arm64

󰀵  tg --version
terragrunt version v0.54.12

To reproduce:

#versions.tf
terraform {
  required_version = ">= 1.6.6"
    google = {
      source  = "hashicorp/google"
      version = ">= 5.5, < 6"
    }
  }
}

#alloydb.tf
module "alloy-db" {
  for_each         = local.alloydb_primary_instances
  source           = "GoogleCloudPlatform/alloy-db/google"
  version = "~> 2.0"
  project_id       = var.project_id
  cluster_location = var.region
  cluster_id       = local.cluster_id
  cluster_labels   = local.cluster_labels

  cluster_initial_user = local.cluster_initial_user

  network_self_link = local.network_self_link

  automated_backup_policy = null


  primary_instance = {
    .
    .
    .  
}

  read_pool_instance = local.read_pool_instances
}

ERROR:

│ Error: Failed to query available provider packages
│ 
│ Could not retrieve the list of available versions for provider
│ hashicorp/google: no available releases match the given constraints >=
│ 3.43.0, >= 3.53.0, >= 4.38.0, >= 4.42.0, != 4.65.0, != 4.65.1, >= 4.74.0,
│ >= 4.76.0, < 5.0.0, >= 5.5.0, < 6.0.0

Trouble seems to come with the google-beta - Installing hashicorp/google-beta v4.84.0

Using module version v1.0.0 has no problem. Any suggestion?

Module tries to access `gce_zone` field via `availability_type` string field

Hi all,

I think that this line:

terraform-google-alloy-db/main.tf

Line 147 in 81522e3

gce_zone = each.value.availability_type == "ZONAL" ? each.value.availability_type.gce_zone : null

Needs to be updated to:

 gce_zone          = each.value.availability_type == "ZONAL" ? each.value.gce_zone : null 

Currently when I try to use the module this error occurs:

╷
│ Error: Unsupported attribute
│ 
│   on terraform-google-alloy-db/main.tf line 147, in resource "google_alloydb_instance" "read_pool":
│  147:   gce_zone          = each.value.availability_type == "ZONAL" ? each.value.availability_type.gce_zone : null
│     ├────────────────
│     │ each.value.availability_type is "ZONAL"
│ 
│ Can't access attributes on a primitive-typed value (string).

These are the provider versions I'm using:

$ terraform version

Terraform v1.5.7
on darwin_arm64
+ provider registry.terraform.io/hashicorp/google v5.5.0
+ provider registry.terraform.io/hashicorp/google-beta v5.5.0
+ provider registry.terraform.io/hashicorp/random v3.5.1

Add support for cross region replica

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): Update dependency go to v1.22.6
• chore(deps): Update Terraform GoogleCloudPlatform/alloy-db/google to v3
• chore(deps): Update Terraform terraform-google-modules/project-factory/google to v16
• feat(deps): Update Terraform google to v6
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

gomod

test/integration/go.mod

• go 1.22
• go 1.22.5
• github.com/GoogleCloudPlatform/cloud-foundation-toolkit/infra/blueprint-test v0.16.1
• github.com/stretchr/testify v1.9.0

regex

Makefile

• cft/developer-tools 1.22

build/int.cloudbuild.yaml

• cft/developer-tools 1.22

build/lint.cloudbuild.yaml

• cft/developer-tools 1

terraform

examples/example_with_multiple_readpool/main.tf

• GoogleCloudPlatform/alloy-db/google ~> 2.0

examples/example_with_primary_instance_public_ip_address/main.tf

• GoogleCloudPlatform/alloy-db/google ~> 3.0

examples/example_with_private_service_connect/cross_region_replica.tf

• GoogleCloudPlatform/alloy-db/google ~> 3.0

examples/example_with_private_service_connect/main.tf

• GoogleCloudPlatform/alloy-db/google ~> 3.0

examples/example_with_readpool_instances/main.tf

• GoogleCloudPlatform/alloy-db/google ~> 3.0

examples/simple_example/cross_region_replica.tf

• GoogleCloudPlatform/alloy-db/google ~> 3.0

examples/simple_example/main.tf

• GoogleCloudPlatform/alloy-db/google ~> 3.0

test/setup/main.tf

• terraform-google-modules/project-factory/google ~> 15.0
• terraform-google-modules/project-factory/google ~> 15.0

test/setup/versions.tf

• google >= 4.64
• google-beta >= 4.64
• hashicorp/terraform >= 1.3

versions.tf

• google >= 5.32, < 6
• hashicorp/terraform >= 1.3

---

• Check this box to trigger a request for Renovate to run again on this repository

Enable network_config and authorized_external_networks blocks configurations in google_alloydb_instance

Please enable network_config and authorized_external_networks blocks with their respective variables in google_alloydb_instance.

More information: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/alloydb_instance#nested_network_config

Ty.

Support for creating Basic (Zonal) Instances

It looks like creation of a basic (zonal) AlloyDB instance is now GA (https://cloud.google.com/alloydb/docs/basic-instance#gcloud)

Can we add Terraform Support for creating this type of AlloyDB instance?