terraform-aws-modules / terraform-aws-rds-proxy

Terraform module to create AWS RDS Proxy resources 🇺🇦

AWS RDS Proxy Terraform module

Terraform module which creates an AWS RDS Proxy and its supporting resources.

Usage

See examples directory for working examples to reference:

module "rds_proxy" {
  source = "terraform-aws-modules/rds-proxy/aws"

  name                   = "rds-proxy"
  iam_role_name          = "rds-proxy-role"
  vpc_subnet_ids         = ["subnet-30ef7b3c", "subnet-1ecda77b", "subnet-ca09ddbc"]
  vpc_security_group_ids = ["sg-f1d03a88"]

  endpoints = {
    read_write = {
      name                   = "read-write-endpoint"
      vpc_subnet_ids         = ["subnet-30ef7b3c", "subnet-1ecda77b", "subnet-ca09ddbc"]
      vpc_security_group_ids = ["sg-f1d03a88"]
    },
    read_only = {
      name                   = "read-only-endpoint"
      vpc_subnet_ids         = ["subnet-30ef7b3c", "subnet-1ecda77b", "subnet-ca09ddbc"]
      vpc_security_group_ids = ["sg-f1d03a88"]
      target_role            = "READ_ONLY"
    }
  }

  auth = {
    "superuser" = {
      description        = "Aurora PostgreSQL superuser password"
      secret_arn         = "arn:aws:secretsmanager:us-east-1:123456789012:secret:superuser-6gsjLD"
    }
  }

  # Target Aurora cluster
  engine_family         = "POSTGRESQL"
  target_db_cluster     = true
  db_cluster_identifier = "my-endpoint"

  tags = {
    Terraform   = "true"
    Environment = "dev"
  }
}

Examples

Examples codified under the examples are intended to give users references for how to use the module(s) as well as testing/validating changes to the source code of the module(s). If contributing to the project, please be sure to make any appropriate updates to the relevant examples to allow maintainers to test your changes and to keep the examples up to date for users. Thank you!

• IAM auth. w/ MySQL Aurora cluster
• IAM auth. w/ MySQL RDS instance
• IAM auth. w/ PostgreSQL Aurora cluster
• IAM auth. w/ PostgreSQL RDS instance

Requirements

Name	Version
terraform	>= 1.0
aws	>= 5.0

Providers

Name	Version
aws	>= 5.0

Modules

No modules.

Resources

Name	Type
aws_cloudwatch_log_group.this	resource
aws_db_proxy.this	resource
aws_db_proxy_default_target_group.this	resource
aws_db_proxy_endpoint.this	resource
aws_db_proxy_target.db_cluster	resource
aws_db_proxy_target.db_instance	resource
aws_iam_role.this	resource
aws_iam_role_policy.this	resource
aws_iam_policy_document.assume_role	data source
aws_iam_policy_document.this	data source
aws_partition.current	data source
aws_region.current	data source

Inputs

Name	Description	Type	Default	Required
auth	Configuration block(s) with authorization mechanisms to connect to the associated instances or clusters	any	{}	no
connection_borrow_timeout	The number of seconds for a proxy to wait for a connection to become available in the connection pool	number	null	no
create	Whether cluster should be created (affects nearly all resources)	bool	true	no
create_iam_policy	Determines whether an IAM policy is created	bool	true	no
create_iam_role	Determines whether an IAM role is created	bool	true	no
db_cluster_identifier	DB cluster identifier	string	""	no
db_instance_identifier	DB instance identifier	string	""	no
debug_logging	Whether the proxy includes detailed information about SQL statements in its logs	bool	false	no
endpoints	Map of DB proxy endpoints to create and their attributes (see aws_db_proxy_endpoint)	any	{}	no
engine_family	The kind of database engine that the proxy will connect to. Valid values are MYSQL or POSTGRESQL	string	""	no
iam_policy_name	The name of the role policy. If omitted, Terraform will assign a random, unique name	string	""	no
iam_role_description	The description of the role	string	""	no
iam_role_force_detach_policies	Specifies to force detaching any policies the role has before destroying it	bool	true	no
iam_role_max_session_duration	The maximum session duration (in seconds) that you want to set for the specified role	number	43200	no
iam_role_name	The name of the role. If omitted, Terraform will assign a random, unique name	string	""	no
iam_role_path	The path to the role	string	null	no
iam_role_permissions_boundary	The ARN of the policy that is used to set the permissions boundary for the role	string	null	no
iam_role_tags	A map of tags to apply to the IAM role	map(string)	{}	no
idle_client_timeout	The number of seconds that a connection to the proxy can be inactive before the proxy disconnects it	number	1800	no
init_query	One or more SQL statements for the proxy to run when opening each new database connection	string	""	no
kms_key_arns	List of KMS Key ARNs to allow access to decrypt SecretsManager secrets	list(string)	[]	no
log_group_kms_key_id	The ARN of the KMS Key to use when encrypting log data	string	null	no
log_group_retention_in_days	Specifies the number of days you want to retain log events in the log group	number	30	no
log_group_tags	A map of tags to apply to the CloudWatch log group	map(string)	{}	no
manage_log_group	Determines whether Terraform will create/manage the CloudWatch log group or not. Note - this will fail if set to true after the log group has been created as the resource will already exist	bool	true	no
max_connections_percent	The maximum size of the connection pool for each target in a target group	number	90	no
max_idle_connections_percent	Controls how actively the proxy closes idle database connections in the connection pool	number	50	no
name	The identifier for the proxy. This name must be unique for all proxies owned by your AWS account in the specified AWS Region. An identifier must begin with a letter and must contain only ASCII letters, digits, and hyphens; it can't end with a hyphen or contain two consecutive hyphens	string	""	no
proxy_tags	A map of tags to apply to the RDS Proxy	map(string)	{}	no
require_tls	A Boolean parameter that specifies whether Transport Layer Security (TLS) encryption is required for connections to the proxy	bool	true	no
role_arn	The Amazon Resource Name (ARN) of the IAM role that the proxy uses to access secrets in AWS Secrets Manager	string	""	no
session_pinning_filters	Each item in the list represents a class of SQL operations that normally cause all later statements in a session using a proxy to be pinned to the same underlying database connection	list(string)	[]	no
tags	A map of tags to add to all resources	map(string)	{}	no
target_db_cluster	Determines whether DB cluster is targeted by proxy	bool	false	no
target_db_instance	Determines whether DB instance is targeted by proxy	bool	false	no
use_policy_name_prefix	Whether to use unique name beginning with the specified iam_policy_name	bool	false	no
use_role_name_prefix	Whether to use unique name beginning with the specified iam_role_name	bool	false	no
vpc_security_group_ids	One or more VPC security group IDs to associate with the new proxy	list(string)	[]	no
vpc_subnet_ids	One or more VPC subnet IDs to associate with the new proxy	list(string)	[]	no

Outputs

Name	Description
db_proxy_endpoints	Array containing the full resource object and attributes for all DB proxy endpoints created
iam_role_arn	The Amazon Resource Name (ARN) of the IAM role that the proxy uses to access secrets in AWS Secrets Manager.
iam_role_name	IAM role name
iam_role_unique_id	Stable and unique string identifying the IAM role
log_group_arn	The Amazon Resource Name (ARN) of the CloudWatch log group
proxy_arn	The Amazon Resource Name (ARN) for the proxy
proxy_default_target_group_arn	The Amazon Resource Name (ARN) for the default target group
proxy_default_target_group_id	The ID for the default target group
proxy_default_target_group_name	The name of the default target group
proxy_endpoint	The endpoint that you can use to connect to the proxy
proxy_id	The ID for the proxy
proxy_target_endpoint	Hostname for the target RDS DB Instance. Only returned for RDS_INSTANCE type
proxy_target_id	Identifier of db_proxy_name, target_group_name, target type (e.g. RDS_INSTANCE or TRACKED_CLUSTER), and resource identifier separated by forward slashes (/)
proxy_target_port	Port for the target RDS DB Instance or Aurora DB Cluster
proxy_target_rds_resource_id	Identifier representing the DB Instance or DB Cluster target
proxy_target_target_arn	Amazon Resource Name (ARN) for the DB instance or DB cluster. Currently not returned by the RDS API
proxy_target_tracked_cluster_id	DB Cluster identifier for the DB Instance target. Not returned unless manually importing an RDS_INSTANCE target that is part of a DB Cluster
proxy_target_type	Type of target. e.g. RDS_INSTANCE or TRACKED_CLUSTER

License

Apache-2.0 Licensed. See LICENSE.

terraform-aws-rds-proxy's Issues

Does this module support the use case of enabling individual user authentication via IAM roles?

Is your request related to a new offering from AWS?

Not a new offering, unsure if the desired feature is supported by the current module and dependencies.

Is your request related to a problem? Please describe.

Not a problem specific to the module. Need clarification if the module supports my use case.

Describe the solution you'd like.

Ideally, a solution that allows assigning a policy to an AWS SSO role that allows users to authenticate to a database using individual credentials without having to maintain a separate password.

If it isn't feasible to implement the solution with individual database user accounts, then using a shared account on the DB side as long as there is an audit trail back to the user who assumed the role and connected to the proxy.

A hypothetical deployment might look like this:

1. have an Aurora Postgres cluster configured without IAM authentication enabled
2. create a pg user with the appropriate permissions for a dev to have for that environment
3. set this tf module up to create an RDS Proxy that authenticates clients connecting to it with a specific IAM role and if they have the role, connect to the database using the pg user from #2
4. create a policy that grants the ability to assume the role from #3 to any AWS SSO user that has a particular group or tag

Describe alternatives you've considered.

Standard AWS help docs on IAMDBAuth don't really touch on using an SSO assumed role, just plain static IAM accounts.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.IAMPolicy.html
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.DBAccounts.html#UsingWithRDS.IAMDBAuth.DBAccounts.PostgreSQL

This sample seems to be a good fit, but it is using CloudFormation and Lambdas that complicate matters.
https://github.com/aws-samples/sso-sync-to-amazon-rds

Export Proxy Resource ID

Is your request related to a new offering from AWS?

Is this functionality available in the AWS provider for Terraform? See CHANGELOG.md, too.

• Yes ✅

Is your request related to a problem? Please describe.

• To use IAM Authentication against the DB Proxy, the proxy resource ID is required, such as prx-123456 - this is an immutable field, unlike name (which can change and is the typical way of interacting with RDS resources).
• See https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.IAMPolicy.html

DbiResourceId is the identifier for the DB instance. This identifier is unique to an AWS Region and never changes. In the example policy, the identifier is db-ABCDEFGHIJKL01234.
If you are connecting to a database through RDS Proxy, specify the proxy resource ID, such as prx-ABCDEFGHIJKL01234.

• Currently, this ID is not exported, which is most likely a side effect of the Attribute export from the db_proxy resource, where the id seems to return the pretty name and arn contains the ARN.
  https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_proxy#attribute-reference (The documentation appears to be wrong here regarding id)
  And the reasoning: https://github.com/hashicorp/terraform-provider-aws/blob/7f0a73253c273a9ef143189f94890fc66d0dcb9c/internal/service/rds/proxy.go#L243-L245

Describe the solution you'd like.

To make integrations easier into IAM policies, a new output (to remain backwards compatible) could be made:
proxy_resource_id: prx-123456

output "proxy_resource_id" {
  description = "The prx- Resource ID for the proxy, for IAM policies"
  value       = try(split(":", try(aws_db_proxy.this[0].arn, null))[6], null)
}

Describe alternatives you've considered.

The current workaround is to use the above try logic defined as a local, then use the string in the relevant IAM policy document, for example:

locals {
  # The Proxy Module and resource doesn't return the actual prx-id, so we have to parse it out for IAM usage
  proxy_resource_id = try(split(":", try(module.postgresql_proxy.proxy_arn, ""))[6], "unset")
}

...
resources = [
     "arn:aws:rds-db:us-east-1:${data.aws_caller_identity.current.account_id}:dbuser:${local.proxy_resource_id}/a_user"
]

It might be decided that this should live in the provider repo to export the specific value in a standalone fashion; however, it can be derived from the arn.

Optionally create a Security Group in *this* module for RDS Proxy use

Is your request related to a new offering from AWS?

No

Is your request related to a problem? Please describe.

RDS Proxy requires a Security Group allowing ingress to the proxy and egress to the target database. Allowing creation of an SG inside this module would greatly simplify many implementations.

Describe the solution you'd like.

A basic security group created by this module.

Describe alternatives you've considered.

Possible alternatives:

• write a wrapper around this module and terraform-aws-security-group to create the necessary Security Group.
• Use the terraform-aws-security-group module separetely to create the necessary SG, then use that as input for the terraform-aws-rds-proxy module.

Additional context

N/A

Can't have more than 1 secret

Can't have more than 1 secret for the input variable secrets.
It errors out with the following:
│ Error: Insufficient auth blocks │ │ on .terraform/modules/rds_proxy/main.tf line 13, in resource "aws_db_proxy" "this": │ 13: resource "aws_db_proxy" "this" { │ │ At least 1 "auth" blocks are required. ╵ ╷ │ Error: Invalid dynamic for_each value │ │ on .terraform/modules/rds_proxy/main.tf line 26, in resource "aws_db_proxy" "this": │ 26: for_each = var.secrets │ │ Cannot use a map of object value in for_each. An iterable collection is required.

Thanks in advance

Terraform fails to refresh state when more than one secret is attached to the Proxy

Describe the bug

  secrets = {
    "${local.db_username_1}" = {
      description = module.creds_1.description
      arn         = module.creds_1.secret_arn
      kms_key_id  = data.aws_kms_alias.secretsmanager.id
    },
    "${local.db_username_2}" = {
      description = module.creds_2.description
      arn         = module.creds_2.secret_arn
      kms_key_id  = data.aws_kms_alias.secretsmanager.id
    }
  }

when I try to attach more than one secrets, the terraform fails to exit after creating the resources. When I try to kill it and start again terraform fails to refresh the state and give below error:

Error: Error reading RDS DB Proxy (): RequestError: send request failed
caused by: Post "https://rds.eu-west-2.amazonaws.com/": net/http: HTTP/1.x transport connection broken: too many transfer encodings: ["chunked" "chunked"]

To Reproduce
Steps to reproduce the behavior:
Just try to attach more than one secret to the same RDS Proxy instance.

Expected behavior
Terraform should be able to refresh the state

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: macOS 11.1
• Terraform Version: 14.4

Set IAM Auth per user not proxy wide

Is your request related to a problem? Please describe.

Currently iam_auth is set proxy wide and pulled into the dynamic auth loop in the aws_db_proxy resource. With this approach you can't break down different users into IAM and non IAM authentication for the proxy.

Describe the solution you'd like.

Make it so that iam_auth can be pulled from the secrets object into the dynamic auth block instead of being set globally for the whole proxy.

MalformedPolicyDocument issue

Describe the bug

Running into the issue when creating the IAM role

│ Error: Error putting IAM role policy chapstick-rds-proxy: MalformedPolicyDocument: Resource  must be in ARN format or "*".
│ 	status code: 400, request id: 2cd81d3c-374c-4b97-9243-96eda3afb0db
│
│   with module.chapstick_rds_proxy.aws_iam_role_policy.this[0],
│   on .terraform/modules/chapstick_rds_proxy/main.tf line 176, in resource "aws_iam_role_policy" "this":
│  176: resource "aws_iam_role_policy" "this" {

To Reproduce
Steps to reproduce the behavior:

1. use the default example to create the proxy

cannot create proxy without `require_tls` = true

Description

• ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
2. Re-initialize the project root to pull down modules: terraform init
3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

• Module version [Required]: 2.1.0
• Terraform version: 1.1.8
• Provider version(s): 4

Reproduction Code [Required]

module "rds_proxy" {
  name = local.name
  source = "terraform-aws-modules/rds-proxy/aws"
  version = "2.1.0"

  # disable TLS check
  require_tls = false

  name                   = local.name
  iam_role_name          = local.name
  iam_policy_name         = local.name
  use_policy_name_prefix = true
  use_role_name_prefix   = true

  vpc_subnet_ids         = module.vpc.private_subnets
  vpc_security_group_ids = [module.rds_proxy_sg.security_group_id]

  db_proxy_endpoints = {
    read_write = {
      name                   = "read-write-endpoint"
      vpc_subnet_ids         = module.vpc.private_subnets
      vpc_security_group_ids = [module.rds_proxy_sg.security_group_id]
      tags                   = local.tags
    }
  }

  secrets = {
    "${local.db_username}" = {
      auth_scheme = "SECRETS"
      iam_auth    = "DISABLED"
      description = aws_secretsmanager_secret.superuser.description
      arn         = aws_secretsmanager_secret.superuser.arn
      kms_key_id  = aws_secretsmanager_secret.superuser.kms_key_id
    }
  }

  engine_family = "MYSQL"

  # Target Aurora cluster
  target_db_cluster     = true
  db_cluster_identifier = module.rds.cluster_id

  tags = local.tags
}

Steps to reproduce the behavior:

Expected behavior

Actual behavior

Error output

│ Error: Error creating DB Proxy: InvalidParameterValue: Must enable TLS, when IAM Auth is required
│ 	status code: 400, request id: de2093bc-e0b0-427c-9683-17d0bb843ece

Terminal Output Screenshot(s)

Additional context

RDS Proxy Endpoint Support

Is your feature request related to a problem? Please describe.
Proxy endpoint support just landed. Could we add this? 😄

Documentation: secrets.kms_key_id

The example shows this as a guid. It needs to be an arn, else the creation of the role will fail.

Issue applying with latest release version

Description

When creating a Proxy that works with a RDS Postgres instance, I receive the following error
╷ │ Error: only lowercase alphanumeric characters and hyphens allowed in "db_instance_identifier" │ │ with module.rds_proxy.aws_db_proxy_target.db_instance[0], │ on ../../main.tf line 63, in resource "aws_db_proxy_target" "db_instance": │ 63: db_instance_identifier = var.db_instance_identifier
I was able to reproduce this both in my configuration and the example configuration provided for postgres-iam-isntance. I was able to resolve this in my configuration by using the identifier attribute instead of the id of the RDS instance resource

Versions

• Module version [Required]:

• Terraform version:
  1.4.5

• Provider version(s):

• provider registry.terraform.io/hashicorp/aws v5.1.0
• provider registry.terraform.io/hashicorp/random v3.5.1

Reproduction Code [Required]

Was able to reproduce by running a terraform plan && terraform apply using the postgres-iam-instance example module provided.

Steps to reproduce the behavior:
Run a terraform plan & apply on the postgres-iam-instance example module provided

Expected behavior

Successful terraform apply

Actual behavior

Terraform failed to apply with an error message.

Terminal Output Screenshot(s)

Malformed IAM Policy

Description

Small bug in module, an IAM Policy is referencing a secret id instead of the arn.

• main.tf :: line 138

• ✋ I have searched the open/closed issues and my issue is not listed.

Versions

terraform {
  required_version = "=1.4.6"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.67.0"
    }
    null = {
      source  = "hashicorp/null"
      version = "3.2.1"
    }
    tls = {
      source  = "hashicorp/tls"
      version = "4.0.4"
    }
  }
}

Reproduction Code

module "rds_proxy" {
  source                  = "terraform-aws-modules/rds-proxy/aws"
  version                 = "~>2.1.2"

  create_proxy            = true

  name                    = "${local.project_name}-rds-proxy"
  iam_role_name           = "${local.project_name}-rds-proxy-role"
  vpc_subnet_ids          = module.subnets.private_subnet_ids
  vpc_security_group_ids  = [module.rds_proxy_sg.security_group_id]

  db_proxy_endpoints      = {
    read_only = {
      name                   = "read-only-endpoint"
      vpc_subnet_ids         = module.subnets.private_subnet_ids
      vpc_security_group_ids = [module.rds_proxy_sg.security_group_id]
      target_role            = "READ_ONLY"
      tags                   = local.rds_proxy_tags
    }
  }

  secrets                 = {
    "rdxproxyadmin" = {
      description = aws_secretsmanager_secret.rds_proxy.description
      arn         = aws_secretsmanager_secret.rds_proxy.arn
      kms_key_id  = aws_secretsmanager_secret.rds_proxy.kms_key_id
    }
  }

  engine_family           = "POSTGRESQL"
  debug_logging           = true
  idle_client_timeout     = 300

  # Target Aurora cluster
  target_db_cluster       = true
  db_cluster_identifier   = module.aurora_postgresql_v2.cluster_id

  tags                    = local.rds_proxy_tags
}

Steps to reproduce the behavior:

• Try running module with IAM role

Expected behavior

• IAM Policy to be created

Actual behavior

• Error: Malformed IAM Policy - Resource must be "*" or ARN

IAM Auth option's default value leading to misleading terraform plan

Description

Terraform plan for RDS Proxy consistently detects changes that are false positives,
as it is trying to set value null instead of "DISABLED" for auth.iam_auth.
I am not even using IAM auth for this proxy.

Versions

• Module version [Required]: 3.1.0
• Terraform version: 1.6.2
• Provider version(s): hashicorp/aws v5.25.0

Reproduction Code [Required]

Steps to reproduce the behavior:

I am not using workspaces,
I have cleared the local cache and performed again terraform init and terraform plan,

Sharing steps:

Considering the complexity of the command to create a RDS proxy (many inputs that depends on your own environment and subnets and secrets), I am sharing here an example in my own environment that may not be reproducible as is (need to set your values) in yours

module "aws_rds_proxy" {
  source  = "terraform-aws-modules/rds-proxy/aws"
  version = "3.1.0"

  name            = local.proxy_identifier
  iam_role_name   = "${local.proxy_identifier}-role"
  
  vpc_subnet_ids         = var.subnet_ids
  vpc_security_group_ids = var.security_group_ids
  require_tls            = var.require_tls

  endpoints = {
    read_write = {
      name                    = "read-write-endpoint"
      vpc_subnet_ids          = var.subnet_ids
      vpc_security_group_ids  = var.security_group_ids

    },
    read_only = {
      name                    = "read-only-endpoint"
      vpc_subnet_ids          = var.subnet_ids
      vpc_security_group_ids  = var.security_group_ids
      target_role             = "READ_ONLY"
    }
  }

  auth = {
    "superuser" = {
      description             = "Target database superuser password"
      secret_arn              = var.target_superuser_secret_arn
    }
  }

  # target RDS cluster or instance
  engine_family               = var.target_engine_family
  target_db_cluster           = var.target_type == "cluster"
  db_cluster_identifier       = var.target_identifier
}

Expected behavior

Given a basic configuration of a RDS Proxy as the one reported,
Given a terraform apply was completed successfully before,
When a terraform plan command is executed,
Then the command should result in an empty plan

Actual behavior

Given a basic configuration of a RDS Proxy as the one reported,
Given a terraform apply was completed successfully before,
When a terraform plan command is executed,
the command consistently results in a non-empty plan as follows:

  # module.aurora_cluster_postgres.module.aurora_postgres_aws_rds_proxy[0].module.aws_rds_proxy.aws_db_proxy.this[0] will be updated in-place
  ~ resource "aws_db_proxy" "this" {
        id                     = "pan-dev-aurora-postgres-cluster-proxy"
        name                   = "pan-dev-aurora-postgres-cluster-proxy"
        tags                   = {}
        # (10 unchanged attributes hidden)

      ~ auth {
          - iam_auth                  = "DISABLED" -> null
            # (4 unchanged attributes hidden)
        }
    }

Terminal Output Screenshot(s)

Additional context