spring-media / terraform-aws-lambda

A Terraform module to create AWS Lambda ressources.

AWS Lambda Terraform module

---

Deprecation warning

Further development of this module will be continued in moritzzimmer/terraform-aws-lambda. Users of spring-media/lambda/aws should migrate to this module as a drop-in replacement for all provisions up to release/tag 5.2.1 to benefit from new features and bugfixes.

module "lambda" {
  source           = "moritzzimmer/lambda/aws"
  version          = "5.2.1"
  filename         = "my-package.zip"
  function_name    = "my-function"
  handler          = "my-handler"
  runtime          = "go1.x"
  source_code_hash = filebase64sha256("${path.module}/my-package.zip")
}

---

Terraform module to create AWS Lambda resources with configurable event sources, IAM configuration (following the principal of least privilege), VPC as well as SSM/KMS and log streaming support.

The following event sources are supported (see examples):

• cloudwatch-event: configures a CloudWatch Event Rule to trigger the Lambda by CloudWatch event pattern or on a regular, scheduled basis
• dynamodb: configures an Event Source Mapping to trigger the Lambda by DynamoDb events
• kinesis: configures an Event Source Mapping to trigger the Lambda by Kinesis events
• s3: configures permission to trigger the Lambda by S3
• sns: to trigger Lambda by SNS Topic Subscription
• sqs: configures an Event Source Mapping to trigger the Lambda by SQS events

Furthermore this module supports:

• reading configuration and secrets from AWS Systems Manager Parameter Store including decryption of SecureString parameters
• CloudWatch Log group configuration including retention time and subscription filters e.g. to stream logs via Lambda to Elasticsearch

Terraform version compatibility

module	terraform	branch
4.x.x	0.12+	master
3.x.x	0.11.x	terraform_0.11x

How do I use this module?

The module can be used for all runtimes supported by AWS Lambda.

Deployment packages can be specified either directly as a local file (using the filename argument) or indirectly via Amazon S3 (using the s3_bucket, s3_key and s3_object_versions arguments), see documentation for details.

basic

provider "aws" {
  region = "eu-west-1"
}

module "lambda" {
  source           = "spring-media/lambda/aws"
  version          = "5.2.1"
  filename         = "my-package.zip"
  function_name    = "my-function"
  handler          = "my-handler"
  runtime          = "go1.x"
  source_code_hash = filebase64sha256("${path.module}/my-package.zip")
}

with event trigger

module "lambda" {
  // see above
  
  event = {
    type                = "cloudwatch-event"
    schedule_expression = "rate(1 minute)"
  }
}

in a VPC

module "lambda" {
  // see above

  vpc_config = {
    security_group_ids = ["sg-1"]
    subnet_ids         = ["subnet-1", "subnet-2"]
  }
}

with access to parameter store

module "lambda" {
  // see above

  ssm_parameter_names = ["some/config/root/*"]
  kms_key_arn         = "arn:aws:kms:eu-west-1:647379381847:key/f79f2b-04684-4ad9-f9de8a-79d72f"
}

with log subscription (stream to ElasticSearch)

module "lambda" {
  // see above

  logfilter_destination_arn = "arn:aws:lambda:eu-west-1:647379381847:function:cloudwatch_logs_to_es_production"
}

Examples

• example-with-cloudwatch-event
• example-with-dynamodb-event
• example-with-kinesis-event
• example-with-s3-event
• example-with-sns-event
• example-with-sqs-event
• example-with-vpc
• example-without-event

How do I contribute to this module?

Contributions are very welcome! Check out the Contribution Guidelines for instructions.

How is this module versioned?

This Module follows the principles of Semantic Versioning. You can find each new release in the releases page.

During initial development, the major version will be 0 (e.g., 0.x.y), which indicates the code does not yet have a stable API. Once we hit 1.0.0, we will make every effort to maintain a backwards compatible API and use the MAJOR, MINOR, and PATCH versions on each release to indicate any incompatibilities.

terraform-aws-lambda's Issues

enhance and clarify secrets management

There are different possibilities and recommendations how to manage and access secrets (e.g. database passwords) inside Lambda functions (see e.g here and here).

Currently this module supports reading (optionally encrypted) parameters from AWS Systems Manager Parameter Store at runtime by creating IAM policies allowing access to and decryption of parameters by setting ssm_parameter_names and kms_key_arn. This is the recommended way for Lambda functions if the Parameter Store API limits are no concern in case of horizontal scaling.

Unfortunately kms_key_arn conflicts with the parameter specified in the Terraform Lambda ressource to specify a key that is used to encrypt environment variables.

Proposal:

• create a new configuration option (e.g. ssm { parameters: [], kms_key_arn: ""}) to configure IAM policies for runtime SSM access (with custom key)
• switch (optional) variable kms_key_arn to it's default meaning an pass it down to lambda submodule
• (optionally) support configuration option for using AWS Secrets Manager

Inconsistent variables differ from defaults

The publish variable defaults to true in the main module, https://github.com/spring-media/terraform-aws-lambda/blob/master/variables.tf#L60. However in the submodule, it defaults false, https://github.com/spring-media/terraform-aws-lambda/blob/master/modules/lambda/variables.tf#L43.

The docs indicate the default is false, https://www.terraform.io/docs/providers/aws/r/lambda_function.html#publish. Was there an intentional reason to set to true?

Deprecation warning

Further development of this module will be continued in moritzzimmer/terraform-aws-lambda.

Users of spring-media/lambda/aws should migrate to this module as a drop-in replacement for all provisions up to release/tag 5.2.1 to benefit from new features and bugfixes.

module "lambda" {
  source           = "moritzzimmer/lambda/aws"
  version          = "5.2.1"
  filename         = "my-package.zip"
  function_name    = "my-function"
  handler          = "my-handler"
  runtime          = "go1.x"
  source_code_hash = filebase64sha256("${path.module}/my-package.zip")
}

Role and function name should be distinct

We should be giving the lambda executing role name a distinct name (e.g prefix it with tf) in order to avoid confusion:

role = "${module.lambda.function_name}"

Support Policy variables for Lambda module

in case consumers need more permissions that CloudWatch Logs (e.g. Parameter Store, KMS, ...)

Request for an example of Lambda pulling from SQS and provisioning all the involved resources.

Hello,

Thanks for this useful component.

I would like to request an example of https://github.com/spring-media/terraform-aws-lambda/tree/master/examples/example-with-sqs-event involving the creation of the required SQS queue instead of hard-coding the ARN on it.

allow ssm:GetParameter with ssm config

Currently this module configures the following policy for ssm_parameter_names:

data "aws_iam_policy_document" "ssm_policy_document" {
  count = length(var.ssm_parameter_names)

  statement {
    actions = [
      "ssm:GetParameters",
      "ssm:GetParametersByPath",
    ]

    resources = [
      "arn:aws:ssm:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:parameter/${element(var.ssm_parameter_names, count.index)}",
    ]
  }
}

This datasource should also permit ssm:GetParameter.

Apply AWSLambdaBasicExecutionRole

AWS console displays a new warning sign about missing CloudWatch logs permissions:

This is not true since the permissions from AWSLambdaBasicExecutionRole are applied by this module. Nevertheless we should add this role to the permissions to get rid of the banner.

Additional permissions

Allow for a new role or additional permissions be added to the role

The module always attempts to recreate the lambda, even with no modifications

Every time i run my terraform the lambda module thinks it needs recreation. This is the output I get when running a plan:

resource "aws_lambda_function" "lambda" {
...
...
      ~ last_modified                  = "2020-07-07T14:23:45.641+0000" -> (known after apply)
        layers                         = []
        memory_size                    = 128
...
...

Is there a way I can avoid this? last_modified is the only attribute that needs changing. It would be good if we could pass some variable so the resource can have a lifecycle (or it could be just added to the resource):

lifecycle {
    ignore_changes = [
      last_modified
    ]
  }

Many thanks.

Unified cloudwatch event rule

Lambda functions can be triggered from CloudWatch either by event_pattern or schedule_expression , see https://www.terraform.io/docs/providers/aws/r/cloudwatch_event_rule.html.

Currently only the latter is supported in this module. Idea: deprecate the current cloudwatch-scheduled-event sub-module in favor of a cloudwatch sub-module supporting both.

CloudWatch Alarms