aws-samples / aws-sam-terraform-examples

AWS SAM and Terraform

This repository contains a sample book reviews serverless application written in Terraform. Full instructions on how to use this repository are contained in blog post Better Together: AWS SAM CLI and Hashicorp Terraform

SAM support for Terraform GA examples

The GA folder of the repository contains the demo applications for the GA blog. See the README.md for more.

aws-sam-terraform-examples's Issues

Lambda broken due to: [ERROR] Runtime.ImportModuleError: Unable to import module 'index': urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips 26 Jan 2017.

Currently getting this error in Cloudwatch Logs with a GET to /serverless_lambda_stage.

[ERROR] Runtime.ImportModuleError: Unable to import module 'index': urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips 26 Jan 2017. See: urllib3/urllib3#2168

Specify shell type for build script

Shell type is not specified resulting in '[[' and 'pushd' from failing when not using bash.

null_resource.build_lambda_function (local-exec): Executing: ["/bin/sh" "-c" "./py_build.sh \"./src\" \"build\" \"publishBookReview.zip\" Function"]
null_resource.build_lambda_function (local-exec): building Function ./src into build
null_resource.build_lambda_function (local-exec): ./py_build.sh: 11: [[: not found
null_resource.build_lambda_function (local-exec): /workspaces/aws-sam-terraform-examples/aws-sam-terraform-examples/zip_based_lambda_functions/api-lambda-dynamodb-example

null_resource.build_lambda_function (local-exec): ./py_build.sh: 23: pushd: not found
null_resource.build_lambda_function (local-exec): mv: cannot stat 'build/tmp_building/publishBookReview.zip': No such file or directory
null_resource.build_lambda_function: Creation complete after 35s [id=7104882419102392583]

Error Message: Could not locate source_path "../src/auth/". Paths are relative to directory where `terraform plan` is being run (...)

I'm following this guide which was posted just two months ago, and I am trying to replicate the steps in the demo (api gateway v1 REST)

Upon cloning the repo and going into ga/api_gateway_v1/tf_resources, I get stuck when trying to run sam build.

Intended Outcome

Build succeeds and from here on I should be able to run sam local invoke

Actual Outcome

% sam build --hook-name terraform --terraform-project-root-path ../

Running Prepare Hook to prepare the current application                                    
Executing prepare hook of hook "terraform"                                                 
Skipping preparation stage, the metadata file already exists at                            
/Users/dev/_learning/aws-sam-terraform-examples/ga/api_gateway_v1/tf-resources/.aw
s-sam-iacs/iacs_metadata/template.json                                                     
Prepare hook completed and metadata file generated at:                                     
/Users/dev/_learning/aws-sam-terraform-examples/ga/api_gateway_v1/tf-resources/.aw
s-sam-iacs/iacs_metadata/template.json                                                     
Building codeuri:                                                                          
/Users/dev/_learning/aws-sam-terraform-examples/ga/api_gateway_v1/src/auth        
runtime: python3.9 metadata: {'SkipBuild': False, 'BuildMethod': 'makefile', 'ContextPath':
'/Users/dev/_learning/aws-sam-terraform-examples/ga/api_gateway_v1/tf-resources/.a
ws-sam-iacs/iacs_metadata', 'WorkingDirectory':                                            
'/Users/dev/_learning/aws-sam-terraform-examples/ga/api_gateway_v1/tf-resources', 
'ProjectRootDirectory':                                                                    
'/Users/dev/_learning/aws-sam-terraform-examples/ga/api_gateway_v1/tf-resources'} 
architecture: x86_64 functions: module.lambda_function_auth.aws_lambda_function.this[0]    
ModuleLambdaFunctionAuthAwsLambdaFunctionThis0A0AF47A8: Running                            
CustomMakeBuilder:CopySource                                                               
ModuleLambdaFunctionAuthAwsLambdaFunctionThis0A0AF47A8: Running CustomMakeBuilder:MakeBuild
ModuleLambdaFunctionAuthAwsLambdaFunctionThis0A0AF47A8: Current Artifacts Directory :      
/Users/dev/_learning/aws-sam-terraform-examples/ga/api_gateway_v1/tf-resources/.aw
s-sam/build/ModuleLambdaFunctionAuthAwsLambdaFunctionThis0A0AF47A8                         
python3 ".aws-sam-iacs/iacs_metadata/copy_terraform_built_artifacts.py" --expression "|values|root_module|child_modules|[?address==module.lambda_function_auth]|resources|[?address==\"module.lambda_function_auth.null_resource.sam_metadata_aws_lambda_function[0]\"]|values|triggers|built_output_path" --directory "/Users/dev/_learning/aws-sam-terraform-examples/ga/api_gateway_v1/tf-resources/.aws-sam/build/ModuleLambdaFunctionAuthAwsLambdaFunctionThis0A0AF47A8" --target "module.lambda_function_auth.null_resource.sam_metadata_aws_lambda_function[0]"

Initializing the backend...

Successfully configured the backend "local"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Reusing previous version of hashicorp/local from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Using previously-installed hashicorp/null v3.2.2
- Using previously-installed hashicorp/aws v4.67.0
- Using previously-installed hashicorp/external v2.3.2
- Using previously-installed hashicorp/local v2.4.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
module.lambda_function_auth.data.external.archive_prepare[0]: Reading...
╷
│ Warning: Resource targeting is in effect
│ 
│ You are creating a plan with the -target option, which means that the
│ result of this plan may not represent all of the changes requested by the
│ current configuration.
│ 
│ The -target option is not for routine use, and is provided only for
│ exceptional situations such as recovering from errors or mistakes, or when
│ Terraform specifically suggests to use it as part of an error message.
╵

Build Failed
Error: CustomMakeBuilder:MakeBuild - Make Failed: ╷
│ Error: External Program Execution Failed
│ 
│   with module.lambda_function_auth.data.external.archive_prepare[0],
│   on .terraform/modules/lambda_function_auth/package.tf line 10, in data "external" "archive_prepare":
│   10:   program = [local.python, "${path.module}/package.py", "prepare"]
│ 
│ The data source received an unexpected error while attempting to execute
│ the program.
│ 
│ Program: /opt/homebrew/bin/python3
│ Error Message: Could not locate source_path "../src/auth/".  Paths are
│ relative to directory where `terraform plan` is being run
│ ("/private/var/folders/2l/7b1lkwg518l19gmfxshs2p7m0000gn/T/tmp96dbm08v")
│ 
│ State: exit status 1
╵
Traceback (most recent call last):
  File "/private/var/folders/2l/7b1lkwg518l19gmfxshs2p7m0000gn/T/tmp96dbm08v/.aws-sam-iacs/iacs_metadata/copy_terraform_built_artifacts.py", line 371, in <module>
    subprocess.check_call(["terraform", "apply", "-target", target, "-replace", target, "-auto-approve"])
  File "/opt/homebrew/Cellar/[email protected]/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['terraform', 'apply', '-target', 'module.lambda_function_auth.null_resource.sam_metadata_aws_lambda_function[0]', '-replace', 'module.lambda_function_auth.null_resource.sam_metadata_aws_lambda_function[0]', '-auto-approve']' returned non-zero exit status 1.
make: *** [build-ModuleLambdaFunctionAuthAwsLambdaFunctionThis0A0AF47A8] Error 1

Something that stood out to me is that terraform plan is being run at │ ("/private/var/folders/2l/7b1lkwg518l19gmfxshs2p7m0000gn/T/tmp96dbm08v") -- is this behavior intended?

Here are potentially relevant details:

MacOS Ventura 13.0, M1 Chip
Python path /opt/homebrew/bin/python3
Docker version 23.0.5, build bc4487a59e
SAM CLI, version 1.103.0

For what it's worth, running terraform init, terraform plan, terraform apply works as intended, but the sam commands seem to give me a path issue.

Any potential hints?

How to deploy code change? Example doesn't deploy if just an application code change.

I was able to deploy with Terraform the example at zip_based_lambda_functions/lambda-example/main.tf. However, if I just change a line in the zip_based_lambda_functions/api-lambda-dynamodb-example/src/index.py and do another terraform apply, the output rebuilds the file, yes, but it never deploys it.

This article doesn't suggest how to deploy changes.
https://aws.amazon.com/blogs/compute/better-together-aws-sam-cli-and-hashicorp-terraform/

Here is the tail of an apply:

Dangerous potenital to delete all files

If there is a problem with the ${build_path} var, or the py_build.sh is run directly in an attempt to test the build (therefore, no ${build_path} var), then this line will delete all user files.

aws-sam-terraform-examples/zip_based_lambda_functions/api-lambda-dynamodb-example/py_build.sh

Line 18 in 6e6c7a7

rm -rf ${build_path}/*

Please add validation to ensure it has a value and forcing a base working path to start from.

Issues with pip dependency with this example

I'm trying to go use this example in order to deploy my lambda functions using terraform with their python library dependencies.

However, when following this example, I hit this error.

null_resource.build_lambda_function (local-exec): pip : The term 'pip' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling

Is there a step that needs to be done to insure terraform has pip installed when it is deploying to aws? If so, where would this go in your examples?

Terraform Issue in api-lambda-dynamodb-example

api-lambda-dynamodb-example % terraform plan -out tfplan
╷
│ Error: "assume_role_policy" contains an invalid JSON policy: leading space characters are not allowed
│ 
│   with aws_iam_role.iam_for_lambda,
│   on main.tf line 67, in resource "aws_iam_role" "iam_for_lambda":
│   67:   assume_role_policy = <<EOF
│   68:     {
│   69:     "Version": "2012-10-17",
│   70:     "Statement": [
│   71:         {
│   72:         "Action": "sts:AssumeRole",
│   73:         "Principal": {
│   74:             "Service": "lambda.amazonaws.com"
│   75:         },
│   76:         "Effect": "Allow",
│   77:         "Sid": ""
│   78:         }
│   79:     ]
│   80:     }
│   81:     EOF

This is the unsightly workaround that I used (dedent the line after <<EOF):

resource "aws_iam_role" "iam_for_lambda" {
  name = "iam_for_lambda"

  assume_role_policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
        "Action": "sts:AssumeRole",
        "Principal": {
            "Service": "lambda.amazonaws.com"
        },
        "Effect": "Allow",
        "Sid": ""
        }
    ]
    }
    EOF

Ability to generate a "Lambda Application"?

Hello,

We're currently using Terraform and AWS SAM together and the lines are just blurry enough in terms of where certain things should live (like Cloudwatch alarms) that this sparked our interest as a way to continue to define all of our infrastructure in TF but continue to leverage AWS SAM for local development.

For extra context, the reason we're using TF and SAM together is that TF sets up the base infra (DB, Redis, etc...) and handles the Route53, SSM, Cloudfront, etc... infra for review apps / staging / production. Our devs started with SAM and we worked TF into the mix after.

Through my research and testing, there does not appear to be a way to create a Lambda application (set of functions grouped together) with Terraform and if I understand AWS SAM correctly, the application is what is run when we invoke sam local start-api.

Here is an excerpt of the SAM template generated by the example code in this repo:

AwsLambdaFunctionPublishBookReviewBFC97F47:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: publish-book-review
      Architectures:
      - x86_64
      Environment:
        Variables:
          DYNAMODB_TABLE_NAME: BookReviews
      Code: AwsLambdaFunctionPublishBookReviewBFC97F47
      Handler: index.lambda_handler
      PackageType: Zip
      Runtime: python3.8
      Layers: []
      Timeout: 30
    Metadata:
      BuildMethod: makefile
      ContextPath: ...
      ProjectRootDirectory: ...
      SamResourceId: aws_lambda_function.publish_book_review
      SkipBuild: false
      WorkingDirectory: ...

And here's an excerpt for a sample function we have in one of our SAM apps:

SampleFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: "src/api/sample"
      Handler: sample.lambda_handler
      Runtime: python3.9
      Tracing: Active
      Environment:
        Variables:
          REDIS_HOST: "redis-host"
          REDIS_KEY_PREFIX: "redis-key-prefix"
          DB_HOST: "database-host"
          DB_USERNAME: "database-username"
          DB_PASSWORD: "database-password"
          DB_NAME: "database-name"
          DB_PORT: "3306"
      Architectures:
        - x86_64
      Events:
        Sample:
          Type: Api
          Properties:
            Path: "/sample/{proxy+}"
            Method: "post"
      Layers:
        - !Ref SomeSharedLibs

Is there a way to achieve the same SAM template configuration? (Notice the missing Events section in the example-generated yaml.

Thanks!