e-mit / aws-lambda-get

Create a repeating AWS Lambda function to GET HTTP json data and transfer it into an AWS SQS queue

AWS Lambda GET

Creates an AWS Lambda function which GETs json data from an HTTP endpoint, on a repeating periodic schedule, and puts the data into an AWS SQS queue.

This project configures AWS using the CLI with a CloudFormation/SAM template. The SQS queue must already exist.

See also

• github.com/e-mit/aws-lambda-db creates an AWS Lambda function which receives data from a AWS SQS queue and stores it in an SQL database.
• github.com/e-mit/aws-create-db creates and configures an AWS Relational Database Service (RDS) instance running PostgreSQL.
• github.com/e-mit/aws-ec2-grafana for configuring and deploying Grafana on an EC2 instance to display a public data dashboard

Readme Contents

• Testing
  
• Deployment
  
• Development
  
• To do
  

Testing

Tests and linting checks run via GitHub actions after each push. Tests can be run locally (no interaction with AWS), or with AWS (cloud tests).

Local tests

Run all tests and linting with local-tests.sh

Cloud tests

These require AWS CLI authentication. Run with cloud-tests.sh

Deployment

1. Provide values for the environment variables listed in example_config.sh
2. Execute script setup.sh. This will create the resources and start the lambda.
3. Optional: change log level using: ./stack.sh <stack name> loglevel <log level string e.g. DEBUG>
4. Stop the lambda and delete all resources using: ./stack.sh <stack name> delete

Development

After deploying the stack, the lambda code and the packaged Python library dependencies can be updated independently, and rapidly, using the following commands:

• Lambda function update: ./stack.sh <stack name> update_function
• Python packages update: ./stack.sh <stack name> update_layer

To do

• Optional headers to send with the request
• Optional body data to send with the request
• Optional cookie data to send with the request
• Support API authentication