AWS . LAMBDA . WORKER

An AWS Lambda Worker that is triggered by SNS Topic in order to index message to an Elastic Search Service.

Build the lambda zip file and upload to S3

First, click on the file package.bat to generate AWS.Lambda.Worker.zip and upload it to the S3 bucket (must be global unique name). Here I chose mine, by instance:

com.aws.lambda.worker.cloudformation

Deploy the setup to AWS

Deploy with the cloud formation template

You need to upload the template and fill the parameters directly in the AWS console.

lambda-xray.yaml

This template will :

retrieve the Lambda zip artifact to deploy it to AWS Lambda Service.
create the Lambda Role with CloudWatch + X-Ray + Elastic Service permissions.
create the SNS Topic.

Deploy with the bat file

First, you need to edit this bat file according to your region (see parameter --region) :

cloudformation/
  |
  |->aws-cli-deploy.bat

You need to configure the cloud formation parameters in the file :

cloudformation/
  |
  |->parameters.json

Customize with your own values :

your S3 bucket (must change to be global and unique name)
your elastic search service url
your SNS Topic
..etc

When clicking the bat file, these parameters will be passed to the cloud formation template.

Important note on the Lambda function timeout

As it takes like more than 10s the first time it is initializing the elastic search client (and indexing), the lambda function should be configured with a timeout of 15s minimum, otherwise it will not run and trace any logs in CloudWatch.

See parameter LambdaTimeoutParam in the cloud formation template.

Deploy solely the lambda worker artifact to AWS

This additionnal section details a way to upload the lambda worker manually.
This will not create the whole setup (create SNS topic, create Lambda Role ..etc).

Publish via Visual Studio

On the project, right click to publish:

Fill the options. Please note that these values will edit the file aws-lambda-tools-defaults.json accordingly.

xxxaccountidxxx           -> will be your AWS account ID
xxxelasticserviceurlxxx   -> will be your elastic search service url (environment variable value)

Testing the application

Publish message in SNS

In order to trigger the lambda function, publish a message using the generated SNS topic, using the following json structure:

	{
		"Title": "Welcome message sample",
		"Content": "Hello world !"
	}

Checking the results

You can check logs in AWS CloudWatch.
You can trace and analyze with X-Ray.
You can check the message being indexed under Elastic Seach Service (using elastic queries and/or Kibana interface).

Details on the lambda function implementation

Please note the lambda function is implemented in dotnetcore2.1 as AWS manages this version.

In order to run, the lambda function ...

is triggered when a message is published to the SNS topic.
indexed the message to an elastic search service.
The url of this service is passed as the environment variable "ESurl".
When the environment variable is not existing, there is a fallback configuration (see file appsettings.json).
But this is not recommended as it is not convenient to configured this way.

Useful links

Add an IAM ROLE for the Lambda to execute actions on AWS ElasticSearch service

danmgs / aws.lambda.worker Goto Github PK