giuliocalzolari / terraform-aws-es-cleanup

AWS Elasticsearch Lambda Curator

AWS Lambda Elasticsearch Index Cleanup

Overview

This AWS Lambda function allows you to delete the old Elasticsearch indexes using SigV4Auth authentication. You configure the AWS Elasticsearch Access Policy authorizing the Lambda Role or the AWS Account number instead of using the IP address whitelist.

Diagram

Terraform version

Module compatible with Terraform 0.12

Module Input Variables

Providers

Name	Version
archive	n/a
aws	n/a
http	n/a
local	n/a
null	n/a

Inputs

Name	Description	Type	Default	Required
delete_after	Numbers of days to preserve	number	15	no
es_endpoint	AWS ES FQDN e.g. search-es-demo-xxxxxxxxxx.eu-west-1.es.amazonaws.com	string	n/a	yes
index	Index/indices to process using regex, except the one matching skip_index regex	string	".*"	no
index_format	Combined with 'index' varible is used to evaluate the index age	string	"%Y.%m.%d"	no
prefix	A prefix for the resource names, this helps create multiple instances of this stack for different environments	string	""	no
python_version	Lambda Python version to be used	string	"3.6"	no
schedule	Cloudwatch Cron Schedule expression for running the cleanup function	string	"cron(0 3 * * ? *)"	no
security_group_ids	Addiational Security Ids To add.	list(string)	[]	no
skip_index	Index/indices to skip	string	".kibana*"	no
subnet_ids	Subnet IDs you want to deploy the lambda in. Only fill this in if you want to deploy your Lambda function inside a VPC.	list(string)	[]	no
suffix	A suffix for the resource names, this helps create multiple instances of this stack for different environments	string	""	no
tags	Tags to apply	map	{ "Name": "es-cleanup" }	no
timeout	Maximum lambda execution time	number	300	no

Outputs

No output.

Example

terraform {
  required_version = ">= 0.12"
}

provider "aws" {
  region = "eu-west-1"
}


module "public_es_cleanup" {
  source       = "giuliocalzolari/es-cleanup/aws"
  version      = "1.14.0"
  prefix       = "public_es_"
  es_endpoint  = "test-es-XXXXXXX.eu-central-1.es.amazonaws.com"
  delete_after = 365
}


module "vpc_es_cleanup" {
  source             = "giuliocalzolari/es-cleanup/aws"
  version            = "1.14.0"
  prefix             = "vpc_es_"
  es_endpoint        = "vpc-gc-demo-vpc-gloo5rzcdhyiykwdlots2hdjla.eu-central-1.es.amazonaws.com"
  index              = "all"
  delete_after       = 30
  subnet_ids         = ["subnet-d8660da2"]
  security_group_ids = ["sg-02dd3aa6da1b5"]
}

Authors

Module is maintained by Giulio Calzolari with help from these awesome contributors.

License

terraform-aws-es-cleanup is licensed under the Apache Software License 2.0. Originally developed by Cloudreach adapted in this repo due to Terraform Registry Requirements

terraform-aws-es-cleanup's Issues

src url 404

terraform-aws-es-cleanup/lambda.tf

Line 2 in a4c4c3e

url = "https://raw.githubusercontent.com/cloudreach/aws-lambda-es-cleanup/master/es-cleanup.py"

The correct url has an underscore in the file name.
https://raw.githubusercontent.com/cloudreach/aws-lambda-es-cleanup/master/es_cleanup.py

Invalid type specification

Using terraform v0.12.6

Error: Invalid type specification

  on .terraform/modules/es_logging_cleanup/giuliocalzolari-terraform-aws-es-cleanup-b36108f/variables.tf line 56, in variable "subnet_ids":
  56:   type        = list()

The list type constructor requires one argument specifying the element type.


Error: Invalid type specification

  on .terraform/modules/es_logging_cleanup/giuliocalzolari-terraform-aws-es-cleanup-b36108f/variables.tf line 62, in variable "security_group_ids":
  62:   type        = list()