terraform-community-modules / tf_aws_ecs

[DEPRECATED] Use https://github.com/terraform-aws-modules/terraform-aws-ecs

ecs terraform module

A terraform module to provide ECS clusters in AWS.

This Module currently supports Terraform 0.10.x, but does not require it. If you use tfenv, this module contains a .terraform-version file which matches the version of Terraform we currently use to test with.

Module Input Variables

Required

• name - ECS cluster name
• key_name - An EC2 key pair name
• subnet_id - A list of subnet IDs
• vpc_id - The VPC ID to place the cluster in

Optional

NOTE About User Data: The user_data parameter overwrites the user_data template used by this module, this will break some of the module features (e.g. docker_storage_size, dockerhub_token, and dockerhub_email). However, additional_user_data_script will concatenate additional data to the end of the current user_data script. It is recomended that you use additional_user_data_script. These two parameters are mutually exclusive - you can not pass both into this module and expect it to work.

• additional_user_data_script - Additional user_data scripts content
• ebs_block_device - EBS block devices to attach to the instance. (default: /dev/xvdcz)
• region - AWS Region - defaults to us-east-1
• servers - Number of ECS Servers to start in the cluster - defaults to 1
• min_servers - Minimum number of ECS Servers to start in the cluster - defaults to 1
• max_servers - Maximum number of ECS Servers to start in the cluster - defaults to 10
• instance_type - AWS instance type - defaults to t2.micro
• load_balancers - List of elastic load balancer (classic only) names to put in front of your instances - defaults to []
• iam_path - IAM path, this is useful when creating resources with the same name across multiple regions. Defaults to /
• associate_public_ip_address - assign a publicly-routable IP address to every instance in the cluster - default: false.
• docker_storage_size - EBS Volume size in Gib that the ECS Instance uses for Docker images and metadata - defaults to 22
• dockerhub_email - Email Address used to authenticate to dockerhub. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html
• dockerhub_token - Auth Token used for dockerhub. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html
• extra_tags - Additional tags to be added to the ECS autoscaling group. Must be in the form of an array of hashes. See https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html for examples.

extra_tags = [
    {
      key                 = "consul_server"
      value               = "true"
      propagate_at_launch = true
    },
  ]

• allowed_cidr_blocks - List of subnets to allow into the ECS Security Group. Defaults to ["0.0.0.0/0"].

• ami - A specific AMI image to use, eg ami-95f8d2f3. Defaults to the latest ECS optimized Amazon Linux AMI.

• ami_version - Specific version of the Amazon ECS AMI to use (e.g. 2016.09). Defaults to *. Ignored if ami is specified.

• heartbeat_timeout - Heartbeat Timeout setting for how long it takes for the graceful shutdown hook takes to timeout. This is useful when deploying clustered applications like consul that benifit from having a deploy between autoscaling create/destroy actions. Defaults to 180"

• asg_delete_extra_timeout - Extra time that terraform apply will wait for ASG deletion (default 600). This is added on top of heartbeat_timeout. This variable is customizable for when the instances take longer than 600sec to shut down once shutdown is initiated.

• security_group_ids - a list of security group IDs to apply to the launch configuration

• user_data - The instance user data (e.g. a cloud-init config) to use in the aws_launch_configuration

• custom_iam_policy - JSON containing the custom IAM policy for ECS nodes. Will overwrite the default one if set.

• consul_image - Image to use when deploying consul, defaults to the hashicorp consul image

• registrator_image - Image to use when deploying registrator agent, defaults to the gliderlabs registrator:latest

• consul_memory_reservation - The soft limit (in MiB) of memory to reserve for the container, defaults 20

• registrator_memory_reservation - The soft limit (in MiB) of memory to reserve for the container, defaults 20

• enable_agents - Enable Consul Agent and Registrator tasks on each ECS Instance. Defaults to false

• spot_bid_price - Use spot instances and request this bid price. Note that with this option you risk your instances shutting down if the market price rises above your bid price.

• enabled_metrics - A list of metrics to collect.

Usage

module "ecs-cluster" {
  source    = "github.com/terraform-community-modules/tf_aws_ecs"
  name      = "infra-services"
  servers   = 1
  subnet_id = ["subnet-6e101446"]
  vpc_id    = "vpc-99e73dfc"
}

Example cluster with consul and Registrator

In order to start the Consul/Registrator task in ECS, you'll need to pass in a consul config into the additional_user_data_script script parameter. For example, you might pass something like this:

Please note, this module will try to mount /etc/consul/ into /consul/config in the container and assumes that the consul config lives under /etc/consul on the docker host.

/bin/mkdir -p /etc/consul
cat <<"CONSUL" > /etc/consul/config.json
{
	"raft_protocol": 3,
	"log_level": "INFO",
	"enable_script_checks": true,
  "datacenter": "${datacenter}",
	"retry_join_ec2": {
		"tag_key": "consul_server",
		"tag_value": "true"
	}
}
CONSUL

data "template_file" "ecs_consul_agent_json" {
  template = "${file("ecs_consul_agent.json.sh")}"

  vars {
    datacenter = "infra-services"
  }
}

module "ecs-cluster" {
  source                      = "github.com/terraform-community-modules/tf_aws_ecs"
  name                        = "infra-services"
  servers                     = 1
  subnet_id                   = ["subnet-6e101446"]
  vpc_id                      = "vpc-99e73dfc"
  additional_user_data_script = "${data.template_file.ecs_consul_agent_json.rendered}"
  enable_agents               = true
}

Outputs

• cluster_id - (String) ECS Cluster id for use in ECS task and service definitions.
• cluster_name - (String) ECS Cluster name that can be used for CloudWatch app autoscaling policy resource_id.
• autoscaling_group (Map) A map with keys id, name, and arn of the aws_autoscaling_group created.
• iam_role (Map) A map with keys arn and name of the iam_role created.
• security_group (Map) A map with keys id, name, and arn of the aws_security_group created.

Authors

• Tim Hartmann
• Joe Stump
• Michal

License

MIT

tf_aws_ecs's Issues

ECS Instance Draining on scale in

This module might benefit from adding this module terraform-community-modules/tf_aws_ecs_instance_draining_on_scale_in as a dependency. Not sure though yet if this is desirable or not...

validate failed

$ terraform validate
Error loading files Error loading github/terraform-community-modules/tf_aws_ecs/main.tf: Error reading config for aws_launch_configuration[ecs]: parse error at 1:10: expected expression but found "["

$ terraform version
Terraform v0.9.11

Release tags/v2.0.0 is fine. Problem is between v2.0.0 and v2.1.0

[BUG] The argument "tags" was already set at .terraform/modules/ecs-cluster/main.tf:62,3-7. Each argument may be set only once.

Error: Attribute redefined

  on .terraform/modules/ecs-cluster/main.tf line 68, in resource "aws_autoscaling_group" "ecs":
  68:   tags = ["${var.extra_tags}"]

The argument "tags" was already set at
.terraform/modules/ecs-cluster/main.tf:62,3-7. Each argument may be set only
once.

I get this error while trying a terraform plan on the following main.tf:

provider "aws" {
  region = "us-east-1"
}

module "ecs-cluster" {
  source = "github.com/terraform-community-modules/tf_aws_ecs"
  name = "infra-services"
  servers = 2
  subnet_id = [
    "subnet-6e101446"]
  vpc_id = "vpc-99e73dfc"
  extra_tags = [
    {
      key = "consul_server"
      value = "true"
      propagate_at_launch = true
    },
  ]
}

Not working for windows ECS

This is not working for windows. The error I get seemed to be related to the way you are adding an EBS volume?

Trying this:

data "aws_ami" "win_ecs" {
  owners = ["amazon"]
  most_recent = true
  filter {
    name = "name"
    values = ["Windows_Server-2016-English-Full-ECS_Optimized*"]
  }
}

module "ecs-cluster" {
  source    = "github.com/terraform-community-modules/tf_aws_ecs"
  name      = "mycluster"
  subnet_id = ["subnet-1235434","subnet-123345"]
  vpc_id    = "vpc-123345"
  security_group_ids = ["sg-1233w45"]
  key_name = "mykey"
  region = "eu-central-1"
  ami = "${data.aws_ami.win_ecs.id}"

  servers   = 2
  instance_type = "t2.medium"
  docker_storage_size = 27
}

And I get this error:

* aws_autoscaling_group.ecs: "asg-ecs-.......": Waiting up to 10m0s: Need at least 2 healthy instances in ASG, have 0. Most recent activity: {
  ActivityId: "0c858809-d3a8-c8d7-a931-c43cbe369e02",
  AutoScalingGroupName: "asg-ecs-......",
  Cause: "At 2018-03-07T21:53:58Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 0 to 2.",
  Description: "Launching a new EC2 instance.  Status Reason: The device 'xvdcz' is used in more than one block-device mapping. Launching EC2 instance failed.",
  Details: "{\"Subnet ID\":\"subnet-1232342\",\"Availability Zone\":\"eu-central-1a\"}",
  EndTime: 2018-03-07 21:53:59 +0000 UTC,
  Progress: 100,
  StartTime: 2018-03-07 21:53:59.346 +0000 UTC,
  StatusCode: "Failed",
  StatusMessage: "The device 'xvdcz' is used in more than one block-device mapping. Launching EC2 instance failed."

[BUG] broken docker configuration on amzn2 ECS image

The user_data.tpl template appends the following line to /etc/sysconfig/docker:

echo 'OPTIONS="$${OPTIONS} --storage-opt dm.basesize=${docker_storage_size}G"' >> /etc/sysconfig/docker

The amzn2 ECS images use a proper systemd unit to start docker instead of a sysvinit script.

The /etc/sysconfig/docker file is referenced for environment variables but the file isn't sourced, as in the way the sysvinit scripts work, so that appended line doesn't result in the OPTIONS envvar being built up, but rather being set to that. This then causes the docker service to fail on reboot of the instance or restart of the service with the error "dockerd accepts no arguments".

This is not noticed normally because the next line in the template /etc/init.d/docker restart is invalid in the amzn2 systemd instances so it doesn't actually restart docker and encounter this broken config .... you only get caught out by it on a service restart or a reboot.

In our use of this module we're now doing a sed -i '/OPTIONS="${OPTIONS}/d' /etc/sysconfig/docker to avoid this particular problem.

Allow enabling metrics and output roles

Hello 👋

I opened a couple of PRs a few months ago, wondering if they're okay to get merged in or if there's outstanding work required for them to be accepted

Could be my bad for not opening a related issue.

The PRs are:

#39
#38

Thanks,
Chris

How do I apply a map of tags to extra_tags?

I do this to apply tags to aws resources:

		  tags = "${merge(
			local.common_tags,
			map(
			  "Name", "awesome-app-server",
			  "Role", "server"
			)
		  )}"

But "extra_tags" has this weird format. How can I pass through my map of tags?

The module creates a default security group that allows blanket access

The module creates a security group allowing all traffic from everywhere. Not sure that's a safe default.

There should at least be a warning about this behavior (I don't think its enough to just mention this in the allowed_cidr_blocks bullet point.

When a cluster is connected to a non-default VPC, its instances won't have internet access by default.

Cluster will show up as not having any instances. Is there a way to display a warning to a user stating that this may happen? Perhaps a dependency via a data.aws_internet_gateway dependency (this currently does not appear to be exposed via terraform)?

Consistent resource naming convetions

The iam.tf file contains two different resource naming conventions:

The ecs_profile resource uses underscores while all other resource names use hyphens (e.g. ecs-role, ecs-policy, etc...).

Can a consistent convention be used throughout? It appears that underscores _ are more common. Happy to submit a pull request if that's the desired approach.

Cheers,
Mike

tf_aws_ecs/iam.tf

Lines 1 to 7 in f7af3a0

	resource "aws_iam_instance_profile" "ecs_profile" {
	name_prefix = "${replace(format("%.102s", replace("tf-ECSProfile-${var.name}-", "_", "-")), "/\\s/", "-")}"
	role = "${aws_iam_role.ecs-role.name}"
	path = "${var.iam_path}"
	}
	resource "aws_iam_role" "ecs-role" {

Release tag

Hi 👋

Could we get a release tag added due to the recently merged PR's?

Thanks 😬

Migrate code to the terraform-aws-modules/terraform-aws-ecs and publish in the registry

Hi @tfhartmann,

Could you please migrate this module to https://github.com/terraform-aws-modules/terraform-aws-ecs ?

Send pull-requests, and we discuss the process as we go. This module is a good candidate to be very feature-rich and powerful.

Thanks,
Anton.

compatibility issues with aws provider v2

experiencing the following error with terraform-provider-aws:v2.1.0:

Error: module.ecs.data.aws_ami.ecs_ami: "owners": required field is not set