dispatch-local docker image about dispatch-docker

[sfc-gh-pkommini]

I see. I'm clear now as to how to proceed. Thanks a ton!

Once we're able to set it up, I'll post task definitions so that they can help other folks on here who're deploying it on AWS ECS.

[kevgliss]

I agree it's a bit confusing, this repo contains the whole system (app server, scheduler, database, etc.). If you'd just like to know how to build the app itself it would be best to look at the Dockerfile in the main repo here:

https://github.com/Netflix/dispatch/blob/master/docker/Dockerfile

We don't currently host anything on docker hub and expect the images to be rebuilt locally by the user atm.

As far as guidance around how to install without docker. We don't have documentation around this but you should be able to see the steps in the docker file.

Dispatch is a pretty simple app to install, it's just a python web app and a frontend UI, if you already have node installed a `pip install -e ." in the Dispatch main repo will install the app and build the frontend for you.

Happy to answer any other questions that may come up.

[sfc-gh-pkommini]

So I'm trying to create ECS tasks for each of the docker-compose service components. Towards that end, I was wondering what's the purpose of the core docker image? On localhost, when I do docker-compose up -d, I see web and scheduler, I don't however see core.

So will I need a separate ECS task for core as well?

[kevgliss]

The core image is just so I don't have to build the same image twice. Since the scheduler and the web service are the same code base I have them both depend on core (building it once).

I'm not sure if the same logic would apply to ECS as well.

[rilutham]

@sfc-gh-pkommini would you mind sharing the task definitions? Thank you

[sfc-gh-pkommini]

@rilutham Happy to share the task definitions. I'm still getting it setup but I can share what I have so far. I'll try to clear any sensitive material and share it later today.

[rilutham]

@sfc-gh-pkommini really appreciates your help. When setup Dispatch on ECS, what is the minimum env key that should exist to get Dispatch run properly (creating incident > get notified on Slack / create JIRA ticket/ notify stakeholders)

[sfc-gh-pkommini]

[
  {
    "command": [
      "database",
      "upgrade"
    ],
    "environment": ${envs},
    "essential": false,
    "image": "${app_image}",
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/aws/ecs/dispatch-migrator",
        "awslogs-region": "${aws_region}",
        "awslogs-stream-prefix": "ecs"
      }
    },
    "name": "migrator",
    "secrets": ${secrets}
  },
  {
    "command": [
      "dispatch plugins install && dispatch server start dispatch.main:app --host=0.0.0.0"
    ],
    "entryPoint": [
      "sh",
      "-c"
    ],
    "dependsOn": [
      {
        "containerName": "migrator",
        "condition": "SUCCESS"
      }
    ],
    "environment": ${envs},
    "essential": true,
    "image": "${app_image}",
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/aws/ecs/dispatch-web",
        "awslogs-region": "${aws_region}",
        "awslogs-stream-prefix": "ecs"
      }
    },
    "name": "web",
    "portMappings": [
      {
        "containerPort": ${container_port},
        "hostPort": ${container_port},
        "protocol": "tcp"
      }
    ],
    "secrets": ${secrets}
  },
  {
    "command": [
      "scheduler",
      "start"
    ],
    "dependsOn": [
      {
        "containerName": "migrator",
        "condition": "SUCCESS"
      }
    ],
    "environment": ${envs},
    "essential": true,
    "image": "${app_image}",
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/aws/ecs/dispatch-scheduler",
        "awslogs-region": "${aws_region}",
        "awslogs-stream-prefix": "ecs"
      }
    },
    "name": "scheduler",
    "secrets": ${secrets}
  }
]

@rilutham This is the dispatch task definition template dispatch.json.tpl I use with terraform. The ${} are the variables passed into the task definition.

The terraform that creates the task definition is as follows:

# Task definition template file
data "template_file" "dispatch" {
  template = file("./templates/ecs/dispatch.json.tpl")

  vars = {
    app_image      = var.app_image
    host_port      = var.host_port
    container_port = var.container_port
    fargate_cpu    = var.fargate_cpu
    fargate_memory = var.fargate_memory
    aws_region     = var.aws_region
    envs           = data.template_file.envs.rendered
    secrets        = data.template_file.secrets.rendered
  }
}

# Dispatch Web Service
resource "aws_ecs_task_definition" "dispatch" {
  family                   = "dispatch-task"
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = var.fargate_cpu / 2
  memory                   = var.fargate_memory / 2
  container_definitions    = data.template_file.dispatch.rendered
  depends_on               = [module.dispatch_db]

  tags = {
    name = "${var.app_name}-task-definition"
    environment = var.env
  }
}

It's setup such that the migrator runs first and both the scheduler and web containers are dependent on the migrator's 'SUCCESS'.

Hope this helps. Sorry for the delay. :)

[rilutham]

Thank you for showing your task definition @sfc-gh-pkommini . Which docker image that you pushed to ECR? Is it using Dockerfile from https://github.com/Netflix/dispatch/blob/master/docker/Dockerfile , build it in local env and push to ECR? Or you have another way to do this?

[sfc-gh-pkommini]

@rilutham

1. git clone https://github.com/Netflix/dispatch
2. docker build -t dispatch .
3. push to ecr
4. run the ECS service that uses the ECR image

[rilutham]

Okay, I used the same way to build the docker image @sfc-gh-pkommini . Thanks a lot.