paypal / dce-go Goto Github PK
View Code? Open in Web Editor NEWDocker Compose Executor to launch pod of docker containers in Apache Mesos.
License: Apache License 2.0
Docker Compose Executor to launch pod of docker containers in Apache Mesos.
License: Apache License 2.0
Remove unsupported multi-line C-style /* */ comments in makefile.
Currently pod monitor only check if containers are up and running in the pod. Health check should also be part of pod monitor.
When one of the compose file has an empty service for example, even though other compose files have the same service with details, DCE is failing. This is a valid compose file and docker compose will succeed.
#---#
#docker-compose-base.yml
version: '2'
services:
##Application container overrides
test:
<all the details about the service like image etc.,>
#---#
#docker-compose-extends-base.yml
version: '2'
services:
##Application container overrides
test:
When ever DCE is killing a pod because of health check failure, it would be helpful to log the reason for health check failure by inspecting the container.
create file /etc/default/marathon and put MARATHON_DECLINE_OFFER_DURATION=5000
This changes the default from 120 secs and competes aggressively with aurora.
set /sys/fs/cgroup/memory/memory.user_hierarchy value to 1 from default 0
pod monitor only monitor the list of containers printed by docker-compose ps. So pod monitor can't detect the failure if container is removed instead of exit.
Currently pod stops at docker daemon restarts. Handle docker daemon restarts via -d option.
I0801 02:42:11.456279 7200 logging.cpp:194] INFO level logging started!
I0801 02:42:11.457401 7200 logging.cpp:199] Logging to /var/log/mesos
I0801 02:42:11.457464 7200 fetcher.cpp:531] Fetcher Info: {"cache_directory":"/tmp/mesos/fetch/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0","items":[{"action":"BYPASS_CACHE","uri":{"cache":false,"executable":false,"extract":true,"value":"https://github.com/mesos/docker-compose-executor/releases/download/0.1.0/sample-app.tar.gz"}},{"action":"BYPASS_CACHE","uri":{"cache":false,"executable":false,"extract":false,"value":"http://192.168.1.64:8090/config.yaml"}},{"action":"BYPASS_CACHE","uri":{"cache":false,"executable":false,"extract":false,"value":"http://192.168.1.64:8090/general.yaml"}},{"action":"BYPASS_CACHE","uri":{"cache":false,"executable":false,"extract":false,"value":"http://192.168.1.64:8090/executor"}}],"sandbox_directory":"/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6"}
I0801 02:42:11.464521 7200 fetcher.cpp:442] Fetching URI 'https://github.com/mesos/docker-compose-executor/releases/download/0.1.0/sample-app.tar.gz'
I0801 02:42:11.464573 7200 fetcher.cpp:283] Fetching directly into the sandbox directory
I0801 02:42:11.464634 7200 fetcher.cpp:220] Fetching URI 'https://github.com/mesos/docker-compose-executor/releases/download/0.1.0/sample-app.tar.gz'
I0801 02:42:11.464690 7200 fetcher.cpp:163] Downloading resource from 'https://github.com/mesos/docker-compose-executor/releases/download/0.1.0/sample-app.tar.gz' to '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/sample-app.tar.gz'
I0801 02:42:13.928701 7200 process.cpp:1246] libprocess is initialized on 172.16.1.17:40272 with 8 worker threads
I0801 02:42:14.029827 7200 fetcher.cpp:121] Extracted '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/sample-app.tar.gz' into '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6'
I0801 02:42:14.029963 7200 fetcher.cpp:580] Fetched 'https://github.com/mesos/docker-compose-executor/releases/download/0.1.0/sample-app.tar.gz' to '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/sample-app.tar.gz'
I0801 02:42:14.029992 7200 fetcher.cpp:442] Fetching URI 'http://192.168.1.64:8090/config.yaml'
I0801 02:42:14.030009 7200 fetcher.cpp:283] Fetching directly into the sandbox directory
I0801 02:42:14.030037 7200 fetcher.cpp:220] Fetching URI 'http://192.168.1.64:8090/config.yaml'
I0801 02:42:14.030053 7200 fetcher.cpp:163] Downloading resource from 'http://192.168.1.64:8090/config.yaml' to '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/config.yaml'
I0801 02:42:14.031924 7200 fetcher.cpp:580] Fetched 'http://192.168.1.64:8090/config.yaml' to '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/config.yaml'
I0801 02:42:14.031980 7200 fetcher.cpp:442] Fetching URI 'http://192.168.1.64:8090/general.yaml'
I0801 02:42:14.032016 7200 fetcher.cpp:283] Fetching directly into the sandbox directory
I0801 02:42:14.032052 7200 fetcher.cpp:220] Fetching URI 'http://192.168.1.64:8090/general.yaml'
I0801 02:42:14.032075 7200 fetcher.cpp:163] Downloading resource from 'http://192.168.1.64:8090/general.yaml' to '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/general.yaml'
I0801 02:42:14.048689 7200 fetcher.cpp:580] Fetched 'http://192.168.1.64:8090/general.yaml' to '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/general.yaml'
I0801 02:42:14.048729 7200 fetcher.cpp:442] Fetching URI 'http://192.168.1.64:8090/executor'
I0801 02:42:14.048740 7200 fetcher.cpp:283] Fetching directly into the sandbox directory
I0801 02:42:14.048763 7200 fetcher.cpp:220] Fetching URI 'http://192.168.1.64:8090/executor'
I0801 02:42:14.048807 7200 fetcher.cpp:163] Downloading resource from 'http://192.168.1.64:8090/executor' to '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/executor'
I0801 02:42:15.202059 7200 fetcher.cpp:580] Fetched 'http://192.168.1.64:8090/executor' to '/data/mesos/slaves/dc0144b9-1cbe-49a0-b5ee-096d0b2cbce7-S0/frameworks/be770577-dded-4778-83cf-b193c2716136-0000/executors/marathon-docker-compose-demo.04650ad6-7663-11e7-a1df-024262d6c900/runs/290402cb-7668-473b-b485-c49d24edf1f6/executor'
./executor: 1: ./executor: ����������: not found
./executor: 9: ./executor: Syntax error: "(" unexpected
Support verbose mode for docker-compose controlled via a flag:
docker-compose . [-f args...] --verbose
If you don't give any argument when start executor, log.Errorf
does not terminate process
https://github.com/paypal/dce-go/blob/develop/dce/main.go#L374-L383
driver, err := exec.NewMesosExecutorDriver(dConfig)
if err != nil {
log.Errorf("Unable to create a ExecutorDriver : %v\n", err.Error())
}
_, err = driver.Start()
Address "how to use" and "how to develop" documentation gap.
https://github.com/paypal/dce-go/blob/develop/utils/pod/pod.go
Has a lot of global variables. This can lead to possible mutations of the variable contents that could create weird behavior.
Ideally, Pod should be an instantiated struct and all the functions which require access to the values inside of pod.
It would be great to set up CI (e.g. Travis) so that it gets executed every time a PR is created or updated. CI should run linting as well as all unit tests.
Docker always pull images from repository to check if images with same tag get override.
Ignore pulling images failures helps reduce failures if images are cached already.
Background
DCE's current plugin mechanism is pod lifecycle centric, and allows custom extensions to be plugged in around the steps needed to launch a pod (i.e. custom plugins can be added pre/post of image pull, compose up steps)
Requirement
There are situations that demand custom logic executions outside of pod lifecycle, and aligned with the executor's API functions. For example, we have a requirement to execute some custom logic to post error related metrics whenever the the control exits LaunchTask implementation of dce-go.
This may be applicable for other executor API methods too but scope of this issue could deal with LaunchTask and allow extensibility for all API methods.
Proposed Design
Executor Hook
type ExecutorHook interface {
// PostExec is invoked post execution of Docker Compose Mesos Executor's lifecycle function
PostExec(taskInfo *mesos.TaskInfo) error
// BestEffort is invoked in case a PostExec returned an error and are expected to return a bool to indicate
// if the execution needs to continue with the next available hook or not
BestEffort(execPhase string) bool
}
...
execHooks:
LaunchTask:
Post: ["hook1", "hook2"]
...
So, on exit from LaunchTask, most definite thing that is done in the current DCE is to send status to mesos. So to perform the post executions, we can introduce a task status channel and have the hooks executed based on various status changes.
mesos-go updated the directory structure and thus, any library depending on mesos-go must alter the import statements to match the update.
For example,
github.com/mesos/mesos-go/mesosproto
--> github.com/mesos/mesos-go/api/v0/mesosproto
github.com/mesos/mesos-go/executor
--> github.com/mesos/mesos-go/api/v0/executor
So, in compose ports section "portx:porty" is correctly handled and the host port replaced by the dynamic port assigned by mesos. However, if the compose file only has "porty" to specify the container port, the dynamic host port addition does not happen.
Create http.Transport instead of using default one to avoid unnecessary issues in future.
More context in #57
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.