ctripcloud / docker-make Goto Github PK

View Code? Open in Web Editor NEW

97.0 97.0 20.0 70 KB

build,tag,and push a bunch of related docker images via a single command

License: Apache License 2.0

Python 99.39% Dockerfile 0.61%

docker-image dockerfile multitple-docker-images

docker-make's People

Contributors

Stargazers

Watchers

Forkers

jizhilong raj347 acestrong shalcanleo linuxfreakus yuehonghui dids mouadelmerchi jmahowald rapatel0 vairamuthu-shanmugaraj sgdevops mitsiek kubilayeksioglu didstopia ceaphyrel gladiopeace mjsmagic

docker-make's Issues

docker-make throws erroneous WARNINGS in a repository that has no git tags applied

Attempting to run docker-make (latest version on dockerhub) emits errors if the repository being run against has never had any git tags applied to it. Even if the .docker-make.yaml file does not specify use of any tags.

docker-make also seems to be attempting to convert my last commit message to a tag erroneously. Perhaps as a result of this issue.

git describe --tags seems to only be called from:
https://github.com/CtripCloud/docker-make/blob/d392161262c8d8d9e97b9e2ac9013f19e4f851a5/dmake/template_args.py#L90

For info on why git returns this error please see:
https://stackoverflow.com/a/5371322/3902629

$ git describe --tags
fatal: No names found, cannot describe anything.

$ make build
docker run --rm -w /usr/src/app -v ~/.docker:/root/.docker -v /var/run/docker.sock:/var/run/docker.sock -v "/Users/glenn/src/chainpoint/chainpoint-node-src":/usr/src/app jizhilong/docker-make:latest docker-make --no-push
WARNING dmake.template_args(101) 2017-10-08 19:27:59,960 3509d79 Upgrade request and request-promise-native is not a valid docker tag name,will be automatically corrected to 3509d79_Upgrade_request_and_request-promise-native
INFO dmake.template_args(59) 2017-10-08 19:28:00,032 failed to run git describe --tags: Command 'git describe --tags' returned non-zero exit status 128
WARNING dmake.template_args(101) 2017-10-08 19:28:00,080 3509d79 Upgrade request and request-promise-native is not a valid docker tag name,will be automatically corrected to 3509d79_Upgrade_request_and_request-promise-native
INFO dmake.template_args(59) 2017-10-08 19:28:00,153 failed to run git describe --tags: Command 'git describe --tags' returned non-zero exit status 128
INFO dmake.build(147) 2017-10-08 19:28:00,156 chainpoint-node: building
INFO dmake.build(147) 2017-10-08 19:28:02,295 chainpoint-node: attaching labels
INFO dmake.build(147) 2017-10-08 19:28:02,421 chainpoint-node: label added: org.chainpoint.git.commitid="3509d79cebab160eebc97b0ea600876cee1017e8"
INFO dmake.build(147) 2017-10-08 19:28:02,422 chainpoint-node: build succeed: 4ab850a176b0
INFO dmake.build(147) 2017-10-08 19:28:02,460 chainpoint-node: tag added: quay.io/chainpoint/chainpoint-node:3509d79cebab160eebc97b0ea600876cee1017e8
INFO dmake.build(147) 2017-10-08 19:28:02,487 chainpoint-node: tag added: quay.io/chainpoint/chainpoint-node:latest

.docker-make.yml:

builds:
  chainpoint-node:
    context: .
    dockerfile: Dockerfile
    pushes:
      - 'always=quay.io/chainpoint/chainpoint-node:{fcommitid}'
      - 'on_branch:master=quay.io/chainpoint/chainpoint-node:latest'
    labels:
      - 'org.chainpoint.git.commitid={fcommitid}'
    dockerignore:
      - .git

Detecting changes (caching)

Does dmake support any kind of caching/change detection when it comes to building and pushing the images?

I'm currently struggling trying to figure out how to only push images with dmake if they have changed, instead of always pushing, no matter what.

In my case, I've come up with a script for various different linux distros, which detects if new apt packages are available, and if so, it marks the images as dirty by removing them locally, then hands over the build & push process to dmake, which doesn't really work, as dmake pushes everything regardless of changes.

Use of `- 'on_branch:master=foo:{git_branch}'` on a branch with a `/` breaks builds

If you are on a branch with a / in it (as is used in the git flow tool), like feature/foo and you have docker-make configured to use the git_branch tag it fails with an error like:

INFO dmake.build(147) 2017-07-08 17:08:25,196 node-base: tag added: REPONAME:latest
ERROR docker-make(91) 2017-07-08 17:08:25,199 failed to build node-base
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/dmake/cli.py", line 86, in _main
    build.tag()
  File "/usr/local/lib/python2.7/site-packages/dmake/build.py", line 112, in tag
    self.docker.tag(self.final_image, repo, tag_name, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/docker/api/image.py", line 510, in tag
    self._raise_for_status(res)
  File "/usr/local/lib/python2.7/site-packages/docker/api/client.py", line 222, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python2.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
APIError: 500 Server Error: Internal Server Error ("invalid tag format")
make: *** [test-aggregator] Error 1

The build setup for that was:

     pushes:
       - 'on_tag=quay.io/chainpoint/bcoin:{git_tag}'
       - 'on_branch:master=REPONAME:latest'
       - 'on_branch:master=REPONAME:{git_branch}'
       - 'on_branch:master=REPONAME:{fcommitid}'
     labels:
       - 'org.chainpoint.git.commitid={fcommitid}'

Removing this resolved this issue:

       - 'on_branch:master=REPONAME:{git_branch}'

Use of {date} tag would be much more useful if it included hours and minutes

Currently, if you tag with {date} it includes a date of the form 20170701 (YYYYMMDD).

However, if you wanted to use this tag to determine which image to use it only allows you to push maximally once per day, otherwise there will be a conflict between a tag made in the morning, and one made in the afternoon. You would end up moving the tag.

I would suggest that the date tag should instead include the hours and minutes as well so that even multiple tags per day would be unique (seconds are not needed since its unlikely you have a build system fast enough to do multiple builds per minute. With this form the tags would look like 201707011401 for July 1, 2017 at 2:01 PM UTC.

Lexical sorting is retained for this form as well.

Python errors when running docker-make

On macOS Sierra (10.12.4), python 2.7.10 and latest docker-make from pip.

> docker-make --no-push
WARNING dmake.template_args(44) 2017-05-12 17:59:31,850 failed to run %s: %s
WARNING dmake.template_args(44) 2017-05-12 17:59:31,866 failed to run %s: %s
INFO dmake.template_args(44) 2017-05-12 17:59:31,883 failed to run %s: %s
WARNING dmake.build(60) 2017-05-12 17:59:31,883 invalid label template: org.label-schema.vcs-ref={fcommitid}
WARNING dmake.build(60) 2017-05-12 17:59:31,883 invalid label template: org.label-schema.vcs-ref={fcommitid}
WARNING dmake.build(60) 2017-05-12 17:59:31,883 invalid label template: org.label-schema.vcs-ref={fcommitid}
WARNING dmake.build(60) 2017-05-12 17:59:31,883 invalid label template: org.label-schema.vcs-ref={fcommitid}
WARNING dmake.build(60) 2017-05-12 17:59:31,883 invalid label template: org.label-schema.vcs-ref={fcommitid}
INFO dmake.build(145) 2017-05-12 17:59:31,883 alpine-3.5: building
ERROR docker-make(86) 2017-05-12 17:59:32,002 failed to build alpine-3.5
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/dmake/cli.py", line 80, in _main
    build.build()
  File "/Library/Python/2.7/site-packages/dmake/build.py", line 86, in build
    self.non_labeled_image = self._build()
  File "/Library/Python/2.7/site-packages/dmake/build.py", line 186, in _build
    image_id = self._do_build(params)
  File "/Library/Python/2.7/site-packages/dmake/build.py", line 217, in _do_build
    ret = json.loads(line)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 369, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 2 column 1 - line 3 column 1 (char 40 - 122)

dmake.build arguments "--build-args"

It would be nice if we have build arguments added to the yaml file that passes through to the --build-args flag in the docker build command.

When I get a chance I'll try to dig more into the code and understand it to see if I can code it up and submit a pull request.

Please update the docs with elaboration for `on_tag` and `on_branch`

Can you please elaborate what those configs do? Does docker-make somehow keep track of which tags it has already published, and only run on_tag when it "sees a new tag" in the repository? If so, how does the tool track which tags it has seen, and which it has not yet seen?

Typo in README.md

/rservered/reserved/
/spcify/specify/

ctripcloud / docker-make Goto Github PK

docker-make's People

Contributors

Stargazers

Watchers

Forkers

docker-make's Issues

docker-make throws erroneous WARNINGS in a repository that has no git tags applied

Detecting changes (caching)

Use of `- 'on_branch:master=foo:{git_branch}'` on a branch with a `/` breaks builds

Use of {date} tag would be much more useful if it included hours and minutes

Python errors when running docker-make

dmake.build arguments "--build-args"

Please update the docs with elaboration for `on_tag` and `on_branch`

Typo in README.md

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent