cloud66-oss / habitus Goto Github PK

View Code? Open in Web Editor NEW

1.3K 35.0 91.0 15.86 MB

A build flow tool for Docker.

Home Page: https://help.cloud66.com/habitus/

License: Apache License 2.0

Go 94.81% Shell 4.40% Dockerfile 0.79%

ci-cd containers docker dockerfile kubernetes

habitus's People

Contributors

Stargazers

Watchers

Forkers

foldingbeauty gooops devopsbox th3architect linearregression killerdesigner adammendoza hungld smartplanetinternational poeblu thoseharvester ngitzzpolice sudozz ggthedev neonecosystem binarytemple-external nagyistge gitqueue sagardeyrakesh abukauser pmartinot jsxtech rastabrane puppycodes sentinelleader ankitchheda wagoodman johnulist verchol linuxentrepreneur sarkartanzil alexxnica kryndex stethd jebcat1982 etsangsplk harleybabe72 fercho-ayala linuxdoom dwtcourses shasthojoy theaggregator notr00t x5u jangocheng qopqopqop mumoshu cw-sakamoto sdscgithub hickey franklinharry pbuditi paquedev awesomegolang x86nick osharav jonathonwalz pinakimsleepiq dockercore eddie-he mysticaltech kidmam l1ahim jayd2446 weilaizhou zhuima ninefive poooooooooke bpowers4 ufo2011 apollusehs-oss megatronga hiteshmah-jan dedsec-9 suryatmodulus positive-spaces-io ossgroupp luis-sousa-pinto thedigitaloctopus iq-scm shakahl

habitus's Issues

travisci command error

So I wanted to test habitus and made this project. When travisci builds it, this is the output:

$ sudo wget https://github.com/cloud66/habitus/releases/download/0.4.9/habitus_linux_amd64 -o habitus

$ sudo chmod +x habitus

$ sudo ./habitus -version
./habitus: 1: ./habitus: --2017-02-19: not found
./habitus: 2: ./habitus: Syntax error: "(" unexpected


The command "sudo ./habitus -version" failed and exited with 2 during .

Your build has been stopped.

Dead link on homepage

When I visit "http://www.habitus.io"
And I click on "Instructions for OSX"
Then I receive a "HTTP/1.1 404 Not Found"

CMD / ENTRYPOINT are not repected in the exec format

Hello

It seems that if I use CMD ["/my_binary"] in my Docker.deployment it is considered as if i typed: CMD /my_binary

This is what i expect when running docker inspect:

            "Cmd": [
                "/main"
            ],

and this is what i get using habitus:

            "Cmd": [
                "/bin/sh",
                "-c",
                "/main"
            ],

If i use docker build with the exact same dockerfile it works as expected.

I did an example to reproduce the issue:
https://github.com/sebest-blog/create-a-small-docker-image-for-a-golang-binary-with-habitus

tar: Option --xattrs is not supported on OS X el Capitan

When I try building an image on an OS X El Capitan host, the following error is thrown once the squash step is reached:

2016/09/01 23:52:01 ▶ Loading export from /tmp/habitus-export-527359243 using /tmp/docker-squash612669813 for tempdir 
2016/09/01 23:52:02 ▶ Loaded image w/ 18 layers 
2016/09/01 23:52:02 ▶ Extracting layers... 
2016/09/01 23:52:02 ▶   -  /tmp/docker-squash612669813/036fc864e6c7d0b6f75fe523e4a1a852f72115062e76f0a9d14877989948eb4a/layer.tar 
tar: Option --xattrs is not supported
Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help

2016/09/01 23:52:02 ▶ Build for step carrier failed due to exit status 1

I am fairly sure this line is causing the problem however I am not knowledgeable enough in this area to actually resolve the problem.

Related issue on docker-squash

Dirty work-around, I have tested and this does work however it feels like there should be a neater solution to this problem

Add default values for most common invocations

I've noticed that I'm running habitus pretty regularly like so:
sudo habitus --host=unix:///var/run/docker.sock --binding=127.0.0.1

So here are my thoughts:

It would be nice if --host defaulted to the the most common value unix:///var/run/docker.sock if the DOCKER_HOST env variable is not present. If there is no socket file or ENV variable, fallback to today's behavior.
Similarly, it would make sense for the binding value to default to a (safe) local value of 127.0.0.1 by default. I could see an argument to use 0.0.0.0 (which would not be preferred for security reasons) or discover the docker bridge IP dynamically.

This would at least get the most common invocations down to just sudo habitus.

"cleanup" commands cannot run if USER was set to != root in Dockerfile

In our Dockerfile, we have some steps that we do execute using another user than root, resulting in the cleanup command rm -rf /root/.ssh/ not to execute:

rm: cannot remove '/root/.ssh/': Permission denied

Easy fix was to have command USER root as the last entry of our Dockerfile, but that's a temporary hack 😄

"no-cache" flag documentation is inverted

The documentation for the no-cache flag (https://github.com/cloud66/habitus/blob/master/main.go#L43) describes it as enabled caching, not disabling it

open cert.pem: no such file or directory

When I run habitus, I get the following error

$ sudo habitus_linux_amd64
2016/10/24 12:16:43 ▶ Using '/home/wael/myProject/build.yml' as build file
2016/10/24 12:16:43 ▶ open cert.pem: no such file or directory

Environment:

OS: Ubuntu 16.04
Docker: 1.12.1, build 23cf638

Docker is installed locally and does not require TLS for connection

Install Instructions

I might be missing it, but have been looking at docs for a while and still unable to find any instructions on how to install for linux. Maybe docs could take the following format:

Step 1: Install
Type this command to install blah blah blah

Step 2: Use
...

Fill out README.md

Currently just has a title. Needs basic description, instructions on running, caveats (if any), etc.

Habitus usage with CI (without admin privileges/sudo)

Attempting to run without sudo yields:

Some of the build steps require admin privileges (sudo). Please run with sudo

The reason I'm trying to use habitus is for a continuous integration process that needs a private key (for github access) within the docker build. The automated CI process does not have sudo on the host, nor do I want to grant such privileges.

I am new to habitus so I could be missing something. Can I accomplish this without sudo while using habitus?

If other images exist locally, incorrect images can be deleted in the final step

In the case below: local/central-b3589d5d7afb365fe9978c9f0f6741d8a1baaaa4:latest was incorrectly identified to be removed (even though it was the last step)

 ---> e02789b39a5d
Removing intermediate container d0075de08c28
Successfully built e02789b39a5d
2017/03/22 12:43:24 ▶ Removing artifact path: /var/lib/buildkite-agent/builds/buildkite-1/cloud-66-1/continuous-build/app-artifact.tar.gz
2017/03/22 12:43:24 ▶ Removing unwanted image local/central-b3589d5d7afb365fe9978c9f0f6741d8a1baaaa4:latest 
2017/03/22 12:43:24 ▶ Removing unwanted image base-b3589d5d7afb365fe9978c9f0f6741d8a1baaaa4

ENV instruction is incorrectly modified

ENV MAVEN_HOME=/opt/maven PATH=$MAVEN_HOME/bin/:$PATH

gets converted into -
env MAVEN_HOME /opt/maven PATH $MAVEN_HOME/bin/:$PATH

should have been -
env MAVEN_HOME=/opt/maven PATH=$MAVEN_HOME/bin/:$PATH

Reference: https://docs.docker.com/engine/reference/builder/#/env

Upstream image not found

Hello,

I have a set of images that depend on an image "go_base".
I am affected by something similar to #34 reported by @kamerblauwlicht

I am running 0.4.7 and I still see this happening.

2016/10/19 12:58:53 Parallel build for go_base
2016/10/19 12:58:53 Building go_base
2016/10/19 12:58:53 Parsing and converting 'Dockerfile.go_base'
2016/10/19 12:58:53 Building the go_base-66b52139 image from Dockerfile.go_base.generated

[cut]

2016/10/19 12:59:07  Writing the new Dockerfile into Dockerfile.message_store.generated 
2016/10/19 12:59:07 Building the go_message_store-66b52139 image from Dockerfile.message_store.generated
Step 1 : FROM go_base
Step 1 : FROM go_base
Pulling repository docker.io/library/go_base
Pulling repository docker.io/library/go_base
2016/10/19 12:59:12  Build for step go_message_store failed due to Error: image library/go_base:latest not found
Build step 'Execute shell' marked build as failure

this is intermittent and re-running the job usually fixes it.

Container may not have `stat`

Currently, the stat command is used before extracting files from the container (see: https://github.com/cloud66/habitus/blob/master/build/builder.go#L340). Certain containers (such asscratch`) will not contain this method and will fail to extract the file. In those cases it may make sense to fallback to not copying over the file permissions.

parse Dockerfiles for _env(x) to get substituted with parent environment variables or with the --env switch

Run single step from build file

(using version 0.4.10)

Is this possible? eg using the build.yml file from the docs as an example:

build:
  version: 2016-03-14 # version of the build schema.
  steps:
    builder:
      name: builder
      dockerfile: Dockerfile.builder
      # snip
    deployment:
      name: ironmountain
      dockerfile: Dockerfile.deployment
      depends_on:
        - builder
    uploader:
      name: uploader
      dockerfile: Dockerfile.uploader
      depends_on:
        - ironmountain
      # snip

I don't necessarily want to run the uploader every time I want to run habitus; most of the time I'm interested in only the builder step. Is there a way to specify that step only?

`_env(VAR)` directive not properly evaluated

hello, I have a simple setup like the following

build:
  version: 2016-03-14 # version of the build schema.
  steps:
    builder:
      name: _env(IMAGE_PREFIX)/go_base:_env(GIT_LABEL)
      dockerfile: Dockerfile.go_base
      artifacts:
        - /comp3
        - /comp2
        - /comp1

When I run it with

habitus -f build2.yml -env GIT_LABEL=label -env IMAGE_PREFIX=prefix

I get the following error:

2016/10/11 16:41:58 ▶ Using 'build2.yml' as build file 
2016/10/11 16:41:58 ▶ Building 1 steps 
2016/10/11 16:41:58 ▶ Step 0 - builder:  
2016/10/11 16:41:58 ▶ Parallel build for  
2016/10/11 16:41:58 ▶ Building  
2016/10/11 16:41:58 ▶ Parsing and converting 'Dockerfile.go_base' 
2016/10/11 16:41:58 ▶ Starting API on 8080 
2016/10/11 16:41:58 ▶ Writing the new Dockerfile into Dockerfile.go_base.generated 
2016/10/11 16:41:58 ▶ Building the  image from Dockerfile.go_base.generated 
sha256:ec15bedfbd0cb2c189099c16a8d92c0aeac1caedd18724567451df61a17a1413
2016/10/11 16:42:00 ▶ Building container based on the image 
2016/10/11 16:42:00 ▶ Build for step  failed due to API error (500): {"message":"Invalid container name (.8tf0SKYpExrgdLvd), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed"}

so it looks as if the _env(VAR) directive in this case is not evaluated

ENTRYPOINT + CMD not working

hello, I have a simple setup like the following

build:
  version: 2016-03-14 # version of the build schema.
  steps:
    builder:
      name: b.gcr.io/test/go_base:_env(GIT_LABEL)
      dockerfile: Dockerfile.go_base
      artifacts:
        - /comp3
        - /comp2
        - /comp1
    comp3:
      name: b.gcr.io/test/comp3:_env(GIT_LABEL)
      dockerfile: Dockerfile.comp3
      depends_on:
        - builder
    comp2:
      name: b.gcr.io/test/comp2:_env(GIT_LABEL)
      dockerfile: Dockerfile.comp2
      depends_on:
        - builder
    comp1:
      name: b.gcr.io/test/comp1:_env(GIT_LABEL)
      dockerfile: Dockerfile.comp1
      depends_on:
        - builder

with a sample Dockerfile.comp1 as simple as:

FROM ubuntu
ADD /artifacts/comp1 /comp1
ENTRYPOINT ["/comp1"]
CMD ["--config", "/configs.yml"]

When I run habitus, I get the following error

2016/10/11 16:36:41 ▶ Build for step b.gcr.io/test/comp1:fix_test-ademaria failed due to Unknown flag: config

open cert.pem: no such file or directory

When I try and run habitus (Linux build, ubuntu machine) I get the following error:

> habitus -f habitus.yaml --host tcp://docker:2345
2016/05/07 14:46:20 ▶ Using 'habitus.yaml' as build file                                 
2016/05/07 14:46:20 ▶ open cert.pem: no such file or directory

My Docker doesn't require TLS to connect to. It looks like builder.go (https://github.com/cloud66/habitus/blob/f133be298d74c444bb260684e7b4f6989f4fdb18/build/builder.go#L59-L63) assumes it does.

default setting of host is not working

-host string must set to unix:///var/run/docker.sock instead unix://var/run/docker.sock

cannot use habitus CLI environment variables with secrets

As soon as I use --env variables I can no longer access secret files. When I remove the environment variables it works.

Random crash when run: "habitus -version" on mac 10.12

 qiuyuzhou  ⋯  git.shundaojia.com  servers  carpoolsch  habitus -version
Habitus - v0.4.6 (c) 2016 Cloud 66 Inc.
 qiuyuzhou  ⋯  git.shundaojia.com  servers  carpoolsch  habitus -version
Habitus - v0.4.6 (c) 2016 Cloud 66 Inc.
 qiuyuzhou  ⋯  git.shundaojia.com  servers  carpoolsch  habitus -version
Habitus - v0.4.6 (c) 2016 Cloud 66 Inc.
 qiuyuzhou  ⋯  git.shundaojia.com  servers  carpoolsch  habitus -version
failed MSpanList_Insert 0x92d000 0xb0892f31fb6 0x0 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.throw(0x5e1d10, 0x10)
    /Users/khash/go/src/runtime/panic.go:530 +0x90 fp=0x7fff5fbff0c0 sp=0x7fff5fbff0a8
runtime.(*mSpanList).insert(0x82d288, 0x92d000)
    /Users/khash/go/src/runtime/mheap.go:933 +0x293 fp=0x7fff5fbff0f0 sp=0x7fff5fbff0c0
runtime.(*mheap).freeSpanLocked(0x82ca80, 0x92d000, 0x100, 0x0)
    /Users/khash/go/src/runtime/mheap.go:809 +0x4be fp=0x7fff5fbff158 sp=0x7fff5fbff0f0
runtime.(*mheap).grow(0x82ca80, 0x8, 0x0)
    /Users/khash/go/src/runtime/mheap.go:675 +0x2a0 fp=0x7fff5fbff1b0 sp=0x7fff5fbff158
runtime.(*mheap).allocSpanLocked(0x82ca80, 0x1, 0x0)
    /Users/khash/go/src/runtime/mheap.go:553 +0x4e3 fp=0x7fff5fbff208 sp=0x7fff5fbff1b0
runtime.(*mheap).alloc_m(0x82ca80, 0x1, 0x15, 0x0)
    /Users/khash/go/src/runtime/mheap.go:437 +0x119 fp=0x7fff5fbff238 sp=0x7fff5fbff208
runtime.(*mheap).alloc.func1()
    /Users/khash/go/src/runtime/mheap.go:502 +0x41 fp=0x7fff5fbff268 sp=0x7fff5fbff238
runtime.systemstack(0x7fff5fbff288)
    /Users/khash/go/src/runtime/asm_amd64.s:307 +0xab fp=0x7fff5fbff270 sp=0x7fff5fbff268
runtime.(*mheap).alloc(0x82ca80, 0x1, 0x10000000015, 0x12a6f)
    /Users/khash/go/src/runtime/mheap.go:503 +0x63 fp=0x7fff5fbff2b8 sp=0x7fff5fbff270
runtime.(*mcentral).grow(0x82e680, 0x0)
    /Users/khash/go/src/runtime/mcentral.go:209 +0x93 fp=0x7fff5fbff320 sp=0x7fff5fbff2b8
runtime.(*mcentral).cacheSpan(0x82e680, 0x826fa8)
    /Users/khash/go/src/runtime/mcentral.go:89 +0x47d fp=0x7fff5fbff360 sp=0x7fff5fbff320
runtime.(*mcache).refill(0x929000, 0x15, 0x7fff5fbff3c8)
    /Users/khash/go/src/runtime/mcache.go:119 +0xcc fp=0x7fff5fbff398 sp=0x7fff5fbff360
runtime.mallocgc.func2()
    /Users/khash/go/src/runtime/malloc.go:642 +0x2b fp=0x7fff5fbff3b8 sp=0x7fff5fbff398
runtime.systemstack(0x7fff5fbff458)
    /Users/khash/go/src/runtime/asm_amd64.s:307 +0xab fp=0x7fff5fbff3c0 sp=0x7fff5fbff3b8
runtime.mallocgc(0x180, 0x57bba0, 0x0, 0x800000000)
    /Users/khash/go/src/runtime/malloc.go:643 +0x869 fp=0x7fff5fbff498 sp=0x7fff5fbff3c0
runtime.newobject(0x57bba0, 0x827790)
    /Users/khash/go/src/runtime/malloc.go:781 +0x42 fp=0x7fff5fbff4c0 sp=0x7fff5fbff498
runtime.malg(0x8000, 0x827b40)
    /Users/khash/go/src/runtime/proc.go:2634 +0x27 fp=0x7fff5fbff4f8 sp=0x7fff5fbff4c0
runtime.mpreinit(0x828300)
    /Users/khash/go/src/runtime/os1_darwin.go:140 +0x1f fp=0x7fff5fbff510 sp=0x7fff5fbff4f8
runtime.mcommoninit(0x828300)
    /Users/khash/go/src/runtime/proc.go:494 +0x105 fp=0x7fff5fbff558 sp=0x7fff5fbff510
runtime.schedinit()
    /Users/khash/go/src/runtime/proc.go:434 +0x79 fp=0x7fff5fbff5a0 sp=0x7fff5fbff558
runtime.rt0_go(0x7fff5fbff5d8, 0x2, 0x7fff5fbff5d8, 0x0, 0x0, 0x2, 0x7fff5fbff7a8, 0x7fff5fbff7b0, 0x0, 0x7fff5fbff7b9, ...)
    /Users/khash/go/src/runtime/asm_amd64.s:138 +0x132 fp=0x7fff5fbff5a8 sp=0x7fff5fbff5a0

Update artifacts copyToHost() to support directories

It would be nice to declare a directory of artifacts (such as a ruby bundle directory) instead of listing all files explicitly... for instance:

build:
  version: 2016-03-14
  steps:
    builder:
      name: myapp/builder
      dockerfile: Dockerfile
      artifacts:
        - /bundler
....

where /bundler contains sub directories:

bin  build_info  bundler  cache  doc  extensions  gems	specifications

and those sub directories have files... etc.

How to install Habitus on mac

could you please give the steps to install and run habitus on mac os?

Evaluate command line flags vs config in yaml

There are some command line flags (such as no-cache) which may make more sense to be on each job in the yml, allowing for greater customization

Add ability to define a volume on intermediate containers

This would be hugely helpful in the case of java maven builds.

I'm familiar with the trick of copying over just the pom.xml (not forgetting your submodules) and running mvn dependency:resolve (demonstrated nicely here). Changes to pom.xml files are common enough occurrences, however, and in my case a change to a pom incurs a ~200mb download. Since my CI is hosted in the cloud that's a lot of network usage, which can get $$$.

If the dependencies were instead exposed in a volume, they would be downloaded only once for the lifetime of the project.

[docs] Updated github/private key working example for OSX

I spent some time figuring this out so I hope it can help others or be added to examples.

Dockerfile

FROM ubuntu
RUN apt-get update && apt-get install -y wget openssh-client

# add the authorized host key for github (avoids "Host key verification failed")
RUN mkdir ~/.ssh && ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts

ARG host
ENV PRIVATE_KEY /root/.ssh/id_rsa
RUN wget -O $PRIVATE_KEY http://$host:8080/v1/secrets/file/id_rsa \
&& chmod 0600 $PRIVATE_KEY \
&& ssh -T [email protected] \
&& rm $PRIVATE_KEY

build.yml

build:
version: 2016-03-14
steps:
  builder:
    name: builder
    dockerfile: Dockerfile
    secrets:
      id_rsa:
        type: file
        value: _env(HOME)/.ssh/id_rsa

Run it (OSX command line)

habitus --build host=$(ipconfig getifaddr en0) --host unix:///var/run/docker.sock --binding=0.0.0.0

running command with habitus fails due to Docker API changes.

update go-dockerclient
fsouza/go-dockerclient#537

Keep flag?

The description for the --keep-all option seems to suggest the existence of a keep flag for individual steps that would prevent image removal for intermediate steps:

	flag.BoolVar(&config.KeepSteps, "keep-all", false, "Overrides the keep flag for all steps. Used for debugging")

Does such a flag actually exist?

How to use Habitus with Docker for Mac?

I just have switched to Docker for Mac and I try to figure out how to use Habitus. So far without any luck.

➜  habitus                                     
2016/05/22 20:34:43 ▶ Using '/path/to/build.yml' as build file                                                                                               
2016/05/22 20:34:43 ▶ open cert.pem: no such file or directory

➜  habitus --use-tls=false
2016/05/22 20:34:50 ▶ Using '/path/to/build.yml' as build file                                                                                               
2016/05/22 20:34:50 ▶ invalid endpoint

ENTRYPOINT instruction is modified from exec format to shell format

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["ls"]

gets modified to

entrypoint /docker-entrypoint.sh
CMD ["ls"]

Is that an expected behaviour?

Suggestion, prefix output logs with step name

The log when running multiple images in parallel seems confusing and difficult to follow.
When it gets to the actual docker steps, it becomes impossible to understand which command relates to which step.

2016/10/19 12:58:53 Parallel build for go_base
2016/10/19 12:58:53 Parallel build for frontend_api
2016/10/19 12:58:53 Building go_base
2016/10/19 12:58:53 Building frontend_api
2016/10/19 12:58:53 Parsing and converting 'Dockerfile.go_base'
2016/10/19 12:58:53 Parsing and converting 'Dockerfile.frontend_api'
2016/10/19 12:58:53 Writing the new Dockerfile into Dockerfile.go_base.generated
2016/10/19 12:58:53 Parallel build for hdl7 �[0m
2016/10/19 12:58:53 Building hdl7 �[0m
2016/10/19 12:58:53 Parsing and converting 'Dockerfile.hdl7

would it be possible to tag these lines with something like the step name or number so it is easies to understand which step they refer to?

regards
Alessandro

Need example yml file

Currently hard to know how to quickly get everything configured

Problem with artifacts

Hello,

I'm using Habitus with this yml file

build:  
    version: 2016-03-14
    steps:
        builder:
            name: builder
            dockerfile: Dockerfile.builder
             - /home/MainUser/test.tar.gz
        deploy:
            name: deploying
            dockerfile: Dockerfile.deploy

When I run it, I get following error:

2017/04/20 15:08:05 notifying bugsnag: regexp: Compile(`^C:\Users\Pawel\Documents\Docker\dockerKinetis/+`): error parsing regexp: invalid escape sequence: `\U`
2017/04/20 15:08:05 ? Collecting artifact information
panic: regexp: Compile(`^C:\Users\Pawel\Documents\Docker\dockerKinetis/+`): error parsing regexp: invalid escape sequence: `\U` [recovered]
        panic: regexp: Compile(`^C:\Users\Pawel\Documents\Docker\dockerKinetis/+`): error parsing regexp: invalid escape sequence: `\U`

goroutine 1 [running]:
panic(0x7dcae0, 0xc0420f7720)
        /usr/local/go/src/runtime/panic.go:500 +0x1af
github.com/cloud66/habitus/vendor/github.com/bugsnag/bugsnag-go.AutoNotify(0xc0420f7740, 0x1, 0x1)
        /usr/local/go/src/github.com/cloud66/habitus/vendor/github.com/bugsnag/bugsnag-go/bugsnag.go:52 +0x13a
panic(0x7dcae0, 0xc0420f7720)
        /usr/local/go/src/runtime/panic.go:458 +0x251
regexp.MustCompile(0xc0420f3440, 0x30, 0xc042093030)
        /usr/local/go/src/regexp/regexp.go:237 +0x181
github.com/cloud66/habitus/build.(*Builder).collectHostArtifactRoots(0xc042072780, 0xc0420f76b0, 0x1, 0x1)
        /usr/local/go/src/github.com/cloud66/habitus/build/builder.go:184 +0x30d
github.com/cloud66/habitus/build.(*Builder).StartBuild(0xc042072780, 0xc042086d80, 0xc042072780)
        /usr/local/go/src/github.com/cloud66/habitus/build/builder.go:109 +0xd49
main.main()
        /usr/local/go/src/github.com/cloud66/habitus/main.go:149 +0xe66
2017/04/20 15:08:05 notifying bugsnag: regexp: Compile(`^C:\Users\Pawel\Documents\Docker\dockerKinetis/+`): error parsing regexp: invalid escape sequence: `\U`
[recovered]

I don't know what I am doing wrong. Without artifacts everything is OK.

Random image not found errors when running Habitus in parallel

I get random errors about images missing when builds run in parallel (with the uid flag set)

Is this project dead?

Am I correct to assume that this project is currently not under development?

Rename artefacts

artefacts is the British spelling and appears quite confusing to Americans; while both are correct, it might be simpler to use the accepted American style simply due to the other projects doing so.

During builds that use habitus we are getting a frequent but intermittent issue.

sometimes the build is done but the image is not proper tagged.

Cannot get ssh keys

Trying to run habitus even with the provided example, I get:

security git:(master) ✗ habitus_linux_amd64 --host=unix:///var/run/docker.sock --binding=127.0.0.1 --build host=127.0.0.1
2016/10/24 16:14:55 ▶ Using '/home/wael/workspace/habitus/examples/security/build.yml' as build file
2016/10/24 16:14:55 ▶ Collecting artifact information
2016/10/24 16:14:55 ▶ Building 1 steps
2016/10/24 16:14:55 ▶ Step 0 - builder: builder
2016/10/24 16:14:55 ▶ Parallel build for builder
2016/10/24 16:14:55 ▶ Building builder
2016/10/24 16:14:55 ▶ Parsing and converting 'Dockerfile'
2016/10/24 16:14:55 ▶ Starting API on 8080
2016/10/24 16:14:55 ▶ Writing the new Dockerfile into Dockerfile.generated
2016/10/24 16:14:55 ▶ Building the builder image from Dockerfile.generated
Step 1 : FROM ubuntu
---> f753707788c5
Step 2 : RUN apt-get update && apt-get install -y wget openssh-client
---> Using cache
---> f149286ec705
Step 3 : RUN mkdir ~/.ssh && ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
---> Using cache
---> d093edca0423
Step 4 : ARG host
---> Using cache
---> e744a38e08e2
Step 5 : ENV PRIVATE_KEY /root/.ssh/id_rsa
---> Using cache
---> 1a1212d088d7
Step 6 : RUN wget -O $PRIVATE_KEY http://$host:8080/v1/secrets/file/id_rsa && chmod 0600 $PRIVATE_KEY && ssh -T [email protected] && rm $PRIVATE_KEY
---> Running in 0bd430f3b4ef
--2016-10-24 14:14:55-- http://127.0.0.1:8080/v1/secrets/file/id_rsa
Connecting to 127.0.0.1:8080... failed: Connection refused.
2016/10/24 16:14:56 ▶ Build for step builder failed due to The command '/bin/sh -c wget -O $PRIVATE_KEY http://$host:8080/v1/secrets/file/id_rsa && chmod 0600 $PRIVATE_KEY && ssh -T [email protected] && rm $PRIVATE_KEY' returned a non-zero code: 4

Command
$ habitus_linux_amd64 --host=unix:///var/run/docker.sock --binding=127.0.0.1 --build host=127.0.0.1

Environment:

Ubuntu 16.04 and Docker 1.12.1

environment variables in parent process are not set in build

when you run a build, the environment variables of a parent process are not inherit.

@lvangool

Meta: How should we file issues and feature requests?

Hey guys!

Do you want us to file issues and feature requests as proposal issues or just send PRs or something else?

Thanks!
Joey

Execute command as part of 'after build' for example docker push

Is there a way to do 'docker push' for example after building the image.
I have a custom script that does 'docker build' and would like to push the image to a private repository for example. I handle this via a shell script. Habitus being a build tool, would be nice to have a section that lists commands that executes outside of the container after each build step so I can get rid of the shell script.

OSX build hangs when getting to 8080

I'm trying to use the secrets feature however my build is hanging when connecting to port 8080

Command ./habitus_darwin_amd64 -f habitus.yaml --build host=192.168.99.1

Dockerfile:

FROM alpine:3.2

ARG host
RUN TOKEN=$(wget http://$host:8080/api/v1/secrets/id_rsa) && echo 'ENV:' && env && echo 'TOKEN ENV' && echo $TOKEN && unset $TOKEN
RUN wget -O ~/.ssh/id_rsa http://$host:8080/api/v1/secrets/id_rsa && ssh -T [email protected] && rm ~/.ssh/id_rsa

Build YAML

build:
  version: 2016-03-14
  steps:
    builder:
      name: builder
      dockerfile: Dockerfile-habitus
      secrets:
        id_rsa:
          type: file
          value: _env(HOME)/.ssh/id_rsa

Output:

tep 1 : FROM alpine:3.2
 ---> 558af09712a4
Step 2 : ARG host
 ---> Using cache
 ---> a20b789279a3
Step 3 : RUN TOKEN=$(wget http://$host:8080/api/v1/secrets/id_rsa) && echo 'ENV:' && env && echo 'TOKEN ENV' && echo $TOKEN && unset $TOKEN
 ---> Running in 43d34c9d7d1c
Connecting to 192.168.99.1:8080 (192.168.99.1:8080)

In a separate terminal window, if I try and access the resource then I get a 404.

curl 192.168.99.1:8080/api/v1/secrets/id_rsa
{"Error":"Resource not found"}%

Any ideas?

build error on log

./main.go:72: cannot use *log (type "github.com/op/go-logging".Logger) as type "github.com/cloud66/habitus/vendor/github.com/op/go-logging".Logger in assignment

From:
go get
then
go build

Relative build.yml and Dockerfile paths not generating the Dockerfile.generated at the right location

Having the following (simplified) build.yml:

build:
  version: 2016-03-14
  steps:
    builder:
      name: builder
      dockerfile: builder/Dockerfile

And when using the following command:
sudo habitus --host=unix:///var/run/docker.sock --binding=127.0.0.1 --keep-all --no-cache -f builder/build.yml (and probably a simpler version of it as well, didn't try out)

the following occurs:

2017/01/06 17:00:49 ▶ Using 'builder/build.yml' as build file 
2017/01/06 17:00:49 ▶ Collecting artifact information 
2017/01/06 17:00:49 ▶ Building 2 steps 
2017/01/06 17:00:49 ▶ Starting API on 8080 
2017/01/06 17:00:49 ▶ Step 0 - builder: builder 
2017/01/06 17:00:49 ▶ Step 1 - app-base: app-base 
2017/01/06 17:00:49 ▶ Parallel build for builder 
2017/01/06 17:00:49 ▶ Building builder 
2017/01/06 17:00:49 ▶ Parsing and converting 'builder/Dockerfile' 
2017/01/06 17:00:49 ▶ Writing the new Dockerfile into builder/Dockerfile.generated 
2017/01/06 17:00:49 ▶ Building the builder image from Dockerfile.generated 
2017/01/06 17:00:49 ▶ Build for step builder failed due to API error (500): {"message":"Cannot locate specified Dockerfile: Dockerfile.generated"}

We can clearly see in the above log that Habitus is writing to builder/Dockerfile.generated but then tries to read from Dockerfile.generated instead.

Being able to run Habitus from a parent directory using the -f argument would allow to share resources (like configuration files) in a common top-level folder.

cannot get into the cloud66 slack team.

Hello folks. You list your slack team on the git readme but it doesn't let people signup unless they have been approved.

The following URL is meant to allow signup:

https://cloud66ers.slack.com/signup

Unwanted Images

hello, I have a simple setup like the following

build:
  version: 2016-03-14 # version of the build schema.
  steps:
    builder:
      name: b.gcr.io/test/go_base:_env(GIT_LABEL)
      dockerfile: Dockerfile.go_base
      artifacts:
        - /comp3
        - /comp2
        - /comp1
    comp3:
      name: b.gcr.io/test/comp3:_env(GIT_LABEL)
      dockerfile: Dockerfile.comp3
      depends_on:
        - builder
    comp2:
      name: b.gcr.io/test/comp2:_env(GIT_LABEL)
      dockerfile: Dockerfile.comp2
      depends_on:
        - builder
    comp1:
      name: b.gcr.io/test/comp1:_env(GIT_LABEL)
      dockerfile: Dockerfile.comp1
      depends_on:
        - builder

with a sample Dockerfile.comp1 as simple as:

FROM ubuntu
ADD ./comp1 /comp1
CMD ["/comp1", "--config", "/configs.yml"]

When I run habitus, the images are always removed:

2016/10/11 16:33:20 ▶ Removing unwanted image b.gcr.io/test/comp1:fix_test-ademaria 
2016/10/11 16:33:20 ▶ Removing unwanted image b.gcr.io/test/comp2:fix_test-ademaria 
2016/10/11 16:33:20 ▶ Removing unwanted image b.gcr.io/test/comp3:fix_test-ademaria

How can I tell habitus that I want to keep an image?
How do I push it to a repository once built?

Add secret type for environment variables

Our building environment is completely isolated and defining our secrets as env variables on the host is the easiest way for us to manage build time secrets. It would be great to see environment variables implemented as a secret type.

Artifacts may only be a file?

Is there a way to collect a whole directory?

Example, with Elixir releases we get a directory that represents the artifact /rel, passing this to artifacts in the build.yml results in an 'Invalid Header' error.

cloud66-oss / habitus Goto Github PK

habitus's People

Contributors

Stargazers

Watchers

Forkers

habitus's Issues

Recommend Projects

Recommend Topics

Recommend Org