qarik-group / concourse-tutorial Goto Github PK
View Code? Open in Web Editor NEWLearn to use https://concourse-ci.org with this linear sequence of tutorials. Learn each concept that builds on the previous concept.
Learn to use https://concourse-ci.org with this linear sequence of tutorials. Learn each concept that builds on the previous concept.
with vagrant you don't need to specify, but you may want to consider using the --target http://192.168.100.4:8080
syntax unless @vito or @xeobus explain what happened.
Is there a reason you're using aggregate
in pipeline-ls-abc-xyz.yml
but not the other two pipelines? From all I can understand about aggregate
you'd want to to use it for fetching all the resources. It also incidentally makes all the resources show up as dependencies in the UI, whereas when you just list them directly under the plan:
it seems to only visualize the first one.
I'd be happy to update it, just not sure if you had some intention with it or were just messing around. I really appreciate the repo, even these minimally documented ones are easier to grok than the docs 😸
While trying to run the tutorial, I got this error
The specified host network collides with a non-hostonly network!
This will cause your specified IP to be inaccessible. Please change
the IP or name of your host only network so that it no longer matches that of
a bridged or non-hostonly network.
when I ran the
vagrant up
How can I resolve this issue?
Thanks!
i desperately need to pass output from one plan to another, wonder if that's duable, or any way of passing parameters between plans, im stuck
Is there any way to include human intervention in alle pipeline, e.g. for approvals before deploying etc?
i installed concourse/stable in kubernetes cluster using helm charts.
when i try hello-world
i get this error
annoy
resource script '/opt/resource/check []' failed: exit status 1
stderr:
failed to ping registry: 2 error(s) occurred:
fly -t tutorial login -c http://10.3.11.17:31868/ -u atc -p atc
but i can't login
Hi,
I'm configuring a simple concourse pipeline to build a spring-boot app with maven and then deploy it to pivotal cloudfoundry. The problem is, that when deploying to CF I am unable to access the jar builded during the execution of maven package. Here is the pipeline and tasks files.
Pipeline:
**jobs:
and the task yaml file:
**platform: linux
image_resource:
type: docker-image
source:
repository: nitram509/debian-openjdk8-maven-cmake-gcc
tag: latest
inputs:
outputs:
run:
path: mvn
args: ["-f", "source-code/pom.xml", "-DskipTests=true","-DoutputDirectory=project-package","package"]**
Here is the error message that pops when trying to PUT the jar file to the CF resource :
"error invalid path: found 0 files instead of 1 at path: /tmp/build/put/project-package/
I think there is some configuration that need to be done to be able to access the output of the package task during the PUT step but I couldn't find my way around it.
Thank you.
What does a "credentials.yml" looks like?
I can't seem to get started... I don't understand where to look for the box. I thought it would be pulled down by clone this git repository.
Can anybody interpret for me?
Rex
$ vagrant box add concourse/lite --box-version 2.2.1
Found VERSION file, using concourse/lite version 2.2.1
The box 'concourse/lite' could not be found or
could not be accessed in the remote catalog. If this is a private
box on HashiCorp's Atlas, please verify you're logged in via
vagrant login
. Also, please double-check the name. The expanded
URL and error message are shown below:
URL: ["https://atlas.hashicorp.com/concourse/lite"]
Error:
A few steps of this tutorial depend on concourse/concourse-ci
- that was never meant to be a publicly used image, as it's only meant for our CI pipeline. It no longer exists - this tutorial should probably refer to a specific image built for the tutorial instead.
Hello,
Thanks for this awesome tutorial, really like it!
I'm currently thinking about using concourse as a tool for our company, but for some reason the vagrant machine eats up all my memory when running the tasks about Publishing Output. I doubt this is the expected behaviour.
I think the version of my softs are pretty up-to-date:
# Linux kernel
> uname -a
Linux dragonfly 4.5.5-1-MANJARO #1 SMP PREEMPT Thu May 19 17:20:19 UTC 2016 x86_64 GNU/Linux
# Go, Vagrant & Chromium
> go version
go version go1.6.2 linux/amd64
> vagrant version
Installed Version: 1.8.1
Latest Version: 1.8.1
# Regarding the latest commit I have & version
> git show | head -n1
commit 39fee8cd012b36b6bcc0fa826894186d2d36a17a
> cat VERSION
1.2.0
> chromium --version
Chromium 51.0.2704.63
If you have already faced this issue, or know where this is coming from it would be appreciated :)
Didn't see anything matching memory problem on the repository's past issues.
It might eventually be similar to: concourse/concourse#334
I think that if it's a known issue in the tutorial, it might be good to warn the users about it.
Cheers!
Thanks for sharing this tutorial, it's really really cool !!!
There's a little problem with inputs_required in 02_task_inputs though:
07/03 2016 11:48:20 jpic@concourse ~/concourse-tutorial/02_task_inputs (master)
$ fly -t tutorial execute -c no_inputs.yml
targeting http://localhost:8080
executing build 21
initializing with docker:///busybox
running ls -al
total 8
drwxr-xr-x 2 root root 4096 Mar 7 11:48 .
drwxr-xr-x 3 root root 4096 Mar 7 11:48 ..
succeeded
07/03 2016 11:48:28 jpic@concourse ~/concourse-tutorial/02_task_inputs (master)
$ fly -t tutorial e -c inputs_required.yml -i some-important-input=.
targeting http://localhost:8080
executing build 22
container: start: exit status 2
errored
^C
aborting...
I'm a newbie and if at least you could give me some clues to debug this it would help a lot !
Thanks in advance
Hi,
I have installed concourse and fly using the following tutorial on mac:
https://gist.github.com/kevin-smets/f20afd45a24ab3f88d01b2049ce7744f
and had to change the yml file like the following:
platform: darwin run: path: echo args: [hello world]
which worked, but now I am getting following error with 02_task_inputs
$ fly --target tutorial execute --config inputs_required.yml --input some-important-input=.
executing build 17
no workers satisfying: resource type 'archive'
available workers:
I tried Example 5 [05_pipeline_task_hello_world] and it fails with following error;
resource script '/opt/resource/check []' failed: exit status 1
stderr:
failed to ping registry: 2 error(s) occurred:
hi friends,
I had a problem while getting the latest version of bosh-stemcell-aws.
I excepted 3312.38 latest version but I got 3263.28 version of bosh-aws-xen-ubuntu-trusty-go_agent.
I didn't modify anything in pipeline.yml in 44_bosh_io path.
resources:
Is there a way to get the latest version 3312.38 ?
I'm using fly CLI 3.5.0 version.
Thank you for helping me in advance.
Thanks for this tutorial!
You now use fly login to set aliases for targets but when running on a local Virtualbox Concourse setup there isn't any auth set up so running fly -t tutorial login -c 'http://192.168.100.4:8080/'
is a no-op.
Solutions:
There is one step missing from the three-step description below the gif animation, not big deal, but I guess it is not too hard to change that. :)
The directions never say to run the job, and it doesn't start automatically. Not sure if it's necessary, but I spent a few minutes debugging this since all previous jobs have been executed.
I've gotten the same problem with Vagrant v. 1.9 and 2.0.1 - running the command
vagrant box add concourse/lite --box-version $(cat VERSION)
gives me this message & error:
Found VERSION file, using concourse/lite version 3.2.1
==> box: Box file was not detected as metadata. Adding it directly...
You specified a box version constraint with a direct box file
path. Box version constraints only work with boxes from Vagrant
Cloud or a custom box host. Please remove the version constraint
and try again.
so I can't get the vagrant box even running.
There is a benefit and a downside to abstracting tasks into YAML files outside of the pipeline.
One benefit is that the behavior of the task can be kept in sync with the primary input resource (for example, a software project with tasks for running tests, building binaries, etc).
One downside is that the pipeline.yml no longer explains exactly what commands will be invoked. Comprehension of pipeline behavior is potentially reduced.
But one benefit of extracting inline tasks into task files is that pipeline.yml files can get long and it can be hard to read and comprehend all the YAML. Instead, give tasks long names so that readers can understand what the purpose and expectation of the task is at a glance.
But one downside of extracting inline tasks into files is that fly set-pipeline is no longer the only step to updating a pipeline.
At first it says there's "a benefit and a downside" but then lists two of each. Another confusing part is that the second benefit reads like a downside. Am I reading this correctly?
the example to trigger the build from curl [1] does not work:
$ curl http://192.168.100.4:8080/pipelines/main/jobs/job-hello-world/builds -X POST
404 page not found
maybe @vito knows why? the concourse api does not seem to be documented on the concourse.ci pages that i could find.
I need to run several tasks to apply the terraform changes.
The tasks will be:
terraform plan ( a dry-run
)
block (waiting for unblock, y/n)
terraform apply (apply the changes)
So every time, when the resource (git) on branch master
detects a new commit, it will trigger the job terraform plan
. Then wait (block).
I need to read the output from the job terraform plan
if the configuration changes are fine. Then I can click a button (or input y/n) to unblock the pipeline and allow the next job run terraform apply
.
I have the same request to run ansible job with --check
as dry-run.
For your reference, you can read this document (https://buildkite.com/docs/pipelines/block-step) , it introduces the block function in buildkite
The Readme gives the example of parameterized a private key with the code:
github-private-key: |-
-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAuvUl9YUlDHWBMVcuu0FH9u2gSi83PkL4o9TS+F185qDTlfUY
fGLxDo/bn8ws8B88oNbRKBZR6yig9anIB4Hym2mSwuMOUAg5qsA9zm5ArXQBGoAr
...
iSHcGbKdWqpObR7oau2LIR6UtLvevUXNu80XNy+jaXltqo7MSSBYJjbnLTmdUFwp
HBstYQubAQy4oAEHu8osRhH1VX8AR/atewdHHTm48DN74M/FX3/HeJo=
-----END RSA PRIVATE KEY-----
This causes an error when setting the pipeline. The first line should be github-private-key: |
instead of github-private-key: |-
I am running the Linux version of the tutorial. After reinstalling multiple times (assuming I had made an error), I still get "file not found" after the step "using version of resource found in cache."
Looking at the output, I notice this:
3172f4 Merge pull request #106 from starkandwayne/windows-users
It says "windows-users." Perhaps somewhere in the pipeline there is a "hidden" reference to "windows?" I searched through the text of all the example files and I did not find any occurrence of "windows" and I did verify that each of the tutorial yml files does contain linux. Since I cannot complete this step, I do not know how to proceed with the rest of the tutorial.
If anybody has seen a similar issue, please reply.
Thanks,
Jo3
Do you have any examples of how to pass params to task files to use them within the task?
# task_file.yml
image_resource:
type: docker-image
source:
repository: {{docker-base-image}}
tag: {{docker-base-version}}
This works when tasks are inline but when you split it into a file I cant find any docs on how to do this.
The task docs (https://concourse.ci/running-tasks.html) state you can define a params section but I cant see an example of how to use them.
Right now the only way to check the the tutorial is running through it manually. Let's practice what we preach and use Concourse.
Work is underway in a branch.
this is a commented out line [1] that should probably be uncommented and modified to be a specific known-to-work box.
i found certain tutorial examples don't work with 0.54.0 because fly c -c pipeline.yml
no longer works without naming the pipeline. there are other changes too.
[1] https://github.com/jbayer/concourse-tutorial/blob/master/Vagrantfile#L16
golang-test-web-app
main.go:7:2: cannot find package "github.com/go-martini/martini" in any of:
/usr/src/go/src/github.com/go-martini/martini (from $GOROOT)
/go/src/github.com/go-martini/martini (from $GOPATH)
take this for example
set-pipeline --config pipeline.yml --pipeline my-pipeline
instead of
set-pipeline -c pipeline.yml -p my-pipeline
or you have to check the source to find out the meaning
(https://github.com/concourse/fly/blob/master/commands/set_pipeline.go)
PS. the document site (https://concourse.ci/fly-cli.html ) use long form.
Walking through the tutorial step-by-step. Made a gist, added
- name: resource-gist
type: git
source:
uri: [email protected]:71c94aad09a4adc12ebec5ac2794bab4.git
branch: master
into 12_publishing_outputs/pipeline.yml. A build attempt fails in resource-gist with
Cloning into '/tmp/build/get'...
Host key verification failed.
fatal: Could not read from remote repository.
Note this is pull, we're not at the push stage yet.
The host key is certainly not in ~/.ssh/known_hosts inside the task container. How / where can I run ssh-keyscan or push `known_hosts' there?
Hi!
I started implementing Concourse but I've hit a big problem - memory is leaked and it kills the machine that it's running on. It seems that memory is leaked by the Docker containers that are re-built every time a build plan is executed or something.
I'm using OSX.
Anybody else facing this problem?
For some reason the "job-run-task" is not triggered automated. I am a bit confused. Is this behaviour expected? I was expecting after running "job-fetch-resource" the second task will start but nothing is happen...
It might be a little hard to accomplish this, but I think it would be really useful every time we do a through pass on the concourse-tutorial we create a git tag that matches concourse's version.
That way anyone following the tutorial will have an idea what version of concourse to use.
Currently, the cf resource is only used to put the app (deploy)
How do I create a service and bind it to an app and then tear it down after testing ?
Seems like anyone using PWS is going to run into the same issue I did, that simple-go-web-app.cfapps.io
is already taken. Not sure what the right solution is, I know manifest files used to support {random-word}
or something in them for this purpose. I ended up creating a gist to host a manifest file with a different app name and adding it as another resource: https://gist.github.com/benmoss/ca6e66a63bbbdc5e8748
it show long time to perform the "checking pipeline is not paused" step, no error, just take long time to perform
After I resolved the issue with martini and got the tests to run, I was getting error reading request from stdin: json: cannot unmarshal string into Go value of type bool
from the deploy task.
I figured out that it was coming from skip-cert-check
in the CF configuration being turned into a string by fly. It turns out you can't parameterize anything in your YML file that's not just a string.
<alexsuraci> yea we force it to be a string to make marshalling weird values like private keys safer
Not sure if you want to remove this, hard code it, or what.
when typing vagrant up, the machine hangs trying to ssh in to the box.
Found VERSION file, using concourse/lite version 2.2.1
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'concourse/lite'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'concourse/lite' is up to date...
==> default: Setting the name of the VM: concourse-tutorial_default_1480446309370_92086
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
default: Adapter 1: nat
default: Adapter 2: hostonly
==> default: Forwarding ports...
default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
Need to confer with @drnic to clarify his intent, update manifests for current concourse, and write the README.
Hello,
Thanks for this tutorial. I am facing this issue.
Joe@SEVEN64 /c/Concourse/concourse-tutorial/02_task_inputs (master)
$ fly -t tutorial e -c inputs_required.yml -i some-important-input=.
executing build 12
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0curl: (7) Failed to connect to 192
.168.0.9 port 80: No route to host
gzip: invalid magic
tar: Child returned status 1
tar: Error is not recoverable: exiting now
exit status 2
�[31mfailed�[0m
upload request failed: Put http://192.168.0.9/api/v1/pipes/0711287c-1dd7-4db7-58a7-8c23fb7b6762: dial tcp 192.16
8.0.9:80: i/o timeout
I have no clue what's going on here. Can you help please?
The current example shows usage of the git protocol for both operations
Thanks for the tutorial!
Unfortunately, I am running into a technical issue on the first exercise. I am trying to run through your tutorial using docker to run concourse and not virtualbox/vagrant. The problem I am encountering is that (after getting concourse running using the concourse/concourse docker-compose.yml) when I try to run the very first task from the tutorial (task_hello_world.yml), it fails because it is trying to use the aufs storage driver as opposed to the default "Docker for Mac" storage driver, overlay2.
Is there a way to fix this without forcing my Docker daemon to use aufs as the storage driver? All other images I've ran use overlay2, so I'd like to stick with it, if possible.
fly -t tutorial execute -c task_hello_world.yml
executing build 1
initializing
Pulling busybox@sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f...
sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f: Pulling from library/busybox
7520415ce762: Pulling fs layer
7520415ce762: Verifying Checksum
7520415ce762: Download complete
7520415ce762: Pull complete
Digest: sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
Status: Downloaded newer image for busybox@sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
Successfully pulled busybox@sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f.
docker: Error response from daemon: error creating aufs mount to /var/lib/docker/aufs/mnt/66174d331c8ee6e48312a93b9bfe8912031752c38b5e4d292383046c59c18493-init: invalid argument.
See 'docker run --help'.
resource script '/opt/resource/in [/tmp/build/get]' failed: exit status 125
errored
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.