cloudfoundry-community / cf-resource

Concourse resource for interacting with Cloud Foundry

Cloud Foundry Resource

An output only resource (at the moment) that will deploy an application to a Cloud Foundry deployment.

Source Configuration

Note: you must provide either username and password or client_id and client_secret.

• api: Required. The address of the Cloud Controller in the Cloud Foundry deployment.
• username: Optional. The username used to authenticate.
• password: Optional. The password used to authenticate.
• client_id: Optional. The client id used to authenticate.
• client_secret: Optional. The client secret used to authenticate.
• organization: Required. The organization to push the application to.
• space: Required. The space to push the application to.
• skip_cert_check: Optional. Check the validity of the CF SSL cert. Defaults to false.
• verbose: Optional. Invoke cf cli using CF_TRACE=true to print all API calls made to Cloud Foundry.

Behaviour

out: Deploy an application to a Cloud Foundry

Pushes an application to the Cloud Foundry detailed in the source configuration. A manifest that describes the application must be specified.

Parameters

• manifest: Required. Path to a application manifest file.
• path: Optional. Path to the application to push. If this isn't set then it will be read from the manifest instead.
• current_app_name: Optional. This should be the name of the application that this will re-deploy over. If this is set the resource will perform a zero-downtime deploy.
• environment_variables: Optional. It is not necessary to set the variables in manifest if this parameter is set.
• vars: Optional. Map of variables to pass to manifest
• vars_files: Optional. List of variables files to pass to manifest
• docker_username: Optional. This is used as the username to authenticate against a protected docker registry.
• docker_password: Optional. This should be the users password when authenticating against a protected docker registry.
• show_app_log: Optional. Tails the app log during startup, useful to debug issues when using blue/green deploys together with the current_app_name option.
• no_start: Optional. Deploys the app but does not start it. This parameter is ignored when current_app_name is specified.

Pipeline example

---
jobs:
- name: job-deploy-app
  public: true
  serial: true
  plan:
  - get: resource-web-app
  - task: build
    file: resource-web-app/build.yml
  - put: resource-deploy-web-app
    params:
      manifest: build-output/manifest.yml
      environment_variables:
        key: value
        key2: value2

resources:
- name: resource-web-app
  type: git
  source:
    uri: https://github.com/cloudfoundry-community/simple-go-web-app.git

- name: resource-deploy-web-app
  type: cf
  source:
    api: https://api.run.pivotal.io
    username: EMAIL
    password: PASSWORD
    organization: ORG
    space: SPACE
    skip_cert_check: false

Development

Prerequisites

• golang is required - version 1.9.x is tested; earlier versions may also work.
• docker is required - version 17.06.x is tested; earlier versions may also work.
• godep is used for dependency management of the golang packages.

Running the tests

The tests have been embedded with the Dockerfile; ensuring that the testing environment is consistent across any docker enabled platform. When the docker image builds, the test are run inside the docker container, on failure they will stop the build.

Run the tests with the following commands for both alpine and ubuntu images:

docker build -t cf-resource -f dockerfiles/alpine/Dockerfile .
docker build -t cf-resource -f dockerfiles/ubuntu/Dockerfile .

Contributing

Please make all pull requests to the master branch and ensure tests pass locally.

cf-resource's Issues

Add ability to override time out environment variable CF_DIAL_TIMEOUT

I have a single VM installation of Concourse. I am not able to login to cf with default timeout of 5s. When I setup the environment variable CF_DIAL_TIMEOUT to 20, it works.

This works on my VM but my Concourse job still fails. Is there a way to override this environment variable value at resource declaration?

Yes the cf-resource to create service

Currently, the cf resource is only used to put the app (deploy)

How do I create a service and bind it to an app and then tear it down after testing ?

Boolean manifest variables don't interpret the same in pipeline vs vars file

When using manifest variable substitution, booleans in the pipeline.yml are interpreted differently than when passed via a vars_file. In one case, booleans must be quote wrapped and in the other they must not be.

Example When Set In Pipeline
Broken config:

  - put: development
    params:
      path: snapshot/*.war
      manifest: src-dev/cf/manifest.yml
      vars:
        random_route: true

Corresponding error:

Incorrect Usage: invalid argument for flag `--var' (expected []template.VarKV): Deserializing variables 'random_route=%!s(bool=true)': yaml: could not find expected directive name

Working config:

  - put: development
    params:
      path: snapshot/*.war
      manifest: src-dev/cf/manifest.yml
      vars:
        random_route: 'true'

Example When Set In Vars File
Config:

  - put: development
    params:
      path: snapshot/*.war
      manifest: src-dev/cf/manifest.yml
      vars_files:
      - /tmp/build/put/src-dev/cf/devstar.yml

Broken devstar.yml:

random_route: 'true'

Corresponding error:

yaml: unmarshal errors:
  line 7: cannot unmarshal !!str `true` into bool

Working devstar.yml:

random_route: true

Concourse Deployment
Version: 4.2.1
IaaS: AWS GovCloud
Deployment: BOSH

Setting "environment_variables" gives Deprecation warning

I've used the configuration like this, in the Concourse pipeline.yml:

...
    - put: cf-resource
      params:
        path: my-app/*.jar
        manifest: my-app/manifest.yml
        environment_variables:
          TEST_ENV_VARIABLE: some-value

Everything worked fine and as expected but I got this Deprecation warning in the Concourse logs:

Deprecation warning: Specifying app manifest attributes at the top level is deprecated. Found: env.
Please see http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#deprecated for alternatives and other app manifest deprecations. This feature will be removed in the future.

I have no experience with Go language but I guess /out/manifest.go should be updated, that env part is set not on the root level. Just my assumption.

Longer timeout for large applications

I'm trying to push some app(stratos in specific), which takes time and always ended up with:
FAILED Error restarting application: console failed to stage within 15.000000 minutes error running command: exit status 1

We can set CF_STAGING_TIMEOUT for cf cli to do cf push, but have no idea how to have a longer timeout in concourse.

Support passing --var

See https://github.com/cloudfoundry/cli/releases/tag/v6.37.0 "Variable Substitution" section and https://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#multi-manifests

cf push --vars-file vars.yml is now a delightful thing. Can we please add vars-file to this resource + bump to v6.37 cli

option to fail `put` if app fails initial heath checks

I have a build which does puts to PWS, which causes the app to crash 1m after starting because it "never became healthy". this to me constitutes a failure to deploy, rather than a crash during runtime.

I would like an option to see this block the pipeline.

Failed pushes leave *-venerable apps the make subsequent builds fail

Given a previous build failed to deploy and app due to app/CF errors

Server error, status code: 400, error code: 100002, message: The app name is taken: donations-presentation-venerable
error: Error executing cli core command
error running command: exit status 1

How do i build it?

I guess i can probably run the CI script locally, once i figure out what the other inputs are. It would be nice if there was some documentation about this, or a driver script or something. I can raise a PR if i get it working.

Deploy new app

I think the resource currently cannot delete an app for the first time:

App haash-broker-1 not found

Colour in the output has disappeared in recent releases

This doesn't appear to be working anymore: https://github.com/concourse/cf-resource/blob/1aeb28f67de5d07b760ebd98d450b4f9be3591a8/out/cloud_foundry.go#L81

A cf-resource env variable should override the same one provided by a manifest

According to [this comment] about env variable support (#4 (comment)): "It now works by adding the environment variables to the manifest, before doing the cf push". I thought an env variable provided by cf-resource would override one provided by a manifest. But it isn't.
What do you think ?

Pipeline extract

   put: resource-deploy-cf-webui
    params:
      manifest: manifest.yml
      environment_variables:
        API_ENDPOINT: https://my-url.com
        API_ENDPOINT2: https://my-url.com 

Manifest extract

  env: 
    API_ENDPOINT: https://api.run.pivotal.io

Cf env

User-Provided:
API_ENDPOINT: https://api.run.pivotal.io
API_ENDPOINT2: https://my-url.com

current_app_name behavior loses environment variables

When the zero-downtime deployment is triggered by the current_app_name property, it does not transfer over the environment variables that were set on the currently deployed app. This results in loss of any environment variables which were set via cf set-env rather than by the manifest or pipeline.

One could say that this is by design, however it runs counter to the cf push behavior, which maintains any currently set environment variables that are not present in the manifest.

Push docker image from Amazon Elastic Container Registry (ECR)

I'm having an issue deploying docker containers to CF from Amazon Elastic Container Registry (ECR).

The way that ECR authentication works, if you want a username/password you need to do a separate aws cli call using your access_key / secret to get a short-lived username/password (see AWS Docs).

Now I could do this in a task, but there is no way that I know of to pass that password into the docker_password param of the following cf-resource put. I'm beginning to think that it isn't possible to push from ECR to CF using the cf-resource, hence I'm raising this issue.

I think, ideally, the cf-resource would be enhanced to offer specific support for ECR as a source, taking AWS key/secret as params, but I accept that this is a bit of work and perhaps not strictly a concern of this resource.

Failing that, I would be able to work around the issue if it were possible to specify a docker_password_file param, pointing to a file containing the password that I have populated in an earlier task.

Pushing a big file with zero down time consistently fails.

Whenever trying to push a pretty big app with a lot of files, the zero down time deployment fails.

error: Error processing app files: Error uploading application.
	
Error performing request: Put https://api.run.pivotal.io/v2/resource_match: http: ContentLength=144633 with Body length 0
	
error running command: exit status 1

When looking at the logs, the request for resource_match never comes back with a response.

If pushing without zero downtime, the cf cli handles the failure to do resource_match appropriately.

I think this is because of the fixes in the CLI for the following issues -
cloudfoundry/cli#1042
cloudfoundry/cli#1123

In our case we are facing the issues that were faced in this issue:
cloudfoundry/cli#1123

[Enhancement] Allow setting up custom startup command

I was wondering if it would be possible to add the cf push -c MYSTARTUPCMD functionality to this resource: http://cli.cloudfoundry.org/en-US/cf/push.html

In particular I would like to be able to deploy a docker container with different startup commands. Of course I could just create individual docker containers, which is what I am currently doing, but it appears that -c option to add a startup command could be valuable.

Please correct me if I'm wrong, but I didn't find a way to actually put this into the manifest file, which of course would also be a great place for it.

Allow CF_TRACE output for failed pushes

During deploys to CF (especially 3rd-party provided CFs) it's useful (or required by support) to be able to provide the CF_TRACE output for debugging failed pushes

Add configuration setting to put `cf push` in verbose mode

For debugging purposes, it would be nice to be able to set CF_TRACE=true when running this resource. I had a problem with an extra / at the end of an API endpoint that caused the CF CLI to fail with an "Unknown command" error that didn't tell me much about what is wrong.

I propose adding a configuration setting debug, that defaults to false, to help with this.

CloudFoundry Zero-Downtime deployment through (community) plugins

Would it be conceivable to add support for running cloud foundry (community) plugins. I'm not sure if this is the right place to add this, so please advice if not.

I am essentially trying to get continuous deployment setup for an application we are deploying via CloudFoundry through concourse.

Currently, I am using the cf-resource in concourse to push my manifest files, but it takes down the application along the way and then restarts it. What I want is to use for example autopilot to manage the deployment so that I have zero-downtime.

Capture `cf logs --recent` on a failed deploy

Since the cf-resource uses a zero-downtime deploy, if the deploy of app fails, then it's deleted and replaced with app-venerable meaning you can't call cf logs --recent on the failed app deployment. The resource should grab that log before it deletes the app.

Colour code resource based on success

The Git resources change to orange in the GUI when a GET fails, could the same be implemented for the CF resource when a PUT fails?

At the moment if a task has many cf resources then it is impossible to tell which CF resource had an issue without drilling down into the task.

Doesn't use current_app_name for the app name

The resource only uses current_app_name to discover which app to rename; but then uses the manifest.yml name for the new app name. Instead it should continue using current_app_name.

Push docker image directly to cloud foundry

I am trying to use the cf resource to push a docker image to CF but that seem to be not supported by this resource. Do you think you will have a update on it?

If path param is a directory, it is not passed to cf push

I'm deploying a .net core app, specifying both the path and manifest parameters. As the documentation implies, I'm expecting the path in my pipeline to take precedence over the path in the manifest. However, in my case the path does not point to a single artifact (i.e., jar), but rather a directory with a bunch of dlls.

If I'm reading the code correctly, when path is a directory, we navigate into that directory and push from there, but never specify the -p flag. Therefore, the pipeline attempts to use the path in the manifest, which is not the desired behavior.

https://github.com/concourse/cf-resource/blob/783280d98c69b0de44eb8678aef14b91542c2353/out/cloud_foundry.go#L47-L60

relevant snippet of pipeline.yml:

jobs:
- name: build-and-deploy
  plan:
  - get: my-app
    trigger: true
  - task: build and test
    file: my-app/ci/build.yml
  - put: deploy
    params:
      manifest: my-app/manifest.yml
      path: build-output # <-- the built dlls are placed here

manifest.yml :

---
applications:
- name: my-app
  path: src/MyApp/bin/Release/netcoreapp1.1/publish/
  random-route: true

Tail cf logs during staging when using blue/green deployment

I was just bitten by not having cf logs available when using cf-resource together with blue/green deployments (specifying params.current_app_name). Because cf-resource deletes the staging app when staging fails, there's no way to recover these logs either.

Proposed solution: cf-resource should connect to cf logs current_app_name and pipe the output to stdout until the app was successfully started.

can't deploy stratos UI from a pipeline

hello

i try to doing a pipeline for manage my deploy of Stratos ui : (https://github.com/cloudfoundry-incubator/stratos )

i've done this pipeline :

resources:
- name: stratos-staging
  type: cf
  source:
    api: https://xxxxxxxxxxx
    organization: Tools
    username: admin
    password: ((cf_admin_password))
    skip_cert_check: true
    space: console

- name: stratos-gh-release
  type: git
  source:
    uri: https://github.com/cloudfoundry-incubator/stratos.git

jobs:
- name : deploy-stratos
  plan:
    - get: stratos-gh-release
    - put: stratos-staging
      params:
        manifest: stratos-gh-release/manifest.yml
        environment_variables:
          CF_API_URL:  https://xxxxxxxxxx

and i have this error :

Setting api endpoint to https://api.xxxxxxxxxxxxxxxxx...
OK

api endpoint:   https://xxxxxxxxxxxxxxxxx
api version:    2.107.0
Not logged in. Use 'cf login' to log in.
API endpoint: https://api.xxxxxxxxxxxxxxxxx
Authenticating...
OK
Use 'cf target' to view or set your target org and space.
api endpoint:   https://api.xxxxxxxxxxxxxxxxx
api version:    2.107.0
user:           admin
org:            Tools
space:          console
Pushing from manifest to org Tools / space console as admin...
Using manifest file /tmp/build/put/stratos-gh-release/manifest.yml


Deprecation warning: Specifying app manifest attributes at the top level is deprecated. Found: env.
Please see http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#deprecated for alternatives and other app manifest deprecations. This feature will be removed in the future.


Using manifest file /tmp/build/put/stratos-gh-release/manifest.yml

Updating app console in org Tools / space console as admin...
OK

Using route console.xxxxxxxxxxxxxxxxx
Uploading console...
Uploading app files from: /root
Uploading 2.9K, 4 files
Done uploading               
OK

Starting app console in org Tools / space console as admin...
Cell 16a44f5d-f092-49e9-afab-fa4676618b68 creating container for instance ebf28d3f-ece1-44ed-9688-952fe70308a0
Cell 16a44f5d-f092-49e9-afab-fa4676618b68 successfully created container for instance ebf28d3f-ece1-44ed-9688-952fe70308a0
Downloading build artifacts cache...
Downloading app package...
Downloaded app package (3.6M)
Downloaded build artifacts cache (20.7M)
Stratos UI Buildpack
-----> Stack cflinuxfs2
-----> Download go 1.9
-----> Download glide 0.13.0
-----> Download Nodejs 6.11.3
-----> Checking versions
---------> Node Version: v6.11.3
---------> NPM Version: 3.10.10
---------> Go Version: go version go1.9 linux/amd64
---------> Glide Version: glide version v0.13.0
Cache directory [/tmp/cache/final] exists
-----> Running stratos build compile [12:47:47]
/tmp/buildpackdownloads/652661b4b3c6701c6981e0b449bf726e/bin/compile: line 77: /tmp/app/deploy/cloud-foundry/build.sh: No such file or directory
Failed to compile droplet: Failed to compile droplet: exit status 1
Exit status 223
Cell 16a44f5d-f092-49e9-afab-fa4676618b68 stopping instance ebf28d3f-ece1-44ed-9688-952fe70308a0
Cell 16a44f5d-f092-49e9-afab-fa4676618b68 destroying container for instance ebf28d3f-ece1-44ed-9688-952fe70308a0
Cell 16a44f5d-f092-49e9-afab-fa4676618b68 successfully destroyed container for instance ebf28d3f-ece1-44ed-9688-952fe70308a0

FAILED
Error restarting application: BuildpackCompileFailed

but if i do same from cli (cf push) with same version of stratos/cf , that works :

 cf push
Pushing from manifest to org Tools / space console as admin...
Using manifest file /home/ubuntu/GIT/stratos/manifest.yml


Deprecation warning: Route component attributes 'domain', 'domains', 'host', 'hosts' and 'no-hostname' are deprecated. Found: host.
Please see http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#deprecated for the currently supported syntax and other app manifest deprecations. This feature will be removed in the future.


Using manifest file /home/ubuntu/GIT/stratos/manifest.yml

Updating app console in org Tools / space console as admin...
OK

Using route console.xxxxxxxxxxxxxxxxx
Uploading console...
Uploading app files from: /home/ubuntu/GIT/stratos
Uploading 4.6M, 3443 files
Done uploading
OK

Starting app console in org Tools / space console as admin...
Cell 16a44f5d-f092-49e9-afab-fa4676618b68 creating container for instance 9511263a-e10b-45cd-a74f-a74be302dfca
Cell 16a44f5d-f092-49e9-afab-fa4676618b68 successfully created container for instance 9511263a-e10b-45cd-a74f-a74be302dfca
Downloading build artifacts cache...
Downloading app package...
Downloaded app package (8.8M)
Downloaded build artifacts cache (20.7M)
Stratos UI Buildpack
-----> Stack cflinuxfs2
-----> Download go 1.9
-----> Download glide 0.13.0
-----> Download Nodejs 6.11.3
-----> Checking versions
---------> Node Version: v6.11.3
---------> NPM Version: 3.10.10
---------> Go Version: go version go1.9 linux/amd64
---------> Glide Version: glide version v0.13.0
Cache directory [/tmp/cache/final] exists
-----> Running stratos build compile [12:56:24]
Preparing application folder for Cloud Foundry deployment
npm WARN deprecated [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue

i see we have not the same behavior :

pipeline :

Using route console.xxxxxxxxxxxxxxxxx
Uploading console...
Uploading app files from: /root
Uploading 2.9K, 4 files
Done uploading               
OK

and cli version :

Using route console.xxxxxxxxxxxxxxxxx
Uploading console...
Uploading app files from: /home/ubuntu/GIT/stratos
Uploading 4.6M, 3443 files
Done uploading
OK

do you know why the upload weigth is not the same between this 2 method ? :-/

cf-resource to start app

Instead of pushing app, can you kindly show how to configure to "start/stop/list" applications?

JAR output, to be used as put input

How do we set the jar as an input for the deploy step?

i build the jar in a separate container, but the put step does not pick it up...

---
resources:
- name: app
  type: git
  source:
    uri: <URI>
    branch: master

- name: deploy-web-app
  type: cf
  source:
    api: ((cf-api))
    username: ((cf-username))
    password: ((cf-password))
    organization: ((cf-organization))
    space: ((cf-space))
    skip_cert_check: true

jobs:
- name: deploy-app
  public: true
  serial: true
  plan:
  - get: app
    trigger: true
#   - task: app-tests
#     config:
#       platform: linux
#       image_resource:
#         type: docker-image
#         source: {repository: java, tag: openjdk-8-alpine}
#       inputs:
#       - name: app
#       run:
#         path: app/test-main.sh
  - task: app-build
    config:
      platform: linux
      image_resource:
        type: docker-image
        source: {repository: java, tag: openjdk-8-alpine}
      inputs:
      - name: app
      outputs:
        - name: build-output
      run:
        path: app/build-main.sh
      

  - put: deploy-web-app
    params:
      manifest: app/manifest.yml
      path: build-output/app.jar

Error setting integer-based env vars of a cf resource

My partner (@athornton2012) and i ran into an interesting bug today when we

Version of concourse: 5.0.0

Background: Our team has a job which puts to a cf resource, and sets a number of env variables as it does so. We added a new env var this afternoon, which was a plain integer that represented the port of a server.

Expected: The resource upload to succeed without issue.

Actual: Received the following error message: error reading request from stdin: json: cannot unmarshal number into Go struct field Params.environment_variables of type string

As a workaround, we had to manually put the int in quotes for it to be interpolated as a string. When we did that, the error stopped and we were able to upload to the resource again

Our question is: is this intended behavior? Is the cf resource unable to process environment variables that are integers? Or is this a bug?

Thank you!

Regression: Docker deployment from private registry appears broken (after upgrade to concourse 3.9.2 from 3.8.0)

Hi, I have upgraded to 3.9.2 from 3.8.0 and am experiencing fatal errors when trying to deploy any docker image from a private registry. The error as shown below.

For some reason the provided CF_DOCKER_PASSWORD is no longer being found or the input arguments to cf push are messed up. Any help or currently working examples with private docker registries would be much appreciated.

Related: nulldriver/cf-cli-resource#24

Add ability to delete app

A common pattern we have is the following:

1. deploy one or more apps in an integration-testing space
2. run integration tests
3. delete the apps deployed in step 1

Currently there's no way to do step 3 with the cf resource, so we have to resort to scripts. Would be nice if the cf resource was extended to allow this kind of operations

Cannot specify environment variable as false

Using Concourse v0.67.1

When I specify an environment variable like this:

environment_variables:
  SOME_VAR: false

And I start my app deployment build
Then I see this error:

error reading request from stdin: json: cannot unmarshal bool into Go value of type string

However, when I specify the environment variable like this:

environment_variables:
  SOME_VAR: "false"

Then my app deployment succeeds
And the environment variable is set

As a user of Concourse I would like to use true and false without special cases in my pipeline configuration.

allow setting CF_DIAL_TIMEOUT

Given Australia is as far away from PWS as is the moon,
When my Concourse in Australia tries to do a deploy to PWS in the USA
It runs out of juice during the huge round trip and times out.

When I do a manual deploy locally
I have to set the timeout with the following:
CF_DIAL_TIMEOUT=300 cf api https://api.run.pivotal.io`

Reading source code with my limited golang glasses on I am not able to clearly see if this can be done.

Failed to have a zero downtime deploy

We're providing the current_app_name which matches the application name in our manifest.yml. We were expecting this configuration to result in a zero-downtime-deploy based on the documentation. However, our application did go down during the deploy, resulting in a 404 on that route (ex: myapp.somecloudfoundry.com)

task yaml

  - put: cloud-foundry-resource
    params:
      manifest: built/manifest.yml
      current_app_name: myapp

manifest.yml

---
applications:
- name: myapp
  path: .
  buildpack: python_buildpack
  command: python app/application.py

Allow overriding of app name in manifest

I have a need to decide, in the pipeline, what the cf app name needs to be. It would be handy to be able to override the name value in the manifest.yml from the cf-resource thusly:

- put: cf-resource
  inputs:
  - name: release
  params:
    new_app_name: my-app-12345
    manifest: release/manifest.yml
    path: release

cf-resource using CLI v6.13 (released October 2015) - please bump CLI version

Description
The current version of CLI bundled into the cf-resource container is 6.13. This older version of CLI is preventing a sample Java app from deploying successfully on PWS (repro steps for app deploy fail below). The app binds to monitoring & log services (hostedgraphite, papertrail).

Can we update the image build process to include latest version of CLI ?

Container is using this version of CLI
6.13.0-e68ce0f-2015-10-15T22:53:58+00:00

The Latest version of CLI currently available
6.17.0+5d0be0a-2016-04-15

Repro steps

1. Clone this repo, follow setup steps (for services & PWS credentials)
   https://github.com/joshlong/operationalized-applications-meetup
   (you need to create file ~/.pws-operationalized.yml with your PWS credentials)

2. Load the pipeline "ci/pipeline.yml" on Concourse
   fly -t lite set-pipeline -p op-apps -c ci/pipeline.yml -l ~/.pws-operationalized.yml

3. Run the pipeline, you'll get this error in the deploy stage, right after the Maven build stage
   Uploading rr-php-ag...
   FAILED
   Error uploading application.
   Server error, status code: 422, error code: 10008, message: The request is semantically invalid: must be an array.
   error: Error executing cli core command
   error running command: exit status 1

4. If you hijack the container and bump CLI to 6.17, the error disappears and the app is deployed successfully without issues.

show_app_log should also work when not using b/g deploy

I totally missed #40 only implements live-tailing of application logs for b/g cf pushs. It would be really useful to have the same capability for "normal" cf pushs.

Support configuring auto-scaling on push

Currently there is no task for configuring auto-scaling when deploying an application. You can bind to an auto-scaler service like you would any other. However, it will have no auto-scaler setting by default. It would be nice to have an option here to provide the manifest or location to it and have it uploaded with the application vs making it another task.

Specify app route in concourse job params

We're trying to blue-green deploy an app to multiple environments with different routes and environment variables in each environment. The only thing preventing us from using the cf-resource is the lack of support for specifying routes in the concourse job. As is, we can use different manifests for each environment, but it seems like overkill since there's only one difference between the environments (the route).

Looking at the code, it seems like this would involve adding an AddRoute method to https://github.com/concourse/cf-resource/blob/master/out/manifest.go . I'd be happy to submit a pull request if you think this idea makes sense.

Custom `routes:` in manifest not honoured

I'm trying to push an app with the following manifest:

---
applications:
  - name: foo
    memory: 64M
    buildpack: staticfile_buildpack
    routes:
    - route: myplace.cfapps.io/v1_2_3
    - route: myother.place.com/v1_2_3

When pushing manually from my local machine, the output includes e.g.:

Creating route myother.place.com/v1_2_3...
OK

Binding myother.place.com/v1_2_3 to foo...
OK

Creating route myplace.cfapps.io/v1_2_3...
OK

Binding myplace.cfapps.io/v1_2_3 to foo...
OK

But the cf-resource only does this:

Creating route foo.cfapps.io...
OK

Names changed to protect the innocent.

Unable to use new manifests when using "current_app_name" param

If I have a put like

    - put: deploy-transaction
      resource: deploy-cf
      params:
        manifest: archive/transaction/manifest.yml
        current_app_name: transaction

And a manifest like

applications:
- name: transaction
  services:
  - redis
  buildpacks:
  - java_buildpack_offline
  memory: 1024M
  path: build/libs/transaction-1.0.0.jar

Then when deploying we get an error like:

Renaming app transaction to transaction-venerable in org MY_ORG / space temp as ci-user...
OK
Deleting app transaction in org MY_ORG / space temp as ci-user...
OK
App transaction does not exist.
Renaming app transaction-venerable to transaction in org MY_ORG / space temp as ci-user...
OK
error: Error reading manifest file:
The following manifest fields cannot be used with legacy push: buildpacks


error running command: exit status 1

Note that this works if we remove current_app_name, like so:

    - put: deploy-transaction
      resource: deploy-cf
      params:
        manifest: archive/transaction/manifest.yml

OR

If we change buildpacks to buildpack:

applications:
- name: transaction
  services:
  - redis
  buildpack: java_buildpack_offline
  memory: 1024M
  path: build/libs/transaction-1.0.0.jar

Add `in` and `check` script

Would it be of use to anyone else to have a check that emitted a version that was some sort of hash of app bits and (generated) app manifest, along with an in to get both?

It'd be nice to drive pipelines a bit more semantically by knowing when an app has changed, rather than relying on passing whatever caused that change to propagate down the pipeline.

CF_DOCKER_PASSWORD doesn't get set through environment-variables

Hi, I'm trying to use this resource (for the first time) and ran into trouble wen trying to push docker images as shown below. My problem is that I get this error from my pipeline and can't figure out why it things I didn't provide CF_DOCKER_PASSWORD:

FAILED
No Docker password was provided. Please provide the password by setting the CF_DOCKER_PASSWORD environment variable.
error running command: exit status 1

resources:
[...]
- name: deploy-dev
  type: cf
  source:
    api: {{cf-api-endpoint}}
    username: {{cf-username-email}}
    password: {{cf-password}}
    organization: {{cf-organization}}
    space: {{cf-target-space}}
    skip_cert_check: false

jobs:
[...]
- name: deploy-dev
  plan:
  - get: git
    passed: [build-docker-image]
  - get: docker-registry
    passed: [push-docker-registry]
    trigger: true
  - put: deploy-dev
    params:
      manifest: git/manifest-dev-docker.yml
      environment_variables:
        CF_DOCKER_PASSWORD: {{docker-registry-password}}
        OTHER: foo

The manifest-dev-docker.yml file looks something like this:

applications:
- name: dev-docker
  docker:
    image: myregistry.com/dev:latest
    username: myusername
  memory: 512M
  instances: 1
  services:
    - dev
  env:
    OTHER: foo

Do I need to completely remove the env section from the manifest, when trying to provide them through the pipeline or where is my error here ?

add ability to keep old version

Not sure if this feature exists but not documented. If I use zero-downtime deployment, the process deploys the new version and deletes the old. Is it possible to add a feature which will always keep the old version and just rename it and remove the route? This allows us to rollback without much downtime in case the new app comes online and crashes later. Has this come up before?

Accept globs as parameters

Sometimes you don't know the exact filename that you are deploying (version numbers, etc.). It would be helpful if the resource resolved these globs for you. Both the application path and the manifest path should accept globs.

If more than one file matches the glob then an error should be raised.

- put: app
  params:
    manifest: config/manifest-*.yml
    path: other-input/app-*.jar

The Cf resource can only be used against one space.

At the moment if you wish to have one pipeline deploy to many cloud foundry organisations and spaces you have to configure many resources.

Could it be changed so that organisation and space are required params instead of source? That would allow to have as many resources as you have cf foundations but target as many organisations and spaces as you would like

Parallel zero-downtime deployment doesn't work

In our project we're using the zero-downtime deployment functionality of this resource by stating a current_app_name. This fails if two deployments are happening at the same time with the error

error: Server error, status code: 400, error code: 100002, message: The app name is taken: my-app-venerable

I think it would make sense to add a random string to the end of the app name (e.g. my-app-venerable-xy123) to avoid these collisions or to be able to serialize just the deployment. Otherwise I have to serialize the whole job using https://concourse.ci/configuring-jobs.html#serial which tremendously slows down our pipeline.

Pass env variable based on pipeline resource

Could I pass an "app version" semver resource into an app's env variable?

bluemix authentication failure

I can't authenticate with bluemix using this resource. Not sure if it happened since taking the latest concourse upgrade 3.2.1, or if it's something on the bluemix side that has changed.

A manual cli cf login works, but a manual cli cf auth fails

https://stackoverflow.com/questions/45029574/concourse-cf-resource-bluemix-authentication

As per my comment above this looks like an issue with version 6.2.7 of the cf cli, 6.2.8 works fine.

Environment variables defined in Manifest cannot be override

It is good practice to set default values for environment variables in the Manifest and only deploy time override those which require change.

While new variables are added to the deployment, the Manifest (default) values remain the same.

See pipeline snippet used where I set a new variable (version) and try to override the eureka client to false.

- name: my-app
  public: false
  serial: true
  plan:
  - get: my-app-deployment
  - get: my-app-release
  - put: cf-deploy
    params:
      manifest: my-app-deployment/cloudfoundry/manifest.yml
      path: my-app-release/my-component-((my-version)).jar
      current_app_name: my-app
      environment_variables:
         version: ((my-version)
         eureka.client.enabled: 'false'

manifest.yml

---
applications:
  - name: my-app
    instances: 1
    buildpack: https://github.com/cloudfoundry/java-buildpack#v4.2
    path: ./my-app.jar
    memory: 1024M
    disk_quota: 256M
    stack: cflinuxfs2
    no-route: true
    env:
       eureka.client.enabled: true

Now, when I inspect the app env after deployment...

$ cf env my-app
<...>
User-Provided:
eureka.client.enabled: true
version: 0.0.3