Currently this plugin can only retrieve docker images from a specified ECR in the same AWS account. Is this by design ?

For an application, the problem is that the prod and dev version runs in different AWS accounts and were deployed using different buildkite agents in different queues.

This means the same docker image would need to be created/stored in multiple ECR in different AWS accounts.

Any suggestions for this issue ?

Feature request

it would be nice to pass in your own ecr repo rather than the plugin create it for you.

also, i see that the repo created has no lifecycle rules. As i am using this to cache node modules, i would like to keep image count less than x

Cheers,
Shane.

Hi,

We have been successfilly using this plugin with the docker image plugin.
We also have our in-house plugin that build images and do a bunch of other things for us, and I'd like to be able to chain this cache plugin with our plugin.
For that to work, we would need to be able to override the name of the exported variable containing the image name and tag.

I forked the repo and have a working change I'm happy to merge into the main repo.

Of course, the change would be backward compatible and the variable name would still default to BUILDKITE_PLUGIN_DOCKER_IMAGE

I get a permission error when mounting my cached node modules

error is

yarn run v1.15.2
--
  | warning Skipping preferred cache folder "/usr/local/share/.cache/yarn" because it is not writable.
  | warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-0".
  | $   eslint --fix src && rimraf reports pacts logs && cross-env NODE_ENV=test nyc mocha
  | /workdir/node_modules/mkdirp/index.js:90
  | throw err0;
  | ^
  |  
  | Error: EACCES: permission denied, mkdir '/workdir/node_modules/.cache'
  | at Object.mkdirSync (fs.js:773:3)
  | at sync (/workdir/node_modules/mkdirp/index.js:71:13)
  | at sync (/workdir/node_modules/mkdirp/index.js:77:24)
  | at sync (/workdir/node_modules/mkdirp/index.js:77:24)
  | at save (/workdir/node_modules/@babel/register/lib/cache.js:50:20)
  | at processTicksAndRejections (internal/process/task_queues.js:79:9)
  | at process.runNextTicks [as _tickCallback] (internal/process/task_queues.js:56:3)
  | at Function.Module.runMain (internal/modules/cjs/loader.js:871:11)
  | at internal/main/run_main_module.js:21:11

looks like there is no user for node modules

drwxr-xr-x 1071 nobody nogroup 28672 Apr 17 04:08 node_modules

pipeline.yaml

plugins: &plugins #alias for shared plugins
  seek-oss/aws-sm#v0.0.3:
    env:
      NPM_TOKEN: "arn:aws:secretsmanager:ap-southeast-2:xxx:secret:npm/xxx"
  seek-oss/private-npm#v1.1.1:
    env: "NPM_TOKEN"
  docker#v3.0.1:
    volumes:
      - /workdir/node_modules
    environment:
      - "DOCKER_BUILDKIT=1"  
  seek-oss/docker-ecr-cache#v1.1.6:
    dockerfile: Dockerfile-builder
    cache-on:
      - yarn.lock   

steps:
  - label: "yarn :yarn: :eslint: Lint  and :mocha: unit test"
    <<: *dev-agent
    plugins:
      <<: *plugins
    env:
      DOCKER_BUILDKIT: 1  
    command:
      - echo "--- list work dir"
      - ls -al /workdir
      - echo "--- Running yarn test:unit :mocha:"
      - yarn test:unit
    artifact_paths: 
      - reports/junit.xml
      - reports/coverage/**/*
  - wait: ~ #annotate test failures
    continue_on_failure: true
  - plugins:
      - junit-annotate#v1.7.0:
          artifacts: reports/junit.xml 

Docker file

FROM node:11.13-slim

RUN curl -L -o terminal-to-html.gz https://github.com/buildkite/terminal-to-html/releases/download/v3.1.0/terminal-to-html-3.1.0-linux-amd64.gz \
&& gzip -d terminal-to-html.gz \
&& mv terminal-to-html /usr/bin/terminal-to-html \
&& chmod +x /usr/bin/terminal-to-html \
&& mkdir -p /workdir

WORKDIR /workdir

COPY package.json yarn.lock .npmrc ./

RUN yarn install --frozen-lockfile

Context:

Sometimes you may have a caching step ahead of time to ensure that the cache is available to pull in future steps. It is pointless to pull it if we know it exists as it just wastes time. Especially if your steps are ephemeral and wipe the cache for future steps anyway.

--skipPullFromCache or something along those lines would be nice.

I can work on this. This would save on average ~1-1:30 on each of my pipelines

Where I work uses both GCR and ECR (we operate inside China, so ECR there, GCR everywhere else).

Would you be willing to accept a PR that adds GCR support to this excellent looking plugin? I reckon I could do that by roughly

• move the ecr-specific functions from pre-command to a lib/ecr.sh or similar
• introduce a config parameter to specify which *CR to use
• use that config parameter to source "lib/${the_choice}.sh" at the top of pre-command
• introduce a lib/gcr.sh

Hi,

Can I get a brief explanation of how this plugin works?

The doc does not go into details about what is happening.

I see an ecr repo is created on the fly first time.
But what about the logic defining whether the image already exists or not. How does that work? Is it a checksum of your lock file, or something else?

As without any lock file changes I see in the buildkite log # BUILDKITE_PLUGIN_DOCKER_IMAGE changed

Thanks,
Shane.

For reference, here is snippets from my dockerfile and pipeline

FROM mhart/alpine-node:11

RUN apk --no-cache update \
&& apk add --no-cache bash curl wget \
&& curl -L -o terminal-to-html.gz https://github.com/buildkite/terminal-to-html/releases/download/v3.1.0/terminal-to-html-3.1.0-linux-amd64.gz \
&& gzip -d terminal-to-html.gz \
&& mv terminal-to-html /usr/bin/terminal-to-html \
&& chmod +x /usr/bin/terminal-to-html \
&& mkdir -p /app

RUN rm /var/cache/apk/*

WORKDIR /app

COPY package.json yarn.lock .npmrc /app/

RUN yarn

dev-agent: &dev-agent
  agents:
    queue: shane-dev:cicd

plugins: &plugins #alias for shared plugins
  seek-oss/aws-sm#v0.0.3:
    env:
      NPM_TOKEN: "arn:xxx"
  seek-oss/private-npm#v1.1.1:
    env: "NPM_TOKEN"
  docker:
    volumes:
      - /app/node_modules
  seek-oss/docker-ecr-cache#v1.1.3:
    dockerfile: Dockerfile-builder
    cache-on:
      - yarn.lock   

steps:
  - label: "yarn :yarn: :eslint: Lint  and :mocha: unit test"
    <<: *dev-agent
    plugins:
      <<: *plugins
    command:
      - echo "--- Running yarn install :yarn:"
      - yarn
      - echo "--- Running yarn test:unit :mocha:"
      - yarn test:unit
    artifact_paths: 
      - reports/junit.xml
      - reports/coverage/**/*
  - wait: ~ #annotate test failures
    continue_on_failure: true
  - plugins:
      - junit-annotate#v1.6.0:
          artifacts: reports/junit.xml
    
    agents:
          queue: adv-dev:cicd

first of all thanks a lot for this plugin! 🙇

if I understand correctly from reading the Design section of the README, the image is built from scratch on a cache miss, is this correct? What would it take to add the ability of having a tag that can be used to reference it in the --cache-from arg for docker build in the case of cache misses? would you be open to having a PR that adds this functionality?

thanks!

We use Buildkite with Compose so we can run secondary services like redis and postgres. Is it possible to use this plugin with compose-based builds?

Is there a way to use cache-on for directories?

This is a feature request

Enable buildkit to improve performance

https://docs.docker.com/develop/develop-images/build_enhancements/#to-enable-buildkit-builds

I'd like to share a common pipeline between several repositories, where the actual repositories' YAML is aws s3 cp ... - | bash | buildkite-pipeline upload and the script stored in S3 does the work. And to speed up the builds, I'd like to utilise the ECR caching in the common pipeline. However, the repositories won't know any of this and will not have a Dockerfile to enable the plugin to work.

I'd like to be able to specify something like this instead of a Dockerfile:

steps:
  - command: npm test
    plugins:
      - seek-oss/docker-ecr-cache#v1.9.0:
          cache-on:
            - package-lock.json
          base-image: node:10-alpine
          steps:
            - npm install
            - curl large_file ...
            - ...

Another option that would solve this problem is specifying a Dockerfile location in S3.

We've run into an issue where cached images that have expired due to this max_age setting cause a buildkite failure, rather than just rebuilding the image:

The error is:

manifest for XXX not found: manifest unknown: Requested image not found