tlwr / registry-tag-resource

a concourse resource for oci registry tags

registry-tag-resource

a concourse resource type for tags in a oci registry

a blog post on www.toby.codes describes how this resource type can be used

public image

all passing commits are built into container images hosted on github

operations

• check - looks for tags
• in - downloads metadata about the tag

configuration

source:
  # where the image lives
  #
  # for docker hub https://hub.docker.com/v2/user_or_org/image_name
  #
  # for example https://hub.docker.com/v2/library/ruby for _/ruby
  # for example https://quay.io/v2/coreos/etcd'
  #
  # mandatory
  uri: https://hub.docker.com/v2/repositories/governmentpaas/cf-cli

  # basic auth
  #
  # only works for registries, does not work for docker hub
  #
  # optional
  username: my-registry-username
  password: my-secret-password

  # how many pages to check in the registry
  #
  # optional ; default 1
  pages: 1

  # how many tags to fetch per page
  #
  # optional ; default 25
  tags_per_page: 50

  # ruby regular expression for filtering tags
  #
  # optional
  regexp: 'v[0-9]+'

  # to specify semantic versions
  # see github.com/jlindsey/semantic
  #
  # optional
  semver:
    # mandatory
    matcher: '~1.5'

    # optional
    prefix: 'v'
    

  # to sort by other methods

  # optional
  sort:
    # exclusive with semver.matcher
    #
    # alphabetical or numerical
    #
    # optional
    method: alphabetical

    # for example, can be used to retrieve oldest tag as most recent version
    #
    # optional
    reverse: true

check - check for new tags

the check step looks at the configured registry for new tags for the image

an example version:

{
  "tag": "2.7.1"
}

in - fetch registry image metadata

produces the following files:

• tag
• digest

examples

dynamically generate docker image using build args

using a Dockerfile with a build arg:

BUILD_ARG ruby_version
FROM ruby:$ruby_version

...

a pipeline that uses the oci-build-task can dynamically build docker images using BUILD-ARG_ params:

name: dynamically-build-image
plan:

  # source code
  - get: my-src 

  # registry-tag
  - get: ruby-img-tag
  
  # make tag file from resource available as a variable
  - load_var: ruby-version
    file: ruby-img-tag/tag

  - task: build-img
    privileged: true
    config:
      platform: linux

      image_resource:
        type: registry-image
        source:
          repository: vito/oci-build-task

      inputs:
        - name: my-src
          path: .

      outputs:
        - name: image

      params:
        # load ruby version for build task
        BUILD_ARG_ruby_version: ((.:ruby-version))

      run:
        path: build

registry-tag-resource's Issues

Resource filters out valid semver version when prefixed

Trying to pull minio/console releases with the following config and I get nothing

- name: minio-console-release
  type: registry-tag
  check_every: 24h
  icon: tag
  source:
    uri: https://hub.docker.com/v2/repositories/minio/console
    regexp: ^v\d+\.\d+\.\d+$
    pages: 10
    tags_per_page: 25
    semver:
      prefix: v
      matcher: '>= 0.8.0'

Looking at the code (I'm no Ruby expert) I think the problem is here:

registry-tag-resource/bin/check

Line 68 in 85b5ad8

v = Semantic::Version.new(t.sub(/#{semver_prefix}/, ''))

Probably should have a # when comparing with line

registry-tag-resource/bin/check

Line 78 in 85b5ad8

tags = tags.sort_by { |t| Gem::Version.new(t.sub(/semver_prefix/, '')) }

Resource incorrectly sorts versions

We use this resource to fetch new versions of the minio/minio docker image, we use the following resource configuration:

- name: minio-release
  type: registry-tag
  check_every: 24h
  icon: tag
  source:
    uri: https://hub.docker.com/v2/repositories/minio/minio
    regexp: ^RELEASE\.\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}Z$
    pages: 25
    tags_per_page: 25

However it currently fetched the releases in this order (only the last 5 shown)

• RELEASE.2021-07-08T01-15-01Z
• RELEASE.2021-07-15T22-27-34Z
• RELEASE.2021-07-12T02-44-53Z
• RELEASE-2021-07-08T19-43-25Z
• RELEASE-2021-06-17T00-10-46Z

I looked at the code and the resource doesn't seem to apply any sorting in the request or after fetching the tag list. Sorting is only applied if using semver (which we can't in our case).

Should the list_tags query docker hub with a sort parameter or could the resource have a parameter (when not semver) to request sorting by name?

Allow login information for private repository

Great resource by the way, I saw your post on the concourse discord server and it was posted exactly at the time I was thinking about something like this.

I've started using it for updating some of our images using docker hub source images however some of our other images are stored in our private docker registry hosted on our Nexus repo and we don't allow un-authenticated request.

Is it possible to update the resource to allow user/password to be specified and used?

Question: is setting number of pages and page size something that should be configurable or should all be always fetched

We've run into an issue today where with the values we had set for pages and tags_per_pages it wasn't enough to fetch new tags. Doesn't look like the API returns the data in any order so this means we basically have to tweak this parameters over time or set them initially quite large.

Should these even be parameters, should the check resource keep fetching more pages until there is no more?