Tag limit on GitHub container repos about image-reflector-controller

[speedst3r]

It looks like it's resolved, I am seeing over 1000 tags returned for the repositories I have configured.

$ flux get image repository tautulli
NAME            READY   MESSAGE                                 LAST SCAN                       SUSPENDED
tautulli        True    successful scan, found 1436 tags        2021-09-24T16:19:18+08:00       False

[speedst3r]

Just realised you might want the repository configuration too

apiVersion: image.toolkit.fluxcd.io/v1alpha2
kind: ImageRepository
metadata:
  name: tautulli
  namespace: flux-system
spec:
  image: ghcr.io/linuxserver/tautulli
  interval: 1h0m0s
  secretRef:
    name: registry-ghcr-speedst3r

[bigkevmcd]

It looks like it's because https://github.com/google/go-containerregistry/blob/main/pkg/v1/remote/list.go#L52 is limited to 1000

Some experimenting indicates that GHCR isn't returning Link: ... pagination links, no matter how many you ask for per https://docs.docker.com/registry/spec/api/#listing-image-tags

A small test dumping the response headers.

Fetching 100 tags...

2021/05/10 07:57:32 Fetching tags with https://ghcr.io/v2/linuxserver/tautulli/tags/list?n=100
2021/05/10 07:57:32 Header Docker-Distribution-Api-Version = [registry/2.0]
2021/05/10 07:57:32 Header Date = [Mon, 10 May 2021 06:57:32 GMT]
2021/05/10 07:57:32 Header X-Github-Request-Id = [EF03:3CE9:13E558:1B57B9:6098D95B]
2021/05/10 07:57:32 Header Content-Type = [application/json]
2021/05/10 07:57:32 Response included 100 tags

Fetching 1000 tags

2021/05/10 07:58:36 Fetching tags with https://ghcr.io/v2/linuxserver/tautulli/tags/list?n=1000
2021/05/10 07:58:36 Header Content-Type = [application/json]
2021/05/10 07:58:36 Header Docker-Distribution-Api-Version = [registry/2.0]
2021/05/10 07:58:36 Header Date = [Mon, 10 May 2021 06:58:36 GMT]
2021/05/10 07:58:36 Header X-Github-Request-Id = [EF0D:9699:35114C:41186A:6098D99C]
2021/05/10 07:58:36 Response included 1000 tags

Fetching 1500 tags

2021/05/10 07:59:12 Fetching tags with https://ghcr.io/v2/linuxserver/tautulli/tags/list?n=1500
2021/05/10 07:59:13 Header Content-Type = [application/json]
2021/05/10 07:59:13 Header Docker-Distribution-Api-Version = [registry/2.0]
2021/05/10 07:59:13 Header Date = [Mon, 10 May 2021 06:59:13 GMT]
2021/05/10 07:59:13 Header X-Github-Request-Id = [EF17:6042:3A9F95:466F85:6098D9C0]
2021/05/10 07:59:13 Response included 1088 tags

[speedst3r]

And it looks like because GitHub container registry doesn't have that limitation, when you pass the n= argument to the API URL it doesn't return links to the next page.

Looks like it's waiting on distribution/distribution#3143

[kingdonb]

I see that distribution/distribution#3143 was merged about 15 days ago. Does this mean there can be progress on this issue, or can it be closed? (I think it most likely still blocks on GCHR taking action, I do not know if we have any contact who works for GitHub and can keep us informed on their progress over at GitHub Container Registry, but I will check my Twitter...)