docker / hub-tool

🧪 Docker Hub experimental CLI tool

🧪 Docker Hub Tool

⚠️ This tool is a Docker experiment to build a Docker Hub CLI tool. The intention of this project is to get user feedback and then to add this functionality to the Docker CLI.

The Docker Hub Tool is a CLI tool for interacting with the Docker Hub. It makes it easy to get information about your images from the terminal and to perform Hub maintenance tasks.

Get started

Prerequisites

• Docker installed on your system
• A Docker Hub account

Install

• Download the latest release for your platform from here
• Extract the package and place the hub-tool binary somewhere in your PATH

OR

• Install from sources: GO111MODULE=on go get github.com/docker/hub-tool

Login to Docker Hub

Login to the Docker Hub using your username and password:

hub-tool login yourusername

Note: When using a personal access token (PAT), not all functionality will be available.

Listing tags

TAG                                   DIGEST                                                                     STATUS    LAST UPDATE    LAST PUSHED    LAST PULLED    SIZE
docker:stable-dind-rootless           sha256:c96432c62569526fc710854c4d8441dae22907119c8987a5e82a2868bd509fd4    stale     3 days ago     3 days                        96.55MB
docker:stable-dind                    sha256:f998921d365053bf7e3f98794f6c23ca44e6809832d78105bc4d2da6bb8521ed    stale     3 days ago     3 days                        274.6MB
docker:rc-git                         sha256:2c4980f5700c775634dd997484834ba0c6f63c5e2384d22c23c067afec8f2596    stale     3 days ago     3 days                        302.6MB
docker:rc-dind-rootless               sha256:ed25cf41ad0d739e26e2416fb97858758f3cfd1c6345a11c2d386bff567e4060    stale     3 days ago     3 days                        103.5MB
docker:rc-dind                        sha256:a1e9f065ea4b31de9aeed07048cf820a64b8637262393b24a4216450da46b7d6    stale     3 days ago     3 days                        288.9MB
docker:rc                             sha256:f8ecea9dc16c9f6471448a78d3e101a3f864be71bfe3b8b27cac6df83f6f0970    stale     3 days ago     3 days                        270.9MB
...
25/957 listed, use --all flag to show all

Contributing

Docker wants to work with the community to make a tool that is useful and to ensure that its UX is good. Remember that this is an experiment with the goal of incorporating the learnings into the Docker CLI so it has some rough edges and it's not meant to be a final product.

Feedback

Please leave your feedback in the issue tracker! We'd love to know how you're using this tool and what features you'd like to see us add.

Code

At this stage of the project, we're mostly looking for feedback. We will accept pull requests but these should be limited to minor improvements and fixes. Anything larger should first be discussed as an issue. If you spot a bug or see a typo, please feel free to fix it by putting up a pull request!

Building

Prerequisites

• Docker
• make

Compiling

To build for your current platform, simply run make and the tool will be output into the ./bin directory:

$ make
docker build --build-arg GO_VERSION=1.16.3 --build-arg ALPINE_VERSION=3.12.0 --build-arg GOLANGCI_LINT_VERSION=v1.31.0-alpine --build-arg TAG_NAME= --build-arg GOTESTSUM_VERSION=0.5.2 --build-arg BINARY_NAME=hub-tool --build-arg BINARY=hub-tool . \
                --output type=local,dest=./bin \
                --platform local \
                --target hub
[+] Building 3.7s (6/13)
...
 => => copying files 22.10MB

 $ ls bin/
 hub-tool

hub-tool's Issues

why would repo ls not always do --all?

What is the use case for a truncated list of repos? I always want the --all option applied.

Grafana dashboard

Put in place a Grafana dashboard to monitor hub-tool calls to the hub APIs, using the specific User-Agent for discrimination.

Ensure that we provide a useful message on getting rate limited

Description
The Hub API is rate limited so it's possible that a user of the tool will get a 429 response if they're using it too heavily. We should make sure that the tool returns a useful error in this case.

token create --format json doesnt display anything

hub-tool token create --description "tmp2" --format json
{}

Add login command

hub-tool enforces hub login, but there's no way to do it just using hub-tool, it relies on docker login command. We should add a hub-tool login command.

Remove hub staging URL from the git history

Add search command

We could use the Hub search APIs, which are slightly better than docker search, and maybe figure a way to search against official images?

^C outputs usage

If you ^C an operation it prints out the usage.

hub-tool org ls
^CError: Get "https://hub.docker.com/v2/orgs/docker/groups/automatedreadwrite/members/": context canceled
Usage:
  hub-tool org ls [flags]

Flags:
      --format string   Print values using a custom format ("json")
  -h, --help            help for ls

Better pagination UX

Some commands (ie: tag ls and repo ls) use paginated hub APIs. Right now it prints only the first page, and rely on a --all flag to let user print all the pages.

We should at least print a message to let the user know there's more items to show.
Another way could be something more interactive a-la-git, piping each page to LESS.

Replace EXPIRES column

Description
Replace EXPIRES column output as part of hub-tool tag ls with something more relevant based on API.

Better help message

~/Downloads> ./hub-tool_darwin_amd64 help                                                                                                        20:23:14
A tool to manage your Docker Hub images
Usage:
  ./hub-tool_darwin_amd64 [flags]
  ./hub-tool_darwin_amd64 [command]
Available Commands:
  account
  help        Help about any command
  org
  repo
  tag
  token
Flags:
  -h, --help      help for ./hub-tool_darwin_amd64
      --version   Display version of the scan plugin
Use "./hub-tool_darwin_amd64 [command] --help" for more information about a command.

The hub-tool doesn't print the descriptions for the manage commands.

Also add better description to the token help message (@shawnaxsom feedback):

It's good that it says Personal Access Token now, but it could be useful to mention what PATs are used for (e.g. you can't login to the UI with them, use them like passwords when using docker CLI or cURL)

Also need to configure cobra the same way we do with docker cli

• Don't print usage on error
• Don't print [flags] on usage command hub-tool token create [OPTIONS] [flags]

token inspect 404 message

should just say "invalid token"

hub-tool token inspect bc966540-5ef5-4c82-93fa-ef68fa6cbaee
Error: bad status code "404 Not Found"
Usage:
  hub-tool token inspect [OPTIONS] TOKEN_UUID [flags]

Flags:
      --format string   Print values using a custom format ("json")
  -h, --help            help for inspect

tag rm on non existent repo not helpful

hub-tool tag rm justincormackfghsdfg/fhjsgfjshda:latest
Please type the name of your repository to confirm deletion: justincormackfghsdfg/fhjsgfjshda
justincormackfghsdfg/fhjsgfjshda
Error: bad status code "404 NOT FOUND"
Usage:
  hub-tool tag rm [OPTIONS] REPOSITORY:TAG [flags]

Flags:
      --force   Force deletion of the tag
  -h, --help    help for rm

should not prompt for repo name if it doesnt exist.
should not print usage
should report proper error

help text unhelpful

top level help text should explain what account, org, repo, tag, token are about, just blank now

Available Commands:
  account     
  help        Help about any command
  org         
  repo        
  tag         
  token       

Cut a release

• Write a changelog
• Advertise the release to the desktop team

provide API

I think we should provide API/SDK support for these operations too. cc @chris-crone

Add color to the list output

uuid for tokens should probably not be displayed

We don't display this anywhere else, so it is probably confusing, people might think it is the token value.

tag ls on non existant repo doesnt give useful error

hub-tool tag ls justincormack/fhjsgfjshda
TAG                 DIGEST              STATUS              EXPIRES             LAST UPDATE         LAST PUSHED         LAST PULLED         SIZE

this repo doesn't exist it should give an error

Prepare SDK

• Move the hub client code outside internal package
• Add some client level testing
• Add some golang documentation

Update all the headers

Source headers should mention Docker Hub tool authors

PR #32

Write a decent README.md

Prepare licensing

• Add Docker EULA

Account Info returns 403?

A couple complaints here..

First, it is confusing that I cannot get my account information, but I can list my orgs? Am I logged in or not?

⛵  docker-desktop in ~
❯ hub-tool account info
Error: bad status code "403 Forbidden"
Usage:
  hub-tool account info [OPTIONS] [flags]

Flags:
      --format string   Print values using a custom format ("json")
  -h, --help            help for info


⛵  docker-desktop in ~
❯ hub-tool org ls
NAMESPACE           NAME                MY ROLE             TEAMS               MEMBERS
dckr                                    Owner               4                   10
dockerinc                               Owner               2                   3
ratelimitalways                         Owner               1                   2
ratelimitnever                          Owner               1                   2
ratelimitpreview                        Owner               1                   2

Secondly, the error from hub-tool account info is murky. It has the error, and then lists the help. The help message obscures the error message and requires me to mentally parse it a big more. Maybe we can have a smaller help message recommending to run hub-tool <cmd> help instead? I think the error is the most important thing return.

Lastly, maybe for 403 we return a more user friendly error like "Not authorized, Please try logging in with if you have not already". I think the Docker cli does something similar if you get a 403 from pulling an image.

Add Call To Action

As a hub tool but non-paid hub user,
So I know where I can sign up on Hub to retain my tags,
I want hub tool to prompt a message and a URL when I list my tags.

PR #28

Control+C not handled by prompt in `hub-tool tag rm`

Description

./bin/hub-tool tag rm ccrone/alpine:latest
Please type the name of your tag to confirm deletion: ccrone/alpine:latest
^C

Control + D works though.

Stabilize code

• Write unit tests
• Write e2e tests

Wrong help for the `--all` flag

When I do hub-tool repo ls --help the help for the --all flag says Fetch all available tags, it should be Fetch all available repositories

403 handling

if you log in with a token not a password, various operations fail with 403. There should probably be a message explaining that your token is not scoped for this operation.

eg token creation is forbidden with a token.

hub-tool token create --description "should fail"
Error: bad status code "403 Forbidden"

account info is also forbidden

something is forbidden in tag ls but it appears to log error and continue ok

hub-tool tag ls justincormack/zfs-kmod 
bad status code "403 Forbidden"TAG                                                                             DIGEST                                                                    STATUS              EXPIRES             LAST UPDATE         LAST PUSHED         LAST PULLED         SIZE
justincormack/zfs-kmod:4.9.105           ...

Alias for `--force`

I would love to be able to do hub-tool repo rm -f rumpl/voting-app and not hub-tool repo rm --force rumpl/voting-app

Rather raw install process.

On osx, has to chmod the binary.
Maybe zipping/tarballing could help a bit here.

Sign binary with Docker, Inc certificate.

on OSX

1. Download from release page
2. run it from terminal gets

Had to manually dance around to fix the quarantine

Build binaries for ARM

The release should also target ARM binaries.

tag rm should ask about latest?

every other docker operation assumes "latest" but tag rm just says "Error: invalid reference: tag must be specified" if you rm without a tag. Maybe it should ask "did you mean tag:latest"?

Resolve misunderstanding between Token itself and its UUID

hub-tool token subcommands needs the user to provide a token UUID for update inspect and revoke.
User may not understand it needs the UUID and not the token, as they have the same form (both UUID). The result is that the token is not found, misleading the user.
Maybe we should add a name field too, generated the same way we do with containers, and only prompt those in the CLI? The backend will need to check their unicity though.

Enforce image reference is a hub image

All commands that take an image reference as parameter (ex: tag inspect, tag delete...) must check the image reference is a hub one and not pointing to a custom registry. Prompt a nice error message in that case.

Show warning and run login if the user never logged-in

See this comment

Enforce user login to the hub

Make sure the user is logged in, and if not prompt a nice CTA.

Print annotations while inspecting a manifest/manifest list

An ocispec Manifest or Manifest list can embed annotations, the tag inspect command should print it.

PR #39

token create wording

"WARNING: This access token will only be displayed once." is weird, it should be "This access token cannot be displayed again"

Improve install instructions

Description
The install instructions in the README are currently very basic. Update #69 or be inspired by it to improve the instructions now that packages have changed.

Sign binaries

Description
macOS binaries are currently not signed or notorized which means that they can't be run without removing the quarantine xattr. We should ensure that we are shipping signed binaries for macOS and Windows.

Flakey inspect test

Found on the CI:

=== Failed
make: *** [test-unit] Error 1
=== FAIL: internal/commands/tag TestPrintImage (0.00s)
    inspect_test.go:105: assertion failed: 
        --- expected
        +++ actual
        @@ -5,6 +5,6 @@
         ··Platform:··os/arch/variant/osversion/feature1/feature2
         ··Annotations:
        +····annotation2:·value2
         ····annotation1:·value1
        -····annotation2:·value2
         ··Os/Arch:·······os/arch
         ··Author:········author

token update --set-active=false is quite confusing

maybe we could just have "activate" and "deactivate" flags? Not every user understands Go's CLI conventions for true and false, they are not common in other languages.

help text wrong

says "Display version of the scan plugin" in the output...

why are tokens revoke not rm?

It just deletes them, I don't understand why we use a special work.

Add custom user agent to talk to the hub API

PR #25

org ls very slow

Takes over a minute to run (for me, I am in 14 orgs if that makes a difference).

tag inspect --format json has empty object for name

hub-tool tag inspect --format json justincormack/nsenter1
{
  "Name": {},
  "Manifest": {
    "schemaVersion": 2,
    "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "digest": "sha256:c81481184b1b001f225769d1482b360b4be18fe7dd53fa93cdbd45d568e905a7",
      "size": 1552
    },

the "Name" field should have the tag name I think?

tag rm asks for repo name for confirmation

why is this repo name not tag name?