kvaps / kubectl-build

Build dockerfiles directly in your Kubernetes cluster.

Kubectl build

Kubectl build mimics the kaniko executor, but performs building on your Kubernetes cluster side.
This allows you to simply build your local dockerfiles remotely without leaving your cozy environment.

Installation

using krew:

kubectl krew index add kvaps https://github.com/kvaps/krew-index
kubectl krew install kvaps/build

or using curl:

curl -LO https://github.com/kvaps/kubectl-build/raw/master/kubectl-build
chmod +x ./kubectl-build
sudo mv ./kubectl-build /usr/local/bin/kubectl-build

Usage

kubectl build [args]

Examples

# Show all kaniko commands
kubectl build --help

# Build from current directory
kubectl build --context . --no-push

# Specify namespace and kubeconfig
kubectl build --context . --no-push --namespace default --kubeconfig ~/.kube/someconfig

# Login to remote registry
docker login docker.io

# Short form
kubectl build -c . -d docker.io/some/image:latest

# Run debug shell
kubectl build -c . --no-push --debug

# Use cache building
kubectl build --context . --destination docker.io/some/image:latest --cache --cache-repo docker.io/some/cache

# Save image name and digest to file
kubectl build --context . --destination docker.io/some/image:latest --digest-file /tmp/digest --image-name-with-digest-file /tmp/image

# Build from stdin
tar -cvf- . | kubectl build --destination docker.io/some/image:latest --context tar://stdin

Extra configuration

While standard behavior of kubectl-build plugin intend to repeat kaniko executor options. The additional configuration can be specified by setting environment variables.

This may be useful for both having permanent configuration and setting CI-systems.

Enivroment Variable	Description	Default value
KUBECTL_BUILD_CONTEXT	Kubernetes context for creating pod (may be overriden by --kubecontext)	current context
KUBECTL_BUILD_DOCKER_CONFIG	Path to dockerconfig file to forward	~/.docker/config.json
KUBECTL_BUILD_IMAGE	Kaniko-executor image	gcr.io/kaniko-project/executor
KUBECTL_BUILD_KEEP_POD	If set to true do not delete pod after finising process	false
KUBECTL_BUILD_KUBECONFIG	Path to kubeconfig file for creating pods (may be overriden by --kubeconfig)	kubectl defaults
KUBECTL_BUILD_METADATA_OVERRIDES	Json patch to override metadata for creating pods	{}
KUBECTL_BUILD_NAME_OVERRIDE	Name for creating pod	kaniko-rand6n
KUBECTL_BUILD_NAMESPACE	Kubernetes namespace for creating pod (may be overriden by --namespace)	current namespace

kubectl-build's Issues

Feat: Add option to specify Dockerfile

Add a --dockerfile=Dockerfile flag to maps kaniko's --dockerfile argument.

Sorry of the numerous issues. I hope to use this and to be able to recommend it to my coworkers. I just want to document the missing bits.

Feat: Add option to specify resources and tolerations

I don't have a good idea of what that would look like and it may well be out of scope, but my build pods are getting evicted. It would be nice to be able to pass cpu, mem and toleration.

In case others are interested, for now I have hardcoded what I needed in `kubectl-build with:

{
  "spec": {
    "hostPID": true,
    "hostNetwork": true,
    "containers": [
      {
        "securityContext": {
          "privileged": true
        },
        "image": "$image",
        "name": "kaniko",
        "stdin": true,
        "stdinOnce": true,
        "terminationMessagePath": "/dev/termination-log",
        "terminationMessagePolicy": "File",
        "args": $args,
        "volumeMounts": $volumemounts,
        "resources": {
          "requests":{
            "memory": "10Gi",
            "cpu": "8000m"
          },"limits":{
            "memory": "10Gi",
            "cpu": "8000m"
          }
        }
      }
    ],
    "tolerations":[{
      "key": "dedicated/highcpu",
      "operator": "Exists"
    }],
    "volumes": $volumes
  }
}

Don't send ~/.docker/config.json by default

The current implementation sends ~/.docker/config.json as a secret if it exist. This isn't great:

1. it's surprising, not mentioned anywhere and a security concern;
2. it's left behind if interrupted (e.g. ctrl-c is pressed);
3. the credential might not be the ones you want to use in the cluster (e.g. a service account with less privilege);
4. sending this config file it often not enough (e.g. in GKE grc.io auth depends on the gcloud command and other credential files);
5. it's often unnecessary (e.g. pushing to gcr.io from GKE doesn't require any secret if the default cluster service account is configured with gcs write permission)

As a result, I think it would be best to place this pushing of ~/.docker/config.json behind a dedicated flag and default to not sending any secrets.

Secrets are not created in the correct namespace

Tar error on macos

On macos I get:

tar: Option --record-size=100K is not supported
Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help

❯ tar --version
bsdtar 3.3.2 - libarchive 3.3.2 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.6

None of the flags in the docs actually exist?

HI - I wanted to cover this on Coffee and Cloud Native today, because I had Kaniko on the show last week. I ran through it last night, and it seems that none of the samples in your README actually work. I looked in the script, and the flags aren't even there. Is there some prerequisite that's missing from the installation guide, or was the README made from a different branch/version?

root@training-a:/var/tmp/build# kubectl build --context . --no-push
/usr/local/bin/kubectl-build: 72: /usr/local/bin/kubectl-build: args+="--no-push", : not found

root@training-a:/var/tmp/build# kubectl build --help
/usr/local/bin/kubectl-build: 72: /usr/local/bin/kubectl-build: args+="--help", : not found

root@training-a:/var/tmp/build# kubectl build -c . -d docker.io/monachus/build-test:latest
/usr/local/bin/kubectl-build: 72: /usr/local/bin/kubectl-build: args+="-d", : not found

error: Invalid JSON PatchKaniko

kubectl build -c . -d nexus.domain.com:445/portal-test:692 --namespace default

Output:

spawning "kaniko-drlmgw"
error: Invalid JSON PatchKaniko 102400 (100KiB, 63MiB/s)
tar: -: Записано только 94208 байт из 102400                                                                                                                                                  
tar: Child returned status 141                                                                                                                                                                
tar: Error is not recoverable: exiting now

With set x

+ tar --version
+ awk {print $1; exit}
+ m=tar
+ [ tar != bsdtar ]
+ [ -f /home/user/.docker/config.json ]
+ tar -P --transform s|^/home/user/.docker/config.json|../.docker/config.json| --record-size=100K --checkpoint=1 --checkpoint-action=ttyout=Sending build context to Kaniko %{r}T\r --exclude-ignore=.dockerignore -C . -czf - . /home/user/.docker/config.json
error: Invalid JSON PatchKaniko 102400 (100KiB, 48MiB/s)
+ ec=1
+ [ false = true ]
+ exit 1

Feat: Don't tar dockerignored files

GNU tar has the --exclude-ignore. Perhaps it would make sense to add --exclude-ignore=.dockerignore to avoid needless transfers.

Trap tries to delete the secret even if it wasn't created

At the very end, the following line

trap "EC=\$?; kubectl delete "secret/$name" "pod/$name" --wait=false $kubecontext $kubeconfig $namespace 2>/dev/null; exit \$EC" EXIT INT TERM

tries to delete the secret even if it wasn't created (e.g. if ~/.docker/config.json didn't exist). This cause the script to return a non-zero exit code.