Currently, only pre-push and pre-receive hooks are supported. This should be extended to other hooks like post-push, etc.

Hi

Gitkube is a new project. We need active feedback. We also want to know who is using gitkube, where it's being used - side projects, inside organizations, some place else?

Please drop in a comment if you are using gitkube and provide any additional details. This will be very helpful!

Summary

In layman's terms: It should be able to allow developers to git push to a gitkube-enabled cluster without a Dockerfile - or so called the "Heroku-style" deployment.

Intro

Heroku's architecture is composed of a dedicated builder for the source, where your project requires a Procfile to determine processes to execute, and a declarative app manifest that includes all the required dependencies and postinstall scripts that the builder will run.

Heroku builds an app using an approach called a Stack-Buildpack approach, whereas:

• use a Base image that has all the dev dependencies.
• use a Buildpack: a set of shell scripts that would automate the building and releasing of the code
• trim dev dependencies from the container and release to the runtime environment.

But in the case for gitkube, we do not need to implement the same flow as Heroku since we can take advantage of the rich Docker ecosystem in Kubernetes.

Process flow

Once the remote recieves the source from the push, it should be able to detect the dockerfile and the manifest, but we should look on two situations:

• Situation 1: Dockerfile found and manifest found.
  • Use the current flow to dpeloy app.
• Situation 2: Dockerfile not found but manifest found
  • Try to find manifest (package.json, pom.xml, etc.).
  • Grab image that works for the specific app (package.json is found for example, then use nodejs:latest)
  • then run npm install, and deploy app.
  • run code using npm start or find app.js, index.js, etc.
    • if not found, declare an error and call a back off.

Conclusion

This is not intended to replace how gitkube currently works, rather, it is a enhancement on the current flow, and it allows migrating users from Heroku to their own provisioned Kubernetes cluster.

Feel free to extend this as well, as I only accounted for cases on migrating users from heroku.

ECR currently does not automatically create repositories on push unlike docker hub. How (where) should we handle this logic ?

Tests should not affect cluster level resources.

First push of gitkube-example on to my local cluster went through fine so I thought of pushing an update and realised that I need to setup the registry to get it working. So, I created a kube secret, with name regsecret, for docker following the instructions on the link in the README of gitkube-example and updated the registry of the example remote to be:

  registry:
    url: "docker.io/sdileep"
    credentials:
      secretKeyRef:
        name: regsecret
        key: .dockercfg

Now when I try to push to the example remote, I get the following error:

remote: WARNING: Error loading config file:/home/default-example/.dockercfg - Invalid Auth config file

So, I updated the registry to be:

  registry:
    url: "docker.io/sdileep"
    credentials:
      secretKeyRef:
        name: regsecret
        key: .dockerconfigjson

.dockerconfigjson field of the secret is the base64 representation of docker credentials - I presumed this is what goes in here.

Now when I try to push to the example remote, I get a timeout. Not sure what the correct config here should be, could you please help? Thanks.

Add instructions on how Docker secret need to be configured in the k8s cluster:

kubectl create secret docker-registry gitkube-secret \
  --docker-server=https://index.docker.io/v1/ \
  --docker-username=arungupta \
  --docker-password=<password> \
[email protected]

Environment ?

minikube version
minikube version: v0.26.1

$ gitkube version
v0.2.0

What did I do?

gitkube install

What I Expected to see?

Gitkube running peacefully

What did I see instead?

$ k get po -n kube-system 
NAME                                    READY     STATUS              RESTARTS   AGE
default-http-backend-ddb2s              1/1       Running             2          2d
etcd-minikube                           1/1       Running             0          26m
gitkube-controller-cc586fcb-zfk8b       0/1       ImageInspectError   0          23m
gitkubed-6dc69968c5-zwqg5               1/1       Running             0          23m
kube-addon-manager-minikube             1/1       Running             2          2d
kube-apiserver-minikube                 1/1       Running             0          26m
kube-controller-manager-minikube        1/1       Running             0          26m
kube-dns-86f4d74b45-9qfxb               3/3       Running             7          2d
kube-proxy-kjncs                        1/1       Running             0          25m
kube-scheduler-minikube                 1/1       Running             2          2d
kubernetes-dashboard-5498ccf677-cxhrm   1/1       Running             6          2d
nginx-ingress-controller-csqzz          1/1       Running             4          2d
storage-provisioner                     1/1       Running             6          2d
tiller-deploy-5f789bd9f7-zmt8g          1/1       Running             2          2d

$ k describe po gitkube-controller-cc586fcb-zfk8b -n kube-system
....
....
Events:
  Type     Reason                 Age                From               Message
  ----     ------                 ----               ----               -------
  Normal   Scheduled              21m                default-scheduler  Successfully assigned gitkube-controller-cc586fcb-zfk8b to minikube
  Normal   SuccessfulMountVolume  21m                kubelet, minikube  MountVolume.SetUp succeeded for volume "gitkube-token-klm4g"
  Warning  Failed                 6m (x71 over 21m)  kubelet, minikube  Error: ImageInspectError
  Warning  InspectFailed          1m (x93 over 21m)  kubelet, minikube  Failed to inspect image "hasura/gitkube-controller:v0.2.0": rpc error: code = Unknown desc = Error response from daemon: readlink /var/lib/docker/overlay2: invalid argument
`

Its useful, but there are some parts that move very quickly. Perhaps its worth remaking/editing it to run a little slower so its easier to take in each of the steps.

Add docs/minikube.md for instructions specific to minikube

Hey

Just wondering what the best workflow is for when i need to update a deployment without wiping out the image? Right now i need to call:

kubectl get deployment app -o yaml

Copy the whole image into into my deployment.yaml file, and then i can do kubectl apply

Basically my question is: is there a better way to handle all of this, and to make it play nice with gitkube?

Mention specifically what kubernetes distros this works with. I've received lots of questions about whether it works on GKE or not.

Because it errors out on creating an already existing docker-registry secret.

FATA[0036] generating remote failed: error creating docker-registry secret: secrets "regsecret" already exists

Gitkube runs on most Kubernetes distributions/vendors as per the install instructions in README. But in some cases, there may be more configuration required to correctly setup for e.g. exposing the gitkubed service may require configuring a Loadbalancer manually.

We need to document the instructions (docs/install.md) covering various vendors like GKE, Rancher, kops based installations, etc. In most cases, the instructions should be default instructions as provided in the README and in other cases the additional steps should be mentioned.

Please use this issue to mention any distribution which is not covered by the installation instructions.

Now that 0.2.0 is out, how do people upgrade?

I just uninstalled it and installed again, but that resulted in my LB ip changing.

Push master
Push a test branch
Best way to use this with different kinds of git patterns: https://www.atlassian.com/git/tutorials/comparing-workflows

Add an option that allows to setup a local Docker registry in the cluster. This allows a simpler workflow OOTB.

Can we add support for docker-compose via docker stack?
That way we would have:

1. Kubernetes manifests
2. Helm
3. Docker-compose

1, 3 are ideal for developers.

docker stack deploy is pretty much exactly like helm install. Uses a compose.yaml instead of Helm charts.
https://docs.docker.com/engine/reference/commandline/stack_deploy/#description

Is this support for AWS EKS service?

gitkubed pods are unschedulable on a low config node on GKE (g1-small: 1 vCPU, 1.70 GB ).

Works like a charm on the next available config. Add information about the min. node config needed.

Currently, pre-receive hook is fixed: https://github.com/hasura/gitkube/blob/master/build/gitkubed/pre_receive.sh

We want to make it scriptable to enable : #20

Each hook probably needs variables from the remote like DEPLOYMENTS, REGISTRY_PREFIX.
For pre-receive, we are currently doing a moustache render during startup.

For giving same variables to each hook, we need to run a moustache render on each hook and then place it in the correct directory.

As mentioned in #71, sometimes a user might need to update the deployment (say kubectl apply) and when it happens, the image name is overwritten.

The controller needs to keep track of latest build tags for deployments in each remote.

When any of these deployments' image field gets updated, controller can patch it to the latest build tag.

So, whenever a new build happends, a relevent key should be updated in Remote.Status. Then controller can watch all the tracked deployments and patch when required.

Makes it easier to copy

It will be awesome of gitkube can integrate with something like img so that images can be built without mounting the docker sock.

ref:
https://github.com/genuinetools/img
https://blog.jessfraz.com/post/building-container-images-securely-on-kubernetes/

There are still some issue to be merged/closed for this to happen, mentioned in the blog above ^

Show how gitkube workflow works with Docker for Desktop

Currently, gitkube deploys to the same cluster where it is running. We should provide support for deploying it to a different cluster.

Currently, sync happens on remotes.

For consistency, we should sync gitkubed (service and deployment) also

Hi,

It seems like naming a remote is subject to certain limitations as groupadd is executed (potentially 32chars?).

Example

apiVersion: gitkube.sh/v1alpha1
kind: Remote
metadata:
  name: app-remote
  namespace: my-really-really-really-long-ns

k logs po/gitkubed
Setting up git remotes
Configuring : my-really-really-really-long-ns-app-remote
Adding user `my-really-really-really-long-ns-app-remote' ...
Adding new group `my-really-really-really-long-ns-app-remote' (1000) ...
groupadd: 'my-really-really-really-long-ns-app-remote' is not a valid group name
adduser: `/usr/sbin/groupadd -g 1000 my-really-really-really-long-ns-app-remote' returned error code 3. Exiting.

Is there any way to overwrite the computed my-really-really-really-long-ns-app-remote identifier so groupadd can run through?

Add docs/registry.md with instructions on how to create docker-secret for different registry providers like dockerhub, gcr, quay, etc

Features:

• Public key based auth
• No assumptions about directory structure (configure dockerfile path)
• Rollout deployments in any namespace
• Easy to fork and customise
• Super easy to use and configure

https://kubernetes.io/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#validation

Tests can use gitkube cli to install and uninstall as well as manage remotes.

I am getting this error in AKS but not in GKE.

Error: forwarding ports: error upgrading connection: Upgrade request required

Opened issue with upstream: helm/helm#4254

I'm trying to experiment with gitkube on a local cluster(not minikube) and hence exposed it as NodePort type service. After creating a remote as per example, I am not able to retrieve the remote URL using $ kubectl get remote example -o json | jq -r '.status.remoteUrl. Am I missing something?

ps: the example worked on GCE without any issues.

start_sshd.sh script needs some modifications due to the difference in ownership of docker.sock on Docker for Mac.

State of Gitkube on Docker for Windows in unknown. Any help in identifying the status is appreciated.

This will support credentials plugin (useful for EKS authentication):

https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins

Hi,

I'm new to gitkube. I'm trying to find a way to build docker images and push it to private registry only, without deploying pods in k8s cluster. Is there an option for that or is it even possible?

Currently, the e2e test fails when the following two resources already exist:

Error from server (AlreadyExists): error when creating "STDIN": customresourcedefinitions.apiextensions.k8s.io "remotes.gitkube.sh" already exists
Error from server (AlreadyExists): error when creating "STDIN": clusterrolebindings.rbac.authorization.k8s.io "gitkube" already exists

For the CRD, we just need to ignore the error and for clusterrolebindings we can just use the test namespace name.

To handle multi-repo configurations, prompt if deployments are present or not.

Support for git push based deployments is currently limited to deployments. This should be extended to:

• daemonset
• statefulset

Package Gitkube as Helm chart

Current pre-receive hook has Hasura specific instructions: https://github.com/hasura/gitkube/blob/master/sshd/pre_receive.sh#L185

Requirement

• Given a the deployment model, specify a Helm Application to be deployed.
• "Any" application from https://hub.kubeapps.com/ should be deployed automatically.

Design

• Specify the name of the Helm Chart to deploy
• Identify the type of ingress instance the cluster contains
  • Use default ingress nginx
  • This provides a default ingress to the cloud provider
• Identify if the cluster has tls secret
  • This provides HTTPS access to the service
• Identify if the cluster has external-dns
  • This provides Route53 endpoint to the service