hisunah / kubernetes-container-service-gitlab-sample

This code shows how a common multi-component GitLab can be deployed on Kubernetes cluster. Each component (NGINX, Ruby on Rails, Redis, PostgreSQL, and more) runs in a separate container or group of containers.

GitLab deployment on Kubernetes Cluster

Read this in other languages: 한국어.

This project shows how a common multi-component workload, in this case GitLab, can be deployed on Kubernetes Cluster. GitLab is famous for its Git-based and code-tracking tool. GitLab represents a typical multi-tier app and each component will have their own container(s). The microservice containers will be for the web tier, the state/job database with Redis and PostgreSQL as the database.

By using different GitLab components (NGINX, Ruby on Rails, Redis, PostgreSQL, and more), you can deploy it to Kubernetes. This example is also deployable using Compose for PostgreSQL in Bluemix as the database.

1. The user interacts with GitLab via the web interface or by pushing code to a GitHub repository. The GitLab container runs the main Ruby on Rails application behind NGINX and gitlab-workhorse, which is a reverse proxy for large HTTP requests like file downloads and Git push/pull. While serving repositories over HTTP/HTTPS, GitLab utilizes the GitLab API to resolve authorization and access and serves Git objects.

2. After authentication and authorization, the GitLab Rails application puts the incoming jobs, job information, and metadata on the Redis job queue that acts as a non-persistent database.

3. Repositories are created in a local file system.

4. The user creates users, roles, merge requests, groups, and more—all are then stored in PostgreSQL.

5. The user accesses the repository by going through the Git shell.

Included Components

• GitLab
• PostgreSQL
• Redis
• Kubernetes Clusters
• Bluemix container service
• Bluemix Compose for PostgreSQL

Objectives

This scenario provides instructions and learning for the following tasks:

• Build containers, and store them in container registry
• Use Kubernetes to create local persistent volumes to define persistent disks
• Deploy containers using Kubernetes pods and services
• Use Bluemix service in Kubernetes applications
• Deploy a distributed GitLab on Kubernetes

Deployment Scenarios

Deploy using Docker

see Deploying Gitlab with Docker

Deploy to Kubernetes

Create a Kubernetes cluster with either Minikube for local testing, or with IBM Bluemix Container Service to deploy in cloud. The code here is regularly tested against Kubernetes Cluster from Bluemix Container Service using Travis.

If you want to use the Bluemix Container Registry start by Uploading the images to the Bluemix Container Registry.

Deploy using DevOps Toolchain to Kubernetes Cluster from Bluemix Container Service

If you want to deploy the Gitlab directly to Bluemix, click on 'Deploy to Bluemix' button below to create a Bluemix DevOps service toolchain and pipeline for deploying the Gitlab sample, else jump to Steps

Please follow the Toolchain instructions to complete your toolchain and pipeline.

Steps

1. Use Kubernetes to create Services and Deployments

• 1.1 Use PostgreSQL in container or
• 1.2 Use PostgreSQL from Bluemix

2. Retrieve external ip and port for GitLab
3. GitLab is ready! Use GitLab to host your repositories

1. Use Kubernetes to create Services and Deployments for GitLab, Redis, and PostgreSQL

Ensure your kubernetes cluster is reachable by running the kubectl command.

$ kubectl get nodes
NAME             STATUS    AGE       VERSION
x.x.x.x   Ready     17h       v1.5.3-2+be7137fd3ad68f

Note: If this step fails see troubleshooting docs at Minikube or IBM Bluemix Container Service.

1.1 Use PostgreSQL in container

If you are using a container image to run PostgreSQL, run the following commands or run the quickstart script bash quickstart.sh with your Kubernetes cluster.

$ kubectl create -f kubernetes/local-volumes.yaml
$ kubectl create -f kubernetes/postgres.yaml
$ kubectl create -f kubernetes/redis.yaml
$ kubectl create -f kubernetes/gitlab.yaml

After you have created all the services and deployments, wait for 3 to 5 minutes. You can check the status of your deployment on Kubernetes UI. Run 'kubectl proxy' and go to URL 'http://127.0.0.1:8001/ui' to check when the GitLab container becomes ready.

Next retrieve your external ip and port for GitLab

1.2 Use PostgreSQL from Bluemix

Use the Bluemix catalog or the bx command to create a service instance of Compose for PostgreSQL and add a set of credentials.

$ bx service create compose-for-postgresql Standard "Compose for PostgreSQL-GL"
$ bx service key-create "Compose for PostgreSQL-GL" Credentials-1

Retrieve the connection string from the credentials object for the service on Bluemix.

$ bx service key-show "Compose for PostgreSQL-GL" "Credentials-1" | grep "postgres:"

Modify your kubernetes/gitlab-postgres-svc.yaml file and replace COMPOSE_PG_PASSWORD with the password, COMPOSE_PG_HOST with the hostname, and COMPOSE_PG_PORT with the port.

Using the above example, the env: section will look like this.

  env:
  - name: GITLAB_OMNIBUS_CONFIG
  value: |
      postgresql['enable'] = false
      gitlab_rails['db_username'] = "admin"
      gitlab_rails['db_password'] = "ETIDRKCGOEIGBMZA"
      gitlab_rails['db_host'] = "bluemix-sandbox-dal-9-portal.6.dblayer.com"
      gitlab_rails['db_port'] = "26576"
      gitlab_rails['db_database'] = "compose"
      gitlab_rails['db_adapter'] = 'postgresql'
      gitlab_rails['db_encoding'] = 'utf8'
      redis['enable'] = false
      gitlab_rails['redis_host'] = 'redis'
      gitlab_rails['redis_port'] = '6379'
      gitlab_rails['gitlab_shell_ssh_port'] = 30022
      external_url 'http://gitlab.example.com:30080'

Run the following commands or run the quickstart script bash quickstart-postgres-svc.sh with your Kubernetes cluster.

$ kubectl create -f kubernetes/local-volumes.yaml
$ kubectl create -f kubernetes/redis.yaml
$ kubectl create -f kubernetes/gitlab-postgres-svc.yaml

After you have created all the services and deployments, wait for 3 to 5 minutes. You can check the status of your deployment on Kubernetes UI. Run 'kubectl proxy' and go to URL 'http://127.0.0.1:8001/ui' to check when the GitLab container becomes ready.

2. Retrieve external ip and port for GitLab

After few minutes run the following commands to get your public IP and NodePort number.

$ kubectl get nodes
NAME             STATUS    AGE
169.47.241.22   Ready     23h
$ kubectl get svc gitlab
NAME      CLUSTER-IP     EXTERNAL-IP   PORT(S)                     AGE
gitlab    10.10.10.148   <nodes>       80:30080/TCP,22:30022/TCP   2s

Note: The 30080 port is for gitlab UI and the 30022 port is for ssh.

Note: The gitlab external url is set to gitlab.example.com add this to your hosts file pointing to your IP address from above in order to use the url that gitlab expects. If you can't do this, then using the IP (in this example 169.47.241.22) should work.

Note: If you using Minikube for local kubernetes deployment, you can access the list of service IPs using the minikube service list command.

Congratulations. Now you can use the link http://gitlab.example.com:30080 or http://<node_ip>:30080 to access your gitlab service from your web browser.

3. GitLab is ready! Use GitLab to host your repositories

Now that Gitlab is running you can register as a new user and create a project.

Troubleshooting

If a pod doesn't start examine the logs.

kubectl get pods
kubectl logs <pod name>

Cleanup

To delete all your services, deployments, and persistent volume claim, run

kubectl delete deployment,service,pvc -l app=gitlab

To delete your persistent volume, run

kubectl delete pv local-volume-1 local-volume-2 local-volume-3

To delete your PostgreSQL credentials and remove the service instance from Bluemix, run

bx service key-delete "Compose for PostgreSQL-GL" Credentials-1
bx service delete "Compose for PostgreSQL-GL"

License

Apache 2.0