-
MongoDB is implemented to perform several database operations. The installation can be done using the go dependency module.
go get go.mongodb.org/mongo-driver/mongo -
Redis Cache is implemented to integrate the data caching in the application. So, the go-redis will cache the second GET request while reading the user details.
go get github.com/go-redis/redis/v8 -
Kafka Message Broker: The confluent-kafka-go is used as a Go client library for Kafka message broker. The library will provide Producer and Consumer architecture to stream messages to the user for a subscribed topic. So, there will be two REST APIs that the user can use for Producing the messages reading from MongoDB and Consuming or Reading messages from the message broker.
go get github.com/confluentinc/confluent-kafka-go/kafkahttps://github.com/confluentinc/confluent-kafka-go
Note: It's recommended to install confluent-kafka-go v1.4.0, as the librdkafka will come with the bundle and no need to install separately.
Kubernetes provides several tools that can be useful to setup Kubernetes in the local environment.
-
minikube tool will run a single-node Kubernetes cluster running inside a Virtual Machine. Virtualization has to be supported in the computer and Hypervisor needed to be enabled.
Installation follows with the Hypervisor installation and Hyperkit is the recommended virtualization toolkit.
$ sudo install minikube $ minikube starthttps://kubernetes.io/docs/setup/learning-environment/minikube/
-
kubectl command-line tool will work to manage a Kubernetes cluster. The tool will be used to deploy, create, analyze, inspect pods that are running under a Kubernetes cluster.
Installation
curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
The application uses Docker for container-based development. The docker image gets published as a public repository at Docker Hub.
-
Build the image
$ docker build -t go-cache-kubernetes-v1 . -
Tag the image
$ docker tag go-cache-kubernetes-v1 deeptiman1991/go-cache-kubernetes-v1:1.0.0 -
Login to docker hub
$ docker login Type Username and Password to complete the authentication -
Push the image to docker hub
$ docker push deeptiman1991/go-cache-kubernetes-v1:1.0.0
There will be several deployments, services that need to be running in the cluster as a Pod. The creation of a Pod requires a YAML file that will specify the kind, spec, containerPort, metadata, volume, and more. So, these parameters will be used to provide resources to the Kubernetes cluster.
Start minikube to begin the deployment process start the minikube
$ minikube start
More details explanation on HashiCorp Vault, please check my Medium article : Secrets in Kubernetes and HashiCorp Vault
Implement the Vault Envs in the pod deployment
spec:
serviceAccountName: vault
containers:
- name: go-cache-kubernetes-container-poc
image: deeptiman1991/go-cache-kubernetes-v1:1.0.0
env:
- name: VAULT_ADDR
value: "http://vault:8200"
- name: JWT_PATH
value: "/var/run/secrets/kubernetes.io/serviceaccount/token"
- name: SERVICE_PORT
value: "8080"
| YAML | go-cache-poc-app.yaml |
So, now SECRET_USERNAME & SECRET_PASSWORD environment variables can be used to connect to the MongoDB database from the application.
This will allocate a volume of 1GB storage in the cluster
| Name | go-cache-poc-pvc |
| Kind | PersistentVolumeClaim |
| YAML | go-cache-poc-pvc.yaml |
$ kubectl apply -f go-cache-poc-pvc.yaml
This will load the web app Docker image in the cluster.
| Name | go-cache-poc |
| Kind | Deployment |
| YAML | go-cache-poc-app.yaml |
$ kubectl apply -f go-cache-poc-app.yaml
Verify
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
go-cache-kubernetes-app-poc 1/1 1 1 14s
There is only one pod is running under this deployment.
This service will create an external endpoint using a LoadBalancer.
| Name | go-cache-poc-service |
| Kind | Service |
| YAML | go-cache-poc-svc.yaml |
$ kubectl apply -f go-cache-poc-svc.yaml
Verify
$ kubectl get services
Kubernetes provides a feature that will allow us to create a stateful application in the cluster. There will be a storage class and services running under the cluster that will allow the databases to connect with services and store records in their persistent database.
More details explanation on MongoDB StatefulSet, please check my Medium article : MongoDB StatefulSet in Kubernetes
MongoDB service will create the Mongo services in the cluster.
| Name | mongodb-service |
| Kind | Service |
| YAML | mongodb-service.yaml |
$ kubectl apply -f mongodb-service.yaml
MongoDB StatefulSet will create the StatefulSet app in the cluster.
| Name | mongod |
| Kind | StatefulSet |
| YAML | mongodb-statefulset.yaml |
$ kubectl apply -f mongodb-statefulset.yaml
There will be three mongo containers in the cluster. We need to connect to anyone of them to define the administrator. Command to exec
$ kubectl exec -it mongod-0 -c mongod-container bash
-it: mongo app name
-c: mongo container name
Bash
$ hostname -f
mongod-0.mongodb-service.default.svc.cluster.local
Mongo Shell
$ mongo
Type to the following query to generate the replica set
> rs.initiate({_id: "MainRepSet", version: 1, members: [
{ _id: 0, host : "mongod-0.mongodb-service.default.svc.cluster.local:27017" }
]});
then verify
> rs.status();
Now create the Admin user
> db.getSiblingDB("admin").createUser({
user : "admin",
pwd : "admin123",
roles: [ { role: "root", db: "admin" } ]
});
So, now the MongoDB is complete setup with ReplicaSet and with an Administrator for the database.
There will be several steps to follow for deploying Redis into the Kubernete cluster.Download Docker images for Redis
$ docker run -p 6379:6379 redislabs/redismod
Redis Deployment
| Name | redismod |
| Kind | Deployment |
| YAML | redis-deployment.yaml |
$ kubectl apply -f redis-deployment.yaml
Redis Service
| Name | redis-service |
| Kind | Service |
| YAML | redis-service.yaml |
$ kubectl apply -f redis-service.yaml
Deploy the redismod image
$ kubectl run redismod --image=redislabs/redismod --port=6379
Expose the deployment
$ kubectl expose deployment redismod --type=NodePort
Now, check for the Redis Connection
$ redis-cli -u $(minikube service --format "redis://{{.IP}}:{{.Port}}" --url redismod)
You must be getting an ip address with a port that can be used as a connection string for Redis
There will be a deployment of ZooKeeper, Kafka Service, and running kafka/zookeeper server script. Please install Apache Kafka in your local machine and gcloud.
There will be deployment and service similar to the other Pods running in the cluster.
| Name | zookeeper-app |
| Kind | Deployment |
| YAML | zookeeper-deployment.yaml |
$ kubectl apply -f zookeeper-deployment.yaml
| Name | zookeeper-service |
| Kind | Service |
| YAML | zookeeper-service.yaml |
$ kubectl apply -f zookeeper-service.yaml
| Name | kafka-service |
| Kind | Service |
| YAML | kafka-service.yaml |
$ kubectl apply -f kafka-service.yaml
| Name | kafka-repcon |
| Kind | Deployment |
| YAML | kafka-repcon.yaml |
$ kubectl apply -f kafka-repcon.yaml
$ cd kafka/
$~/kafka/ bin/zookeeper-server-start.sh config/zookeeper.properties
$ cd kafka/
$~/kafka/ bin/kafka-server-start.sh config/server.properties
The kubectl is a very handy tool while troubleshooting application into the Kubernetes.
Few useful commands
- kubectl get pods
- kubectl describe pods
- kubectl logs
- kubectl exec -ti --bash
The go-swagger toolkit is integrated for the REST APIs documentation. The API doc can be accessible via http://localhost:5000/docs
https://github.com/go-swagger/go-swagger
- Getting started Kubernetes
- Kubernetes Secret Managements
- Hello Minikube
- Docker Documentation
- Swagger ReDoc
This project is licensed under the MIT License
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent