I'm going to deploy my java based application in Docker Container and the K8S cluster. I have used below repository to deploying application.
https://github.com/kohlidevops/jpetstore.git
Launch new EC2 t2.large instance with Ubuntu-22 Image.
SSH to Jenkins instance and run below commands to install Jenkins
sudo apt update -y
sudo apt upgrade -y
wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | sudo apt-key add -
echo "deb https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | sudo tee /etc/apt/sources.list.d/adoptium.list
sudo apt update
sudo apt install temurin-17-jdk
/usr/bin/java --version
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
/usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl start jenkins
sudo systemctl status jenkins
After installation of Jenkins, I will create Inbound Port 8080, since Jenkins works on Port 8080.
But for my case, we are running Jenkins on another port. Because my application has to be use 8080 port. So, I'm going to change the port to 8090 using the below commands.
sudo systemctl stop jenkins
sudo systemctl status jenkins
cd /etc/default
sudo vi jenkins
<change port HTTP_PORT=8090 and save and exit>
cd /lib/systemd/system
sudo vi jenkins.service
<change Environments="Jenkins_port=8090" save and exit>
sudo systemctl daemon-reload
sudo systemctl restart jenkins
sudo systemctl status jenkins
Now access the Jenkins webui using IP with port - 8090 and login the console then install suggested plugins.
To install a docker and configure using below commands.
sudo apt-get update
sudo apt-get install docker.io -y
sudo usermod -aG docker $USER
newgrp docker
sudo chmod 777 /var/run/docker.sock
docker run -d --name sonar -p 9000:9000 sonarqube:lts-community
Now, I can able to access the sonarqube docker container. Remember! default username is admin and password is admin. Then I have to reset the admin password.
sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y
To install below plugins and restart the Jenkins.
Eclipse Temurin Installer
SonarQube Scanner
To configure Java and Maven in Jenkins using Global tool configuration. Jenkins -> Manage Jenkins -> Tools
Add JDK
Add Maven
To create a jenkins job with pipeline
Im going to keep the maximum number of build as 4
Navigate to Pipeline and select Pipeline script and paste the below script then check whether its working or not
pipeline{
agent any
tools {
jdk 'jdk17'
maven 'maven3'
}
stages{
stage ('Clean Workspace'){
steps{
cleanWs()
}
}
stage ('Git Checkout') {
steps {
git branch: 'main', url: 'https://github.com/kohlidevops/jpetstore.git'
}
}
stage ('Maven Compile') {
steps {
sh 'mvn clean compile'
}
}
stage ('Maven Test') {
steps {
sh 'mvn test'
}
}
}
}
Now start the build to see the result. Perfect My build has been succeeded.
I have launched Sonarqube application using docker container in Jenkins server. You can access the Sonarqube using the Jenkins IP address with Port 9000.
By default, user name and password is "admin" for sonarqube. Then we have to reset the password.
To access the Sonarqube application from Jenkins, we have to create Token in Sonarqube. You can navigate using below steps.
Sonarqube Application -> Login -> Administration → Security → Users → Click on Tokens and Update Token → Meaningful name → and click on Generate Token
This token will shown for one time. So save it locally for later use.
Navigate to Jenkins console and do below steps to save Sonarqube token as securely.
Jenkins console -> Manage Jenkins -> Credentials
Add Credentials -> Secret Text -> Paste the Sonarqube token in Secret label -> provide meaningful name and save it.
Navigate to Jenkins console -> Manage Jenkins -> System -> Sonarqube servers -> Sonarqube installation -> Add
Provide a meaningful name -> Server URL (Sonarqube URL with port) -> Server authentication token - Select the token which is created just before in Jenkins credentials.
Apply and save.
To install a Sonarscanner in Jenkins console using Global Tool.
Jenkins -> Manage Jenkins -> Tools
Select -> Sonarqube scanner installation -> Add
Provide a meaningful name and install sonarscanner from Maven central.
Apply and save it.
Login to Sonarqube application -> Administration -> Configuration -> Webhooks -> Create a webhook
Provide a meaningful name and URL should be "Jenkins-URL:Port/sonarqube-webhook/ and create it.
To add a Sonarqube stage in Jenkins pipeline using below code
environment {
SCANNER_HOME=tool 'sonar-scanner'
}
stage("Sonarqube Analysis "){
steps{
withSonarQubeEnv('sonar-server') {
sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Petshop \
-Dsonar.java.binaries=. \
-Dsonar.projectKey=Petshop '''
}
}
}
stage("quality gate"){
steps {
script {
waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
}
}
}
tool 'sonar-scanner' -> I have configured with this name in Jenkins Global tool configuration
Environamet 'sonar-server' -> I have installed sonar-scanner from Maven in Jenkins System with this same name
Token 'Sonar-token' -> I have created with this Id in Jenkins credentials for Quality status check
Now, update this code in pipeline and start the build again.
This build too successfully completed without fail.
If you want to check with sonarqube application for code analysis, Then please logon to the sonarqube application check the code status.
Jenkins console → Manage Jenkins → Plugins → OWASP Dependency-Check. Click on it and install it without restart.
In order to configure OWASP in Jenkins Tools
Jenkins console -> Manage Jenkins -> Tools
Apply and save it.
Jenkins console -> select your Job -> Navigate to Pipeline and add below stages
stage ('Build WAR file'){
steps{
sh 'mvn -N io.takari:maven:wrapper'
sh 'mvn clean install -DskipTests=true'
}
}
stage("OWASP Dependency Check"){
steps{
dependencyCheck additionalArguments: '--scan ./ --format XML ', odcInstallation: 'DP-Check'
dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
}
}
Apply and save it - Then start the build to see the result.
Jenkins console → Manage Plugins → Available plugins → Search for Docker and install these plugins.
Docker
Docker Commons
Docker Pipeline
Docker API
docker-build-step
Click install without restart.
Jenkins console -> Manage Jenkins -> Tools -> Docker Installations -> Add Docker
Apply and save.
Jenkins console -> Manage Jenkins -> Credentials -> System -> Global credentials -> Add -> User nme and password
Then create.
Add below stages in existing pipeline script and start the build to see the result.
stage ('Build and push to docker hub'){
steps{
script{
withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
sh "docker build -t petshop ."
sh "docker tag petshop latchudevops/petshop:latest"
sh "docker push latchudevops/petshop:latest"
}
}
}
}
stage("TRIVY"){
steps{
sh "trivy image latchudevops/petshop:latest > trivy.txt"
}
}
stage ('Deploy to container'){
steps{
sh 'docker run -d --name pet1 -p 8080:8080 latchudevops/petshop:latest'
}
}
This build stage will build the docker image using below dockerfile.
https://github.com/kohlidevops/jpetstore/blob/main/Dockerfile
After the build image, the image should push to Docker repository.
Then this image will scanned by Trivy before deploy on docker container in Jenkins machine.
The build has been succedded.
I can able to see my docker container in Jenkins machine.
If i hit my URL with Port 8080 - Because myapp listening on Port 8080.
I can able to see my Images in Docker hub repository.
SSH to Jenkins machine and install below things to make available kubectl.
sudo apt update
sudo apt install curl
curl -LO https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
To launch two t3.medium ubuntu-20 machines for Kubernetes Master and Worker node.
sudo apt-get update
sudo apt-get install -y docker.io
sudo usermod –aG docker ubuntu
newgrp docker
sudo chmod 777 /var/run/docker.sock
sudo curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo tee /etc/apt/sources.list.d/kubernetes.list <<EOF
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo snap install kube-apiserver
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
sudo kubeadm join 172.31.39.217:6443 --token p9nqcl.t37afz1pxvz2ubls \
--discovery-token-ca-cert-hash sha256:12d30c9d1c32738701c7247240502524188bc4ec402bc07d7dc4b77b0dbea507
cd .kube cat config
To copy and paste this content in local server and this file called as Secret File.txt. I will use this file in jenkins later.
To configure the Secret File.txt (which is created in last step) in Jenkins Global credentials.
Jenkins -> Manage Jenkins -> Credentials -> System -> Global credentials -> New credentials -> Secret file -> upload the text file (Secret File.txt)
Thats it! save the credentials.
To install kubernetes plugins in Jenkins console.
Jenkins -> Manage Jenkins -> Plugins -> Available -> Select and install below plugins without restart
To configure mail server in Jenkins to receive notification when build has performed actions such as passed, failed and so on.
Jenkins -> Manage plugins -> Available -> install below plugin.
Go to your Gmail account and click on your profile. Then click on Manage Your Google Account -> click on the security tab on the left side panel you will get this below page.
2-step verification should be enabled. Search for the app in the search bar and you will get a app passwords like the below image.
Then create a App name as Jenkins or any meaning ful name and create a password -> Then note it for later use.
Jenkins -> Manage Jenkins -> System
Note: Password shoudl be generated password for app in last step.
Then apply and save.
Jenkins -> manage jenkins -> credentials -> system -> global credentials -> add user name and password
Note: Password shoudl be generated password for app in last step.
Then create the credentials
Jenkins -> Manage jenkins -> System -> under Extended Email notification
Then Apply and save. You can Test out too before start build.
To add below stage in your pipeline -> This stage will use the global credentials to deploy the app on kubernetes worker node.
stage('K8 deployment stage'){
steps{
script{
withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
sh 'kubectl apply -f deployment.yaml'
}
}
}
}
Apply and save.
post {
always {
emailext attachLog: true,
subject: "'${currentBuild.result}'",
body: "Project: ${env.JOB_NAME}<br/>" +
"Build Number: ${env.BUILD_NUMBER}<br/>" +
"URL: ${env.BUILD_URL}<br/>",
to: '[email protected]',
attachmentsPattern: 'trivy.txt'
}
}
Apply and save.
Now start the build to see the results of all the stages.
My build has been succeded as i expect.
If i'm going to check with my kubernetes master with below command after the build.
kubectl get all
I can able to see the my kubernetes worker node is running with my docker app.
Now try to access the kubernetes worker node public ip and port number. Here we go!
I can check with my email to ensure the receiving email reports.
That's it!
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent