logandonley / cb-core-oc-workshop

K8s configuration, JCasC, etc for the workshop OC.

cb-core-oc-workshop

K8s configuration, JCasC, etc for the workshop OC.

Jenkins Config-as-Code (JCasC)

The JCasC configuration for CJOC is managed as a K8s ConfigMap - see k8s/casc.yml.

Groovy Scripts

JCasC doesn't currently support all types of Jenkins configuration to include the configuration for a number of proprietary plugins for CJOC. One of those is the Operations Center Kubernetes Cloud. Using groovy scripts allows you to configure anything and everything - always had and always will. But it isn't the easiest thing to write. The nice thing about a number of CJOC proprietary features is that they are actually just Jenkins jobs. So you can retrieve the confing.xml for any of the special CJOC jobs and then createProjectFromXML method to create a jog via a groovy script. See [groovy-scripts/k8s-shared-cloud.groovy] for an example of this approach.

Core v2 Pod Security Policies

Kubernetes Pod Security Policies (PSP) allow controlling security sensitive aspects of the pod specification - and in the case of Core v2 on K8s that means the CJOC pod itself, master pods and agent pods.

For the most part, one very restrictive PSP will be adequate for all pods related to a Core v2 install - see cb-restricted.

But there is one use case where that PSP will not work - using Kaniko to build container images. Kaniko must run as root and therefore requires its own PSP (and ServiceAccount, Role, and RoleBinding) - see the K8s config for Kaniko.

PSPs and Jenkins Kubernetes Agents

The Jenkins Kubernetes plugin (for ephemeral K8s agents) defaults to using a K8s emptyDir volume type for the Jenkins agent workspace. This causes some issues when using a restrictive PSP such as the cb-restricted PSP mentioned above. Kubernetes defaults to mounting emptyDir volumes as root:root and permissions set to 750. When using a PSP with Jenkins K8s agent pods that doesn't allow containers to run as root the containers will not be able to access the default workspace directory provided by the Jenkins K8s plugin. One approach for dealing with this is to set the K8s securityContext at the pod spec level. You can do this in the K8s plugin UI via the Raw yaml for the Pod field or in the raw yaml of a pod spec that you load into your Jenkins job - here is an example of that approach.