redhat-cop / project-initialize-operator

Project Initialize Operator

This repository is currently undergoing active development. Functionality may be in flux

Overview

This repository contains the Project Initialize Operator which provides functionality for creating new projects within OpenShift and triggering custom on-boarding processes, specifically around the GitOps solution ArgoCD.

Install (OpenShift)

The operator will require cluster-admin permissions that can be applied using the resources provided in the deploy/ folder.

Create the expected namespace

$ oc new-project project-operator

Add the ProjectInitialize CRD and resources to the cluster

$ oc apply -f deploy/service_account.yaml
$ oc apply -f deploy/role.yaml
$ oc apply -f deploy/role_binding.yaml

Add ProjectInitialize CRD

4.X OCP

$ oc apply -f deploy/crds/redhatcop.redhat.io_projectinitializes_crd.yaml

3.X OCP

$ oc apply -f deploy/crds/redhatcop.redhat.io_projectinitializes_crd_3x.yaml

Add ProjectInitialize CRD

4.X OCP

$ oc apply -f deploy/crds/redhatcop.redhat.io_projectinitializequota_crd.yaml

3.X OCP

$ oc apply -f deploy/crds/redhatcop.redhat.io_projectinitializequota_crd_3x.yaml

Deploy Operator (OpenShift)

Run the following command when ready to deploy the operator into cluster it will monitor

$ oc apply -f deploy/operator.yaml

Namespace Labels/Annotations

Labels and annotations can be added to the namespace that is generated through the operator by specifying the values within the ProjectInitialize CR.

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitialize
metadata:
  name: example-projectinitialize
spec:
  team: test
  env: dev
  cluster: clusterA
  displayName: "Test Project"
  desc: "A test project for showing the functionality of the Project Initialize Operator"
  quotaSize: small
  namespaceDetails:
    annotations:
      testKey: testValue
    labels:
      testKey: testValue

Adding Defined Quota Sizes to Cluster

When the quotaSize attribute is defined in the ProjectInitializeQuota Custom Resource (CR) the operator will search for a cluster level ProjectInitializeQuota CR that defines a particular quota size. This can be used to define predetermined t-shirt sizes when creating new projects (small, medium, large, etc)

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitializeQuota
metadata:
  name: small
spec:
  hard:
    cpu: "5"
    memory: "10Gi"
    pods: "10"

Example Workflow

The Project Initialize Operator will need to be running in the project-operator namespace before running the following example workflow.

Apply T-Shirt Size

First start by applying the ProjectInitializeQuota CR that will be a global t-shirt size placeholder that the initializer can reference when applying quotas to new projects.

$ oc apply -f deploy/examples/small_projectqouta_cr.yaml

Apply Project Initializer

Apply the ProjectInitialize CR which contains details about the dev team name, cluster name, and a reference to the ProjectInitializeQuota which will specify the quota to assign the namespace.

Creating a ProjectInitialize object will result in a new project (namespace) being created.

$ oc apply -f deploy/examples/basic_projectinit_cr.yaml

The project name will be a derivation of the team and env specified in the ProjectInitalize object. The result will be ${team}-${env}. For example

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitialize
metadata:
  name: phoenix-dev-projectinitialize
spec:
  team: phoenix
  env: dev
  cluster: us-west-2
  displayName: "Phoenix project for Dev environment"
  desc: "a test project for showing the functionality of the project initialize operator"
  quotaSize: small
  namespaceDetails:
    annotations:
      testkey: testValue
    labels:
      testkey: testValue

Will result in a namespace like this:

$ oc apply -f phoenix-dev.yaml
projectinitialize.redhatcop.redhat.io/phoenix-dev-projectinitialize created
$ oc get projects phoenix-dev
NAME          DISPLAY NAME                          STATUS
phoenix-dev   Phoenix project for Dev environment   Active

Examining the YAML definition is instructive:

$ oc get projects phoenix-dev -o yaml

apiVersion: project.openshift.io/v1
kind: Project
metadata:
  annotations:
    openshift.io/description: a test project for showing the functionality of the
      project initialize operator
    openshift.io/display-name: Phoenix project for Dev environment
    openshift.io/requester: system:serviceaccount:project-operator:project-initialize
    openshift.io/sa.scc.mcs: s0:c24,c9
    openshift.io/sa.scc.supplemental-groups: 1000570000/10000
    openshift.io/sa.scc.uid-range: 1000570000/10000
    testkey: testvalue
  creationTimestamp: "2020-10-01T19:07:20Z"
  labels:
    app: phoenix
    env: dev
  name: phoenix-dev
  resourceVersion: "233538"
  selfLink: /apis/project.openshift.io/v1/projects/phoenix-dev
  uid: c2ce8b0a-8354-4777-b7fb-fae08354ccb5
spec:
  finalizers:
  - kubernetes
status:
  phase: Active

Development

For help with development, see docs/development.md

project-initialize-operator's Issues

ProjectInitialize resource cannot have explicit Name

Request to make the following specification change for ProjectInitialize resource type:

Current

Currently, the name for a namespace and project is create by way of <spec.team>-<spec.env>. This does not allow for a user to set a name if they wish.

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitialize
metadata:
  name: test-namespace-1234b
spec:
  cluster: us-east-1.example.opentlc.com
  desc: ''
  displayName: Test Namespace 3
  env: production
  namespaceDetails:
    annotations:
      koncierge.io/quota: small
      koncierge.io/rolebindings: admin:view
    labels:
      app: koncierge
      component: server
      version: 0.0.1
  quotaSize: small
  team: backend-dev

Proposed

Update specification to allow for spec.namespace.name. This would allow for a user to set the name explicitly. Additionally, it could still default to <spec.team>-<spec.env> if spec.namespace.name is not specified. Also note that namespaceDetails is now just namespace as it is implied that attributes under namespace are details for it.

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitialize
metadata:
  name: test-namespace-1234b
spec:
  cluster: us-east-1.example.opentlc.com
  desc: ''
  displayName: Test Namespace 3
  env: production
  namespace:
    name: test-namespace-1234b
    annotations:
      koncierge.io/quota: small
      koncierge.io/rolebindings: admin:view
    labels:
      app: koncierge
      component: server
      version: 0.0.1
  quotaSize: small
  team: backend-dev

Provide the ability to specify policies that will be applied upon project creation

Provide the ability to create policies that will be applied to the namespace upon creation?

• Roles
• RoleBindings
• ClusterRoleBindings

ProjectInitialize resource does not update after initial application

After applying the following ProjectInitialize resource, saved as PI.yaml, using oc apply -f PI.yaml:

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitialize
metadata:
  name: test-namespace-1234b
spec:
  cluster: us-east-1.example.opentlc.com
  desc: ''
  displayName: Test Namespace
  env: production
  namespaceDetails:
    annotations:
      koncierge.io/quota: small
      koncierge.io/rolebindings: admin:view
    labels:
      app: koncierge
      component: server
      version: 0.0.1
  quotaSize: small
  team: backend-dev

All resources are created properly; however, if you want to update information such as the following items:

• spec.cluster
• spec.desc
• spec.displayName
• spec.production
• spec.team

by updated that PI.yaml file and running oc apply -f PI.yaml again, the resource is not updated. According to https://github.com/redhat-cop/project-initialize-operator/blob/master/pkg/controller/projectinitialize/projectinitialize_controller.go#L150, the logic for these attributes is just missing at the moment.

quota does not get applied if ProjectInitializeQuota is added after ProjectInitialize is created

I found this behavior. It's somewhat of a corner case, but I think adding logic to handle it at some point would make sense.

I was playing with some order of operations and created a ProjectInitialize before creating the appropriate ProjectInitializeQuota. The operator throws this error:

{"level":"info","ts":1586358358.1973412,"logger":"controller_projectinitialize","msg":"Reconciling ProjectInitialize","Request.Namespace":"","Request.Name":"example-projectinitialize"}                          
{"level":"error","ts":1586358358.6634288,"logger":"controller_quayecosystem","msg":"Quota definition not found: small","error":"ProjectInitializeQuota.redhatcop.redhat.io \"small\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/gnekic/git/project-initialize-operator/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/redhat-cop/project-initialize-operator/project-initialize/pkg/controller/projectinitialize/ocp/project.GetQuotaSizeFromCluster\n\t/home/gnekic/git/project-initialize-operator/pkg/controller/projectinitialize/ocp/project/quotas.go:29\ngithub.com/redhat-cop/project-initialize-operator/project-initialize/pkg/controller/projectinitialize.(*ReconcileProjectInitialize).Reconcile\n\t/home/gnekic/git/project-initialize-operator/pkg/controller/projectinitialize/projectinitialize_controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/gnekic/git/project-initialize-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:256\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/gnekic/git/project-initialize-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/home/gnekic/git/project-initialize-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/home/gnekic/git/project-initialize-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/home/gnekic/git/project-initialize-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/home/gnekic/git/project-initialize-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
{"level":"error","ts":1586358358.6635063,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"projectinitialize-controller","request":"/example-projectinitialize","error":"ProjectInitializeQuota.redhatcop.redhat.io \"small\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/gnekic/git/project-initialize-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/gnekic/git/project-initialize-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:258\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/gnekic/git/project-initialize-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/home/gnekic/git/project-initialize-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/home/gnekic/git/project-initialize-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/home/gnekic/git/project-initialize-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/home/gnekic/git/project-initialize-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

Maybe I didn't wait long enough after I created the ProjectInitializeQuota to let the operator apply it to the project. I restarted the operator to see if that would trigger it, but it found that the project existed and did nothing.

{"level":"info","ts":1586358690.1084354,"logger":"controller_projectinitialize","msg":"Reconciling ProjectInitialize","Request.Namespace":"","Request.Name":"example-projectinitialize"}                          
{"level":"info","ts":1586358690.1136522,"logger":"controller_quayecosystem","msg":"Found project test-dev"}

I then created a new project reffering to the now created ProjectInializeQuota, and it added it properly. Log output here:

{"level":"info","ts":1586358937.7144792,"logger":"controller_projectinitialize","msg":"Reconciling ProjectInitialize","Request.Namespace":"","Request.Name":"nekic-projectinitialize"}                            
{"level":"info","ts":1586358938.1459012,"logger":"controller_quayecosystem","msg":"Created quota nekic-test-quota for namespace nekic-test-dev"}                                                                  
{"level":"info","ts":1586358938.1459234,"logger":"controller_quayecosystem","msg":"Created new project nekic-test-dev"}  

Here are the configs I used.

First ProjectInitialize

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitialize
metadata:
  name: example-projectinitialize
spec:
  team: test
  env: dev
  cluster: np1
  displayName: "Test Project"
  desc: "A test project for showing the functionality of the project initialize operator"
  quotaSize: small

Small ProjectInitializeQuota

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitializeQuota
metadata:
  name: small
spec:
  hard:
    cpu: "5"
    memory: "10Gi"
    pods: "10"

Second ProjectInitializeQuota

apiVersion: redhatcop.redhat.io/v1alpha1
kind: ProjectInitialize
metadata:
  name: nekic-projectinitialize
spec:
  cluster: np1
  desc: A test project for showing the functionality of the project initialize operator
  displayName: nekic Project
  env: dev
  quotaSize: small
  team: nekic-test