kishen-v / cpe-operator

automated continuous benchmarking framework on Kubernetes

• Installation
  • 1. Install supplementary/complementary systems
  • 2. Deploy controller (and parser)
    • 2.1. Deploy default CPE system (controller with parser)
    • 2.2. Deploy with recommended CPE system (controller with parser and service monitor)
    • 2.3. Custom deploy using kustomize
  • 3. Deploy Operators and Benchmark
• Roadmap

CPE Operator

CPE operator is a project that originally implements the AutoDECK framework. AutoDECK (Automated DEClarative Performance Evaluation and Tuning Framework on Kubernetes) is an evaluation system of Kubernetes-as-a-Service (KaaS) that automates configuring, deploying, evaluating, summarizing, and visualizing benchmarking workloads with a fully declarative manner.

Installation

1. Install supplementary/complementary systems

System	Description	Installation guide
Prometheus Operator	for monitoring and visualization	read more
Openshift Build Controller	for image tracking	read more
Node Tuning Operator	for node tuning	read more
Cloud Object Storaget (COS)	for job result logging	read more

2. Deploy controller (and parser)

Clone the repo and enter the workspace

git clone https://github.com/IBM/cpe-operator.git
cd cpe-operator

Deploy with either of the following choices:

2.1. Deploy default CPE system (controller with parser)

kubectl create -f ./config/samples/cpe-operator/default.yaml

2.2. Deploy with recommended CPE system (controller with parser and service monitor)

• require Prometheus Operator to be deployed.
• need to replace openshift-monitoring with the namespace that prometheus has been deployed for correcting RBAC resource.

kubectl create -f ./config/samples/cpe-operator/recommended.yaml

2.3. Custom deploy using kustomize

2.3.1. Set IMAGE_REGISTRY to your registry and update image in kustomization.yaml

export IMAGE_REGISTRY=[your registry URL]
export VERSION=[your image version tag]

** VERSION value must be specified as a valid semantic version for operator-sdk (Major.Minor.Patch)

2.3.2. Modify kustomization in /config

TAG	description	dependencies	to-modify kustomization	note
[PARSER]	Deploy CPE parser	-	default	Can specifiy PARSER_IMG environment for custom parser image
[PROMETHEUS]	Deploy ServiceMonitor and RBAC	Prometheus Operator	default	May need to modify namespace label in manager.yaml and RBAC depending on Prometheus deployment
[AUTO-TUNE]	Deploy tuning namespace and mounted to controller	Node Tuning Operator	default
[LOG-COS]	Set environment for COS secret	Cloud Object Storage secret (see /output)	default (and parser if [PARSER] enabled)

2.3.3. Deploy custom manifests

# require operator-sdk (>= 1.4), go (>= 1.13)
make deploy

# confirm cpe operator is running
kubectl get po -n cpe-operator-system
# see manager log
kubectl logs $(kubectl get po -n cpe-operator-system|grep controller|tail -1|awk '{print $1}') -n cpe-operator-system -c manager

To remove this operator run: make undeploy

3. Deploy Operators and Benchmark

See How to use the off-the-shelf/your own operator

Deploy simple coremark benchmark deployment:

kubectl create -f examples/none/cpe_coremark.yaml

Try benchmark with custom benchmark operator:

kubectl create -f examples/benchmark_operator/cpe_v1_benchmarkoperator_helm.yaml
# confirm ripsaw operator is running
kubectl get po -n my-ripsaw
kubectl create -f examples/benchmark_operator/cpe_v1_benchmark_iperf3.yaml
# confirm the job
kubectl get po -n my-ripsaw

Roadmap

• design custom resource; see benchmark_types.go, benchmarkoperator_types.go
• integrate to off-the-shelf benchmark operator; see benchmarks
• implement build tracker; see tracker
• raw output collector/parser; see output
  • integrate wrapper from snafu
• iteration support; see iteration
  • app-parameter variation (scenario)
  • spec configuration
  • node profile tuning
• visualize multi-cluster; see multi-cluster
• insert a sidecar if set
• combine resource usage metric; see metric
  • prometheus-export metrics
  • app-export metrics
  • eBPF metric collector