hyperfoil / horreum-operator

Operator managing Horreum instance in Openshift

Hyperfoil is microservice-oriented distributed benchmark framework that solves the coordinated-omission fallacy.

Project website: https://hyperfoil.io.

Prerequisites

• Java 17
• Apache Maven 3.8

Getting Started

mvn package

To run without test cases do

mvn -DskipTests=true package

Then the distribution is either in distribution/target/hyperfoil-<version>-SNAPSHOT.zip or in

cd distribution/target/distribution/

Image

We publish the image at quay.io/hyperfoil/hyperfoil.

Contributing

Contributions to Hyperfoil are managed on GitHub.com

• Ask a question
• Raise an issue
• Feature request
• Code submission

Contributions are most welcome !

You can reach the community on Zulip.

Please, consult our Code of Conduct policies for interacting in our community.

Consider giving the project a star on GitHub if you find it useful.

License

Apache-2.0 license

horreum-operator's Issues

Go version

use v1.21 atm

TLS not working in OCP cluster with self signed cert.

A deployed Horreum replica is reporting this error in the log.

2023-03-23 19:34:29,728 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (vert.x-eventloop-thread-1) HTTP Request to /api/user/administrators failed, error id: 086eb16d-d056-4b99-a1e0-6b8ff8122923-1: java.util.concurrent.CompletionException: javax.ws.rs.WebApplicationException: HTTP 500 Internal Server Error
712at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
713at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319)

Caused by: javax.net.ssl.SSLHandshakeException: Failed to create SSL connection
1059at io.vertx.core.net.impl.ChannelProvider$1.userEventTriggered(ChannelProvider.java:130)
1060... 25 more

Introduce versioning for operator releases, using stable horreum images

Separate dev/stable branches using versioned releases of artifacts, e.g. : https://github.com/Hyperfoil/horreum-operator/blob/master/controllers/keycloak.go#L54 should refer to stable version

Operator derived Horreum instance default users.

Users deploy Horreum in a cluster. User needs accounts created with sufficient roles. Either a default account or created after deployment.
Currently users are unable to use existing horreum-admin account to fulfil basic tasks. Users are seeing row level security errors raised by PostgrSQL database. Tweaking the roles for horreum-admin account does not allow them to proceed with basic tasks.

2023-03-23 22:16:43,379 ERROR [org.hib.eng.jdb.spi.SqlExceptionHelper] (executor-thread-184) ERROR: new row violates row-level security policy for table "test"
2023-03-23 22:16:43,379 INFO  [org.hib.eng.jdb.bat.int.AbstractBatchImpl] (executor-thread-184) HHH000010: On release of batch it still contained JDBC statements```

Version 0.2.0

We need new version to update the resource:

• drop Hyperfoil report
• change all External/ExternalHost/ExternalPort to just ExternalUrl if the operator should not handle deployment
• Make sure we can configure HTTPS for service-to-service communication; this should be the default (using generated service certs)

[x86]: Operator deployment seems broken as of now

Motivation:

We are already using Hyperfoil-operator on IBM Z(s390x) architecture
Since we are utilise it to benchmark operators for OCP on Z. This are some custom test scenarios specifically related to Z infra.
We need to port Horreum for storing these results.
Since Horreum have a lot of dependency on components like Postgres,keycloak.This might take some time for complete migration on IBM Z but we wanted to maintain a running instance in a x86 based cluster for now

NOTE:
Horreum operator was build from master branch and all issues mentioned are in horreum-app deployment/pod.

Issues:

1. There is a logic in code that expects the code to populate secret crd-name-app-certs(horreum-app-certs in my case) which is specific being done in code for vanilla k8s but logic is absent for open shift
2. After code changes for issue 1 was done, the init container complained that it needs jq to parse some stuff
   I modified this file Dockerfile and added command to install jq in horreum base image
3. After code changes for issues 2 was done, still the init container complains that it is unable to parse some values

Can not change realm name for deployment - Horreum realm is hardcoded

If the horreum realm is imported into an existing realm in a existing keycloak instance, the k8's deployment scripts do not work as they assume a horreum realm by default

Can not run Horreum with keycloak instance in different namespace

Trying to run Horreum, configured against an existing keycloak instance running in a different namespace fails due to self signed certificate for keycloak namespace

Keycloak version

In api/v1alpha1/horreum_types.go we have :latest, but we need to align that with what Horreum is actually using

docker build fails

Building the Dockerfile locally fails with:

Step 9/14 : RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o manager main.go
 ---> Running in f73d98b821af
# github.com/Hyperfoil/horreum-operator/api/v1alpha1
api/v1alpha1/horreum_types.go:174:25: cannot use &Horreum{} (value of type *Horreum) as type runtime.Object in argument to SchemeBuilder.Register:
	*Horreum does not implement runtime.Object (missing DeepCopyObject method)
api/v1alpha1/horreum_types.go:174:37: cannot use &HorreumList{} (value of type *HorreumList) as type runtime.Object in argument to SchemeBuilder.Register:
	*HorreumList does not implement runtime.Object (missing DeepCopyObject method)

$ docker --version
Docker version 23.0.3, build 3e7cbfd

horreum-db fails to start up when backed by PersistentVolume

The container is stuck in crashLoopBackOff , with message in the log
mkdir: cannot create directory '/var/lib/pgsql/data/userdata': Permission denied

By setting fsGroup at pod level to same as runAsUser might help in resolving the issue.

Remove grafana from operator

Grafana is no longer a requirement for Horreum infrastructure, a grafana instance should not be installed by the operator

Fix broken links in README

Step-by-step tutorial

Add a tutorial that describes the step-by-step commands to get a basic environment running, from the curl command to download minikube to the URL for the Horreum deployment

sample post-hook to upload-to-horreum needs to be fixed

I noticed the Operator generates a ConfigMap containing a sample post-hook that can be used to upload reports to Horreum using its API.

horreum-operator/controllers/horreum_controller.go

Lines 539 to 545 in 104e3a4

	TOKEN=$(curl -s -X POST ` + keycloakURL + `/auth/realms/horreum/protocol/openid-connect/token ` +
	` -H 'content-type: application/x-www-form-urlencoded' ` +
	` -d 'username='$HORREUM_USER'&password='$HORREUM_PASSWORD'&grant_type=password&client_id=horreum-ui'` +
	` | jq -r .access_token)
	curl -s '` + horreumURL + `/api/run/data?owner='$HORREUM_GROUP'&access=PUBLIC&test=$.info.benchmark&start=$.info.startTime&stop=$.info.terminateTime'` +
	` -X POST -H 'content-type: application/json' -d @$RUN_DIR/all.json -H 'Authorization: Bearer '$TOKEN

This sample post-hook needs some tweaks in order to work properly like the curl command used to retrieve a OIDC Token from keycloak needs to include the -k option in case of self-signed certs...
The other thing is to fix the OIDC token api URL from

/auth/realms/horreum/protocol/openid-connect/token

to

/realms/horreum/protocol/openid-connect/token

and also include the $RUN_ID to the json file path

-d @$RUN_DIR/$RUN_DIR/all.json

Step-by-step guide to install the operator

Create a step-by-step guide to install the operator into a Kubernetes environment.

Use minikube in the guide

provide a standalone Hyperfoil CLI image

Would be useful having the ability to run the CLI separately from the controller on a containerized environment.
So, having an image containing only the Hyperfoil CLI would offer more flexibility in a cloud-native environment.