hyperfoil / horreum

Benchmark results repository service

Hyperfoil is microservice-oriented distributed benchmark framework that solves the coordinated-omission fallacy.

Project website: https://hyperfoil.io.

Prerequisites

• Java 17
• Apache Maven 3.8

Getting Started

mvn package

To run without test cases do

mvn -DskipTests=true package

Then the distribution is either in distribution/target/hyperfoil-<version>-SNAPSHOT.zip or in

cd distribution/target/distribution/

Image

We publish the image at quay.io/hyperfoil/hyperfoil.

Contributing

Contributions to Hyperfoil are managed on GitHub.com

• Ask a question
• Raise an issue
• Feature request
• Code submission

Contributions are most welcome !

You can reach the community on Zulip.

Please, consult our Code of Conduct policies for interacting in our community.

Consider giving the project a star on GitHub if you find it useful.

License

Apache-2.0 license

horreum's Issues

Schema definitions for repeating data reports

Removing of last test run removes the test

If I create a test, upload the test results
and after that remove the test result from Runs page
then the whole test gets removed and I cannot see it in Tests window anymore

Managing benchmark definitions

UI, adding webhooks

Regression vars UI issues

• The selects don't gather existing groups. fd00902
• Add the 'Add variable' button to the bottom as well. 1ef102a
• It would be nice to be able to rename a group. 3f2503c
• Copy group vs. copy all variables from other test 3f2503c

Filter Runs by json content

query either all runs or runs for a specific test based on data content.
e.g. find all the runs that collected dstat

Use quarkus' container-image extension instead of Docker.jvm

Examples of the SQL/materialized view

Server side errors are not captured in UI

Steps to recreate:

1. upload data to a test without a schema defined
2. Click on run count link in "Test" view .e.g. http://localhost:3000/run/list/10

A spinner is displayed continuously and following is logged to the browser console;

"Failed to load resource: the server responded with a status of 500 (Internal Server Error)"

Uncaught (in promise) Error: Internal Server Error
    at index.js:41

The UI should display a suitable error message to the user

Below is the following server side exception;

2020-05-11 20:21:21,644 ERROR [io.und.req.io] (executor-thread-12) Exception handling request 1037a9e8-edb4-4ac0-9143-1476f1f62de3-18 to /api/test/10: org.jboss.resteasy.spi.UnhandledException: javax.ws.rs.ProcessingException: RESTEASY008205: JSON Binding serialization error javax.json.bind.JsonbException: Unable to serialize property 'defaultView' from io.hyperfoil.tools.repo.entity.json.Test
	at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:381)
	at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:216)
	at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:610)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:520)
	at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:259)
	at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:160)
	at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:362)
	at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:163)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:245)
	at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:249)
	at io.quarkus.resteasy.runtime.ResteasyFilter$ResteasyResponseWrapper.sendError(ResteasyFilter.java:65)
	at io.undertow.servlet.handlers.DefaultServlet.doGet(DefaultServlet.java:172)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:503)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
	at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
	at io.quarkus.resteasy.runtime.ResteasyFilter.doFilter(ResteasyFilter.java:28)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at io.hyperfoil.tools.repo.server.RouteFilter.doFilter(RouteFilter.java:26)
	at io.hyperfoil.tools.repo.server.RouteFilter_ClientProxy.doFilter(RouteFilter_ClientProxy.zig:430)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
	at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:63)
	at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
	at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
	at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:67)
	at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:133)
	at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
	at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:65)
	at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
	at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
	at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
	at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:247)
	at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:56)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:111)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:108)
	at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
	at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
	at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$10$1$1.call(UndertowDeploymentRecorder.java:563)
	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:227)
	at io.undertow.servlet.handlers.ServletInitialHandler.handleRequest(ServletInitialHandler.java:152)
	at io.undertow.server.handlers.HttpContinueReadHandler.handleRequest(HttpContinueReadHandler.java:43)
	at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$1.handleRequest(UndertowDeploymentRecorder.java:113)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:290)
	at io.undertow.server.DefaultExchangeHandler.handle(DefaultExchangeHandler.java:18)
	at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$6.handle(UndertowDeploymentRecorder.java:391)
	at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$6.handle(UndertowDeploymentRecorder.java:373)
	at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1034)
	at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:131)
	at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:130)
	at io.quarkus.vertx.http.runtime.devmode.VertxHotReplacementSetup.handleHotReplacementRequest(VertxHotReplacementSetup.java:39)
	at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1034)
	at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:131)
	at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:130)
	at io.quarkus.vertx.http.runtime.security.HttpAuthorizer.doPermissionCheck(HttpAuthorizer.java:107)
	at io.quarkus.vertx.http.runtime.security.HttpAuthorizer.access$100(HttpAuthorizer.java:26)
	at io.quarkus.vertx.http.runtime.security.HttpAuthorizer$2.accept(HttpAuthorizer.java:124)
	at io.quarkus.vertx.http.runtime.security.HttpAuthorizer$2.accept(HttpAuthorizer.java:113)
	at io.smallrye.mutiny.helpers.UniCallbackSubscriber.onItem(UniCallbackSubscriber.java:68)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.DefaultUniEmitter.complete(DefaultUniEmitter.java:36)
	at io.smallrye.mutiny.groups.UniCreate.lambda$item$2(UniCreate.java:205)
	at io.smallrye.mutiny.operators.UniCreateWithEmitter.subscribing(UniCreateWithEmitter.java:22)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:43)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:38)
	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:49)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.lambda$subscribing$0(ContextPropagationUniInterceptor.java:51)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.subscribing(ContextPropagationUniInterceptor.java:51)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:43)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:38)
	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:49)
	at io.smallrye.mutiny.groups.UniSubscribe.with(UniSubscribe.java:69)
	at io.quarkus.vertx.http.runtime.security.HttpAuthorizer.doPermissionCheck(HttpAuthorizer.java:113)
	at io.quarkus.vertx.http.runtime.security.HttpAuthorizer.checkPermission(HttpAuthorizer.java:91)
	at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$3.handle(HttpSecurityRecorder.java:155)
	at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$3.handle(HttpSecurityRecorder.java:147)
	at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1034)
	at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:131)
	at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:130)
	at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$2$5.accept(HttpSecurityRecorder.java:135)
	at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$2$5.accept(HttpSecurityRecorder.java:129)
	at io.smallrye.mutiny.helpers.UniCallbackSubscriber.onItem(UniCallbackSubscriber.java:68)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.UniOnEventConsume$1.onItem(UniOnEventConsume.java:27)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.UniOnEventConsume$1.onItem(UniOnEventConsume.java:27)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.UniDelegatingSubscriber.onItem(UniDelegatingSubscriber.java:24)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.DefaultUniEmitter.complete(DefaultUniEmitter.java:36)
	at io.smallrye.mutiny.groups.UniCreate.lambda$item$2(UniCreate.java:205)
	at io.smallrye.mutiny.operators.UniCreateWithEmitter.subscribing(UniCreateWithEmitter.java:22)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:43)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:38)
	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:49)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.lambda$subscribing$0(ContextPropagationUniInterceptor.java:51)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.subscribing(ContextPropagationUniInterceptor.java:51)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:43)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:38)
	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:49)
	at io.smallrye.mutiny.operators.UniFlatMapOnItem.invokeAndSubstitute(UniFlatMapOnItem.java:48)
	at io.smallrye.mutiny.operators.UniFlatMapOnItem$2.onItem(UniFlatMapOnItem.java:65)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.UniDelegatingSubscriber.onItem(UniDelegatingSubscriber.java:24)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.DefaultUniEmitter.complete(DefaultUniEmitter.java:36)
	at io.smallrye.mutiny.groups.UniCreate.lambda$item$2(UniCreate.java:205)
	at io.smallrye.mutiny.operators.UniCreateWithEmitter.subscribing(UniCreateWithEmitter.java:22)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:43)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:38)
	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:49)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.lambda$subscribing$0(ContextPropagationUniInterceptor.java:51)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.subscribing(ContextPropagationUniInterceptor.java:51)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:43)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:38)
	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:49)
	at io.smallrye.mutiny.operators.UniFlatMapOnItem.invokeAndSubstitute(UniFlatMapOnItem.java:48)
	at io.smallrye.mutiny.operators.UniFlatMapOnItem$2.onItem(UniFlatMapOnItem.java:65)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.UniDelegatingSubscriber.onItem(UniDelegatingSubscriber.java:24)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.DefaultUniEmitter.complete(DefaultUniEmitter.java:36)
	at io.smallrye.mutiny.groups.UniCreate.lambda$item$2(UniCreate.java:205)
	at io.smallrye.mutiny.operators.UniCreateWithEmitter.subscribing(UniCreateWithEmitter.java:22)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:43)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:38)
	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:49)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.lambda$subscribing$0(ContextPropagationUniInterceptor.java:51)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.subscribing(ContextPropagationUniInterceptor.java:51)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:43)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:38)
	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:49)
	at io.smallrye.mutiny.operators.UniFlatMapOnItem.invokeAndSubstitute(UniFlatMapOnItem.java:48)
	at io.smallrye.mutiny.operators.UniFlatMapOnItem$2.onItem(UniFlatMapOnItem.java:65)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$0(SmallRyeThreadContext.java:215)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:35)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:72)
	at io.smallrye.mutiny.operators.DefaultUniEmitter.complete(DefaultUniEmitter.java:36)
	at io.quarkus.security.runtime.QuarkusIdentityProviderManagerImpl$1$1$1$1.run(QuarkusIdentityProviderManagerImpl.java:58)
	at io.quarkus.runtime.CleanableExecutor$CleaningRunnable.run(CleanableExecutor.java:231)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2046)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1578)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1426)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at org.jboss.threads.JBossThread.run(JBossThread.java:479)
Caused by: javax.ws.rs.ProcessingException: RESTEASY008205: JSON Binding serialization error javax.json.bind.JsonbException: Unable to serialize property 'defaultView' from io.hyperfoil.tools.repo.entity.json.Test
	at org.jboss.resteasy.plugins.providers.jsonb.JsonBindingProvider.asyncWriteTo(JsonBindingProvider.java:158)
	at org.jboss.resteasy.core.interception.jaxrs.ServerWriterInterceptorContext.writeTo(ServerWriterInterceptorContext.java:87)
	at org.jboss.resteasy.core.interception.jaxrs.AbstractWriterInterceptorContext.asyncProceed(AbstractWriterInterceptorContext.java:203)
	at org.jboss.resteasy.core.interception.jaxrs.AbstractWriterInterceptorContext.getStarted(AbstractWriterInterceptorContext.java:166)
	at org.jboss.resteasy.core.interception.jaxrs.ServerWriterInterceptorContext.lambda$getStarted$0(ServerWriterInterceptorContext.java:73)
	at org.jboss.resteasy.core.interception.jaxrs.ServerWriterInterceptorContext.aroundWriteTo(ServerWriterInterceptorContext.java:93)
	at org.jboss.resteasy.core.interception.jaxrs.ServerWriterInterceptorContext.getStarted(ServerWriterInterceptorContext.java:73)
	at org.jboss.resteasy.core.ServerResponseWriter.lambda$writeNomapResponse$3(ServerResponseWriter.java:162)
	at org.jboss.resteasy.core.interception.jaxrs.ContainerResponseContextImpl.filter(ContainerResponseContextImpl.java:403)
	at org.jboss.resteasy.core.ServerResponseWriter.executeFilters(ServerResponseWriter.java:251)
	at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:101)
	at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:74)
	at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:590)
	... 242 more
Caused by: javax.json.bind.JsonbException: Unable to serialize property 'defaultView' from io.hyperfoil.tools.repo.entity.json.Test
	at org.eclipse.yasson.internal.serializer.ObjectSerializer.serializeInternal(ObjectSerializer.java:71)
	at org.eclipse.yasson.internal.serializer.AbstractContainerSerializer.serialize(AbstractContainerSerializer.java:107)
	at org.eclipse.yasson.internal.Marshaller.serializeRoot(Marshaller.java:147)
	at org.eclipse.yasson.internal.Marshaller.marshall(Marshaller.java:73)
	at org.eclipse.yasson.internal.Marshaller.marshall(Marshaller.java:101)
	at org.eclipse.yasson.internal.JsonBinding.toJson(JsonBinding.java:126)
	at org.jboss.resteasy.plugins.providers.jsonb.JsonBindingProvider.asyncWriteTo(JsonBindingProvider.java:154)
	... 254 more
Caused by: javax.json.bind.JsonbException: Unable to serialize property 'components' from io.hyperfoil.tools.repo.entity.json.View
	at org.eclipse.yasson.internal.serializer.ObjectSerializer.serializeInternal(ObjectSerializer.java:71)
	at org.eclipse.yasson.internal.serializer.AbstractContainerSerializer.serialize(AbstractContainerSerializer.java:107)
	at org.eclipse.yasson.internal.serializer.AbstractContainerSerializer.serializerCaptor(AbstractContainerSerializer.java:125)
	at org.eclipse.yasson.internal.serializer.ObjectSerializer.marshallProperty(ObjectSerializer.java:121)
	at org.eclipse.yasson.internal.serializer.ObjectSerializer.serializeInternal(ObjectSerializer.java:69)
	... 260 more
Caused by: javax.json.bind.JsonbException: Error getting value on: io.hyperfoil.tools.repo.entity.json.View@572f0a22
	at org.eclipse.yasson.internal.model.ReflectionPropagation.getValue(ReflectionPropagation.java:112)
	at org.eclipse.yasson.internal.model.PropertyModel.getValue(PropertyModel.java:342)
	at org.eclipse.yasson.internal.serializer.ObjectSerializer.marshallProperty(ObjectSerializer.java:97)
	at org.eclipse.yasson.internal.serializer.ObjectSerializer.serializeInternal(ObjectSerializer.java:69)
	... 264 more
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.yasson.internal.model.ReflectionPropagation.lambda$acceptMethod$0(ReflectionPropagation.java:51)
	at org.eclipse.yasson.internal.model.ReflectionPropagation.getValue(ReflectionPropagation.java:110)
	... 267 more
Caused by: org.hibernate.exception.SQLGrammarException: could not extract ResultSet
	at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:103)
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:67)
	at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.getResultSet(AbstractLoadPlanBasedLoader.java:390)
	at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeQueryStatement(AbstractLoadPlanBasedLoader.java:163)
	at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:104)
	at org.hibernate.loader.collection.plan.AbstractLoadPlanBasedCollectionInitializer.initialize(AbstractLoadPlanBasedCollectionInitializer.java:87)
	at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:710)
	at org.hibernate.event.internal.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:76)
	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:102)
	at org.hibernate.internal.SessionImpl.initializeCollection(SessionImpl.java:2153)
	at org.hibernate.collection.internal.AbstractPersistentCollection$4.doWork(AbstractPersistentCollection.java:589)
	at org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:264)
	at org.hibernate.collection.internal.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:585)
	at org.hibernate.collection.internal.AbstractPersistentCollection.read(AbstractPersistentCollection.java:149)
	at org.hibernate.collection.internal.AbstractPersistentCollection$1.doWork(AbstractPersistentCollection.java:178)
	at org.hibernate.collection.internal.AbstractPersistentCollection$1.doWork(AbstractPersistentCollection.java:163)
	at org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:264)
	at org.hibernate.collection.internal.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:162)
	at org.hibernate.collection.internal.PersistentBag.size(PersistentBag.java:371)
	at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.takeCollectionSizeSnapshot(LazyAttributeLoadingInterceptor.java:160)
	at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.lambda$loadAttribute$0(LazyAttributeLoadingInterceptor.java:110)
	at org.hibernate.bytecode.enhance.spi.interceptor.EnhancementHelper.performWork(EnhancementHelper.java:127)
	at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.loadAttribute(LazyAttributeLoadingInterceptor.java:76)
	at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.fetchAttribute(LazyAttributeLoadingInterceptor.java:72)
	at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.handleRead(LazyAttributeLoadingInterceptor.java:53)
	at org.hibernate.bytecode.enhance.spi.interceptor.AbstractInterceptor.readObject(AbstractInterceptor.java:153)
	at io.hyperfoil.tools.repo.entity.json.View.$$_hibernate_read_components(View.java)
	at io.hyperfoil.tools.repo.entity.json.View.getComponents(View.java)
	... 273 more
Caused by: org.postgresql.util.PSQLException: ERROR: relation "viewcomponent" does not exist
  Position: 314
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2533)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2268)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:313)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:159)
	at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:109)
	at io.agroal.pool.wrapper.PreparedStatementWrapper.executeQuery(PreparedStatementWrapper.java:75)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:57)
	... 299 more

New schema save fails

Saving a newly defined schema fails with the following constraint violations, even though the failed fields are supplied;

javax.validation.ConstraintViolationException: Validation failed for classes [io.hyperfoil.tools.repo.entity.json.Schema] during persist time for groups [javax.validation.groups.Default, ]
List of constraint violations:[
	ConstraintViolationImpl{interpolatedMessage='must not be null', propertyPath=owner, rootBeanClass=class io.hyperfoil.tools.repo.entity.json.Schema, messageTemplate='{javax.validation.constraints.NotNull.message}'}
	ConstraintViolationImpl{interpolatedMessage='must not be null', propertyPath=access, rootBeanClass=class io.hyperfoil.tools.repo.entity.json.Schema, messageTemplate='{javax.validation.constraints.NotNull.message}'}
]

Http Request:

POST /api/schema HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 168
Accept: application/json
Sec-Fetch-Dest: empty
Authorization: Bearer {BEARER_TOKEN}
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36
Content-Type: application/json
Origin: http://localhost:8080
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Referer: http://localhost:8080/schema/_new
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8,fr;q=0.7
Cookie: G_AUTHUSER_H=0

Payload:

{"name":"quarkus-quickstart","uri":"http://quarkus.io/startup-schema/0.1","description":"Quarkus quickstart schema","schema":false,"access":"PUBLIC","owner":"dev-team"}

The schema defined in the JSON Schema text area is also not sent in the payload to POST /api/schema

Constraint Violations not displayed to user

If a user tries to create a new Schema that fails constraint violations, the user is shown a "Failed to save the Schema" message

Any constraint violations are returned from the server in the HTTP 500 response e.g.

javax.validation.ConstraintViolationException: Validation failed for classes [io.hyperfoil.tools.repo.entity.json.Schema] during persist time for groups [javax.validation.groups.Default, ]
List of constraint violations:[
	ConstraintViolationImpl{interpolatedMessage='must not be null', propertyPath=access, rootBeanClass=class io.hyperfoil.tools.repo.entity.json.Schema, messageTemplate='{javax.validation.constraints.NotNull.message}'}
	ConstraintViolationImpl{interpolatedMessage='must not be null', propertyPath=owner, rootBeanClass=class io.hyperfoil.tools.repo.entity.json.Schema, messageTemplate='{javax.validation.constraints.NotNull.message}'}
]

These should be displayed in a meaningful way to the user.

Shouldn't the response be a HTTP 400 Bad Request, instead on Internal Server error?

Rest API to get multiple benchmarks

Export templates

Use case: Horreum collects data from runs, I'd like to manually process them in spreadsheets. It would be useful to store export templates that will select the data and store them in CSV (instead of having that as jq scripts locally).

Add code completion to build JSON path

Dashboard for each benchmark schema

Update JVM docker image to use ubi8.2+openJDK11

Upgrade JVM dockerfile to use supported ubi8.2 image

Validate schema extractor syntax

Just check SELECT jsonb_path_query_first('{}', path) to see if the syntax is correct when saving the extractor.

Schema UI to add schemas for benchmark

Deleting runs

I am still split on whether we should add the option to delete a run. On one hand we want to keep what we had and prevent accidental removal (like with git history), on the other hand I often have a run that crashed or was cancelled and pollutes the run history.

Should we add an option to trash it which just won't display it (flagging the database row(s)) and extra option to display trashed runs, too? Trashed runs won't pollute UI and probably won't affect load times, but these will still occupy the DB. Maybe an admin-only option to delete everything trashed older than X months?

WDY'all think?

Test should support multiple schemas

Right now the data model assumes 1:1 relation between test and schemas. I was told this is one pain-point of PerfRepo as the schema needs to evolve over time, still being able to do the same comparison between tests with old schema and new one (as long as the evolution is compatible).

NullPointerException accessing /api/run/{id}/structure

An NPE is thrown if the rest endpoint /api/run/{id}/structure is invoked

java.lang.NullPointerException
	at io.hyperfoil.tools.yaup.json.Json.lambda$typeStructure$20(Json.java:969)
	at io.hyperfoil.tools.yaup.json.Json.lambda$forEach$27(Json.java:1403)
	at java.base/java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:671)
	at io.hyperfoil.tools.yaup.json.Json.forEach(Json.java:1403)
	at io.hyperfoil.tools.yaup.json.Json.typeStructure(Json.java:948)
	at io.hyperfoil.tools.yaup.json.Json.lambda$typeStructure$20(Json.java:950)
	at io.hyperfoil.tools.yaup.json.Json.lambda$forEach$27(Json.java:1403)
	at java.base/java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:671)
	at io.hyperfoil.tools.yaup.json.Json.forEach(Json.java:1403)
	at io.hyperfoil.tools.yaup.json.Json.typeStructure(Json.java:948)
	at io.hyperfoil.tools.yaup.json.Json.lambda$typeStructure$20(Json.java:950)
	at io.hyperfoil.tools.yaup.json.Json.lambda$forEach$27(Json.java:1403)
	at java.base/java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:671)
	at io.hyperfoil.tools.yaup.json.Json.forEach(Json.java:1403)
	at io.hyperfoil.tools.yaup.json.Json.typeStructure(Json.java:948)
	at io.hyperfoil.tools.repo.api.RunService.getStructure(RunService.java:213)
	at io.hyperfoil.tools.repo.api.RunService_Subclass.getStructure$$superaccessor8(RunService_Subclass.zig:1879)
	at io.hyperfoil.tools.repo.api.RunService_Subclass$$function$$8.apply(RunService_Subclass$$function$$8.zig:41)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
	at io.quarkus.security.runtime.interceptor.SecurityHandler.handle(SecurityHandler.java:24)
	at io.quarkus.security.runtime.interceptor.PermitAllInterceptor.intercept(PermitAllInterceptor.java:23)
	at io.quarkus.security.runtime.interceptor.PermitAllInterceptor_Bean.intercept(PermitAllInterceptor_Bean.zig:264)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
	at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
	at io.hyperfoil.tools.repo.api.RunService_Subclass.getStructure(RunService_Subclass.zig:350)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:130)
	at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:621)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:487)
	at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:437)
	at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:362)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:439)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:400)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:374)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:67)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:488)
	at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:259)
	at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:160)
	at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:362)
	at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:163)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:245)
	at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:249)
	at io.quarkus.resteasy.runtime.ResteasyFilter$ResteasyResponseWrapper.sendError(ResteasyFilter.java:65)
	at io.undertow.servlet.handlers.DefaultServlet.doGet(DefaultServlet.java:172)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:503)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
	at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
	at io.quarkus.resteasy.runtime.ResteasyFilter.doFilter(ResteasyFilter.java:28)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at io.hyperfoil.tools.repo.server.RouteFilter.doFilter(RouteFilter.java:26)
	at io.hyperfoil.tools.repo.server.RouteFilter_ClientProxy.doFilter(RouteFilter_ClientProxy.zig:430)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
	at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:63)
	at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
	at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
	at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:67)
	at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:133)
	at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
	at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:65)
	at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
	at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
	at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
	at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:247)
	at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:56)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:111)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:108)
	at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
	at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
	at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$10$1$1.call(UndertowDeploymentRecorder.java:563)
	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:227)
	at io.undertow.servlet.handlers.ServletInitialHandler.handleRequest(ServletInitialHandler.java:152)
	at io.undertow.server.handlers.HttpContinueReadHandler.handleRequest(HttpContinueReadHandler.java:43)
	at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$1.handleRequest(UndertowDeploymentRecorder.java:113)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:290)
	at io.undertow.server.DefaultExchangeHandler.handle(DefaultExchangeHandler.java:18)
	at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$6$1.run(UndertowDeploymentRecorder.java:396)
	at io.quarkus.runtime.CleanableExecutor$CleaningRunnable.run(CleanableExecutor.java:231)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2046)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1578)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at org.jboss.threads.JBossThread.run(JBossThread.java:479)
Resulted in: org.jboss.resteasy.spi.UnhandledException: java.lang.NullPointerException
	at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:106)
	at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:372)
	at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:216)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:515)
	... 60 more

Deleting test

When deleting the test it doesn't cascade view deletion correctly and DB error is thrown.
Actually the cascading error is due to view deletion.

• we should have a dialog confirming that where you type the number of runs.

Horreum gui errors with clean install

From a fresh build of Horreum, the gui fails with the following exception in the browser;

Uncaught Error: Duplicate columns were found with ids: "id" in the columns array above
    at useTable (react-table.development.js:1139)
    at Table (Table.js:6)
    at renderWithHooks (react-dom.development.js:14796)
    at mountIndeterminateComponent (react-dom.development.js:17408)
    at beginWork (react-dom.development.js:18486)
    at HTMLUnknownElement.callCallback (react-dom.development.js:189)
    at Object.invokeGuardedCallbackDev (react-dom.development.js:238)
    at invokeGuardedCallback (react-dom.development.js:291)
    at beginWork$1 (react-dom.development.js:23052)
    at performUnitOfWork (react-dom.development.js:22016)
    at workLoopSync (react-dom.development.js:21992)
    at performSyncWorkOnRoot (react-dom.development.js:21610)
    at scheduleUpdateOnFiber (react-dom.development.js:21042)
    at updateContainer (react-dom.development.js:24191)
    at react-dom.development.js:24574
    at unbatchedUpdates (react-dom.development.js:21760)
    at legacyRenderSubtreeIntoContainer (react-dom.development.js:24573)
    at Object.render (react-dom.development.js:24656)
    at Module.<anonymous> (index.js:12)
    at Module../src/index.js (index.js:23)
    at __webpack_require__ (bootstrap:785)
    at fn (bootstrap:150)
    at Object.0 (utils.js:7)
    at __webpack_require__ (bootstrap:785)
    at checkDeferredModules (bootstrap:45)
    at Array.webpackJsonpCallback [as push] (bootstrap:32)
    at main.chunk.js:1

It appears to be caused by referencing id here;

https://github.com/Hyperfoil/Horreum/blob/master/repo/webapp/src/domain/tests/AllTests.js#L27

and here;

https://github.com/Hyperfoil/Horreum/blob/master/repo/webapp/src/domain/tests/AllTests.js#L49

Rename test to benchmark

Improve documentation for running standalone

When running Horreum on bare metal or in docker, the Keycloak realm for Horreum needs to be updated;

Horreum-ui client needs the following updating;

• Root URL
• Valid Redirect URIs
• Admin URL
• Web Origins

The documentation needs updating to reflect this

Do not expose SqlService

The SqlService should not be a public API

Wrong message in Series

When the test has multiple tags, the big message says that there are no regression variables (wrong).

When the data is loading it says 'no datapoints in the range' rather than showing spinner.

Missing runs w/o tags in AllRuns

Error logging for calculations on server

When the regression variables calculation fails on server side there's no easy way to figure out what happened unless the user has access to server logs (should not have).

We probably need to log the errors to some other log and make that available through UI.

Guidelines, how you structure data/JSON

Updating descriptions

While the run data itself should be immutable, sometimes I forget to add a description to the run (configuration parameters for the SUT) and it would be nice to update it afterwards. However the description is just one JSONPath-fetched piece of data from the immutable run data.

I think that setting up a generic way to update certain piece of the data is excessive. I would rather:

1. Add a description column to the Run itself
2. Add an optional jsonpath field to Schema that will allow to fetch the Run data
3. Add a column to all run views that will display this description, an option to edit it

That way the original description stays the same but you could overwrite it. The downside is that this won't be propagated to anyone who fetches the data using /api/run/123456/data but I don't think it's a big issue.

Endpoint to upload HF all.json

Automatically extract start,stop from $.info.startTime and $.info.terminateTime
Find test by name = $.info.benchmark or create a test?

Implement security with keycloak

We need to be able to identify users. We need security integration with keycloak as described in: https://quarkus.io/guides/security-keycloak-authorization

Series: Add timespan and position to URL

...to be able to send the current chart to someone else.

Maybe add a copy-link-to-clipboard button, too.

refactoring module layout of project

The Horreum project is currently configured as a Maven multi-module project, with a single module repo. Originally this POC was to contain a number of different modules that consumed data from Horreum (e.g. Hyperfoil reporting tool), but projects consuming data from Horreum are being developed in their own repos.

I think it makes sense to just have Horreum as a single module project. Wanted to discuss this so that it does not interfere with current dev branches pple might have

@willr3 @rvansa thoughts?

Depend on a published version of yaup

Need to publish yaup and to remove snapshot dependency

Runs without a schema

When we try to recalculate datapoints in a test and

1. some runs don't map to any schema
2. or accessors can't fetch data from a subset of runs

this should be visually indicated (especially (1)).

Filter /run/list table by tag

It is possible to assign tag(s) to runs. The run/list view displays all runs for a particular test

e.g. https://benchserver1.perf.lab.eng.rdu2.redhat.com:8443/run/list/13

It would be convenient for the user to be able to filter the runs listed in run/list table by tags

Hide schema JSON when not used

JSON schema validation is an optional part of the schema concept, but in UI it's very apparent and can confuse users. We should rather hide it when it's null and add it using a button that will scaffold it (the $schema and $id parts, permissive).

Create a json-schema from a Run

provide a json-schema that is valid for a select Run

Extraction during upload

Extracting test name, start & stop times etc. works only for top-level schema.

Series UI

Data not reloaded on timespan change

Change detection improvements

A pseudo-issue tracking ideas for improvements in regression monitoring

Unable remove invalid schema

Cannot remove schema with invalid URI

Internal Server error
{
"details": "Error invoking subclass methodError id 7e370d55-57cd-4254-9429-aae1c81ef4ec-9",
"stack": ""
}

Steps to reproduce:

1. Go to Login -> Schemas -> New Schema
2. Fill in URI "test"
3. Press "Save"
4. Press ...
5. Press "Delete" x2

Rebase on UBI8 image?

I was able to rebase on UBI8 with this Dockerfile:

FROM registry.access.redhat.com/ubi8/openjdk-11
USER root
RUN dnf install -y http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/centos-gpg-keys-8.1-1.1911.0.9.el8.noarch.rpm && \
    curl -s http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/centos-repos-8.1-1.1911.0.9.el8.x86_64.rpm -o centos-repos.rpm && \  
    rpm2archive centos-repos.rpm && tar xzf centos-repos.rpm.tgz -C / && rm centos-repos.rpm centos-repos.rpm.tgz
RUN dnf install -y jq && dnf clean all
ENV JAVA_OPTIONS="-Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
ENV AB_ENABLED=jmx_exporter
COPY target/lib/* /deployments/lib/
COPY target/*-runner.jar /deployments/app.jar
COPY src/main/resources/keycloak-horreum.json /deployments/imports/
COPY src/main/resources/structure.sql /deployments/imports/
COPY src/main/resources/auxiliary.sql /deployments/imports/
COPY src/main/resources/policies.sql /deployments/imports/
EXPOSE 8080

# run with user 1001 and be prepared for be running in OpenShift too
RUN adduser -G root --no-create-home -u 1001 horreum \
  && chown -R 1001 /deployments \
  && chmod -R "g+rwX" /deployments \
  && chown -R 1001:root /deployments
USER 1001

ENTRYPOINT [ "/opt/run-java/run/run-java.sh" ]

However the resulting image size (760 MB) is much higher than the one based on alpine (387 MB) - though I doubt these numbers are correct (docker history shows 98 MB for adduser ... && chmod/chown ...) I still don't know if it's worth it.

Alerting for missing runs

When a run is supposed to be executed nightly, missing the results should trigger an alert.

Test matrix view

In some cases all runs from single test can't be treated as equal; some runs might be flavours of the test with different parameters. At the same time the execution infrastructure (CI) might not allow to upload all flavours as single Run; it might not be convenient either to define all the schema extractors for each variant.

For such setup we might want to extract all variables (performance counters) from given test and display the difference in a matrix (as a chart).

storage - alerting coupling

This is a pseudoissue that summarizes point in which horreum storage and alerting are tighly coupled. It's rather a place for discussion than a bug.

Let me first define some terms:

• variable is basically a function that gets some data from the run and produces a scalar value
• datapoint is the scalar value from above with metadata such as original run and variable
• criterion is a condition on sequence of datapoints. The criterion can determine that there has been a significant change in the sequence, emitting a change
• change is a datapoint that seems to indicate a regression (or improvement)

In the relational model we try to avoid linking entities belonging to either service, although it might be actually implicitly coupled, just without proper @OneToOne referential integrity. This may lead to leaks of some unreachable entities and suboptimal performance.

Test: this is a central element of scoping the runs; any variables and criteria defined for alerting belong to a test. Current POC references test IDs.

Schema and schemaextractors: it's useful to reuse schemaextractors from views for alerting, too. Were the application monolithic we could use similar process to view data (database triggers on run insertion would extract info similar to view_data table), but here we'll probably fetch the extractors explicitly. Technically easier solution is to get them from the database directly, though we can fetch them from storage using REST API, too.

Ownership and access rights: the most convenient solution is to implicitly assume that variables, criteria and observed regressions are owned by the test owner, while data points derived from run data are owned by the run owner (datapoint reference run ID). However the access settings would not be directly reachable from alerting should these two be decoupled, using separate database.

Changing variables: when the alerting does not store runs (nor fetch them from DB directly) it cannot re-evaluate variables into datapoints (or it does have to explicitly fetch the runs again from the storage).

Anonymous access to test fails due to hooks

Retrieving hooks for given test

Horreum/src/main/java/io/hyperfoil/tools/horreum/api/HookService.java

Lines 230 to 241 in 31b1071

	@RolesAllowed(Roles.ADMIN)
	@GET
	@Path("test/{id}")
	public List<Hook> variables(@PathParam("id") Integer testId) {
	try (@SuppressWarnings("unused") CloseMe closeMe = sqlService.withRoles(em, identity)) {
	if (testId != null) {
	return Hook.list("target", testId);
	} else {
	return Variable.listAll();
	}
	}
	}

is limited to ADMIN role. Is that intended @johnaohara ? Alternatively, do we want a custom priviledge to set that?

The current problem is that this opening test by anonymous user results in an error message being shown rather than hiding/graying out the webhooks section.

Btw. it seems that the code was copied from elsewhere but not fully finished (method name, else branch returning variables...)

Can not run dev mode for node.js and quarkus with latest quarkus release

see: 4452b2c