octoperf / kraken

The Load Testing IDE

Kraken - The Load Testing IDE

Kraken is a load testing IDE based on Gatling.

As such, Kraken provides a complete development environment to software programmers and load testers that seek to make the most out of Gatling:

• A code editor to create and update .scala Gatling simulations with autocomplete suggestions and code snippets,
• Simulations debugging and comparison with HAR imports,
• Load tests analysis with aggregated data in InfluxDb displayed in comprehensive reports generated with Grafana,
• Multi-hosts load injection using a Kubernetes cluster.

Support and Community

Blog

Load Testing blog on OctoPerf

Tutorials

1. Gatling - Getting Started with Simulation Scripts
2. Gatling - Simulation Scripts Parameterization
3. Gatling - Loops, Conditions and Pauses

Email

Contact us at [email protected].

Development

Source code structure

The source code of Kraken is divided in several folders:

• backend: this folder contains a Gradle multi-modules project that build all Java backends of Kraken,
• deployment: private Git sub-module used to create and deploy all Docker containers,
• development: contains shell scripts and docker-compose configuration to start the InfluxDb/Grafana stack in dev mode,
• documentation: private Git sub-module that builds Kraken's documentation,
• frontend: this folder contains an Angular project with several libraries and two applications: administration and gatling.

Prerequisites

• Requires the make command sudo apt-get install build-essential
• Requires docker
• Requires docker compose
• Requires the JDK 11

Run the application from the source code

To run Kraken from the source code, execute one of the following commands from the root folder of the repository:

• make launch-docker to run the Docker version,
• make launch-k8s to run the Kubernetes version.

Open the Makefile to know in detail what each of these tasks does.

Tests and bug finders

Frontend

• ./test-coverage-all to generate test coverage for the complete project,
• make test APP=<library-or-app> to run tests for a single module (for example make test APP=commons),
• make lint to run TSLint on the whole project.

Backend

• make check to run all unit tests and SpotBugs,
• make test APP=<module-path> to run unit test for a single sub-module (for example make test APP=:commons:docker-client).

Roadmap

Check out the next milestone.

License

Kraken is an open core product:

• The majority of the code base is licensed under the Apache V2 License,
• Some closed-source components, not available directly in this repository, requires a license (free or paid) from OctoPerf.

kraken's Issues

CI Documentation

Add Swagger (or other tools https://dzone.com/articles/rest-api-documentation-generators-for-java) to all Java backends that expose a REST API.

Add CURL sample to call the REST API in order to:

• upload/download files
• start debug
• start test
• get gatling result
• get grafana result

uninstall.sh script should ask user if the data folder must be removed

Java Heap Space on HAR import

Must set the XMX to 512Mo.

It could be handy to pass the XMX as an environment variable for all Docker containers that run a Java application.

Improve Gatling's scripts auto-completion

Add auto-completion for :

• asJson
• headers
• Http Proxy configuration
• SSE
• WebSockets
• JMS

Use a Scala parser?

• https://tomassetti.me/parsing-in-javascript/#tools
• https://github.com/tunnelvisionlabs/antlr4ts
• https://github.com/antlr/grammars-v4/blob/master/scala/Scala.g4
  Does not work in the web browser: tunnelvisionlabs/antlr4ts#326

Use JS version instead?
https://github.com/antlr/antlr4/blob/master/doc/javascript-target.md

Install Antlr locally https://www.antlr.org/download.html
Generate the parser java -Xmx500M -cp antlr-4.7.2-complete.jar org.antlr.v4.Tool -Dlanguage=JavaScript Scala.g4
Use the genrated code
https://stackoverflow.com/questions/40463441/typescript-how-to-import-a-class-from-a-javascript-fil

Result table contextual menu

Must click twice on the (...) icon if the result is not selected to open the contextual menu.

Probably caused by the lazy loading of the menu needed to list Gatling reports.

Does it work using the right click ?

Update to Angular 8

https://blog.ninja-squad.com/2019/05/29/what-is-new-angular-8.0/

Task execution scheduler

Another backend to call task execution using cron jobs.

Move the telegraf.conf file into the gatling/runner folder

It is only used when running a load test. No need to have it in a dedicated folder.

Grafana dashboard export as PDF

https://community.grafana.com/t/automatic-sharing-of-dashboard-pdf-png-img-link-sent-to-email/2974/8
https://github.com/IzakMarais/reporter
https://github.com/kubernauts/jmeter-operator

Performance issues when too many nodes are opened in the files tree

The Virtual Scroll is probably a good solution.

Add navigation keyboard shortcuts for the file trees

• Up to select the previous file
• Shift + Up to add the previous file to the current selection
• Down to select the next file
• Shift + Down to add the next file to the current selection
• Enter to collapse/expand the current folder
• Right to select the next file or expand the current collapsed folder
• Left to select the previous file or collapse the current expanded folder
• CTRL + SUPPR to delete files without confirmation dialog
• Update the scrollbar position to follow the selection
• CTRL + Menu item click on delete to skip confirmation dialog
• Rename file / directory shortcut (F2)
• Launch run in tree and in the simulation editor (Ctrl + shift + X)
• Launch Debug in tree and in the simulation editor (Ctrl + shift + D)
• Add tooltip with shortcuts (rename, run, debug)
• Add scroll animation
• Add shortcuts to the documentation

Support Gatling extensions

Gatling extensions are installed in the lib folder.

• Map the gatling/lib/ folder in the docker-compose.yml,
• Add documentation: Jar files must be copied in this folder using the Administration application.

Display Gatling session from the DebugChunk editor

The session is only visible in the tooltip of an 'info' icon next to the request name.

Either add a text field in the editor or make the icon clickable (opens a dialog that shows the session value).

Document Gatling scripts code snippets usage

https://cloud9-sdk.readme.io/docs/snippets

Gatling 3.1

Update the Docker image octoperf/kraken-gatling-runner to use Gatling 3.1

New Gatling Simulation button/menu

• Design backend that generates a simulation Class given a name and package.
• UI button to call the backend.

The 'Execute Simulation' dialog sometimes does not close

After running several load test, clicking on the 'Run' button has no effect.

The test only starts when the dialog is closed by clicking on the backdrop.

Are these the only dialogs subject to this bug?

DebugEditor: add title to fields wth css class 'text-ellipsis'

The user has to expand the window to view the complete request URL for example.

Display backend containers in the administration UI

And allow for logs displaying.

K8S: Get all FlatContainers in the kraken namespace.
Docker: Get all FlatContainers on the local machine.

EventSource polyfills

EventSource is not supported in IE/Edge, making the frontend applications unusable on these web browsers.

We could use a library like EventSource Polyfill. The backend might have to be modified to support this.

TODO:

• SSEService to wrap Flux in Flux<SSE> with a keep alive
• Unit and integrations test SSEService
• Update StorageController + Unit tests
• Update CommandController + Unit tests
• Add Polyfills to the frontend
• Update HAProxy configuration (no 12h timeout) + manual tests
• Test on Windows / Edge

Debug entries generated when parsing a HAR are not in the proper order

The order is not the same as in the generated .scala script

Check for Java backend updates

backend only !

Tooltips are missing on Administration app buttons

Must also check the Gatling frontend

Logs panel scrollbar

The scrollbar does not always go to the bottom of the logs when a new log tab is displayed.

It seems to fail only if the Logs tab was hidden when the new log tab is added. Maybe firing the "open-logs-tab" even first will solve this issue.

We might as well add a button to go to the end of the logs on the upper right corner.

Debugging a Gatling script should set the injection policy

A warning message is displayed when we detect that the user load injection policy starts more than one iteration.

We may be able to replace the injection policy with .inject(atOnceUsers(1) automatically before starting the test. We would have to copy the script (or the whole user-file folder) into a temporary folder.

Add copy button to debug editor bodies

It allows the user to quickly copy the body of a request or response and paste it in an online regexp matcher for example.

Place it next to the Open Body in new browser tab button.

Update paths for DockerController and CommandController

remove the /command and /docker

Memory settings configuration

Handle environment variables for memory and CPU requests/limits:

• Docker: https://docs.docker.com/compose/compose-file/#resources
• Kubernetes: https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/#specify-a-memory-request-and-a-memory-limit

Set the Xmx accordingly https://developers.redhat.com/blog/2017/03/14/java-inside-docker/

Configuration in runtime's application.yml file for default values:

resources-allocation:
  cpu: 
    request: 0.5
    limit: 1.0 (optional)
  memory:
    request: 512M
    limit: 1G (optional)

Gatling 3.2.1

Update gatling images

Text editor focus

Clicking in a text editor (center pane) does not focus its tab automatically.

Update backend and frontend dependencies

• Fix ContextualMenuComponent unit tests
• Fix IconComponent unit tests
• RuntimeWebClientTest.shouldWaitForStatus

The following dependencies have later milestone versions:
 - com.github.ben-manes.versions:com.github.ben-manes.versions.gradle.plugin [0.21.0 -> 0.27.0]
 - com.github.spotbugs:com.github.spotbugs.gradle.plugin [2.0.1 -> 3.0.0]
 - com.google.guava:guava [28.0-jre -> 28.2-jre]
     https://github.com/google/guava
 - com.google.guava:guava-testlib [28.0-jre -> 28.2-jre]
     https://github.com/google/guava
 - com.gradle.build-scan:com.gradle.build-scan.gradle.plugin [2.3 -> 3.1.1]
 - com.squareup.okhttp3:mockwebserver [4.0.0 -> 4.4.0]
     https://square.github.io/okhttp/
 - io.kubernetes:client-java [6.0.1 -> 8.0.0-alpha1]
     https://github.com/kubernetes-client/java
 - io.methvin:directory-watcher [0.9.6 -> 0.9.9]
     https://github.com/gmethvin/directory-watcher
 - io.projectreactor:reactor-test [3.2.10.RELEASE -> 3.3.2.RELEASE]
     https://github.com/reactor/reactor-core
 - io.spring.dependency-management:io.spring.dependency-management.gradle.plugin [1.0.8.RELEASE -> 1.0.9.RELEASE]
 - nl.jqno.equalsverifier:equalsverifier [3.1.9 -> 3.1.12]
     http://www.jqno.nl/equalsverifier
 - org.apache.tika:tika-core [1.21 -> 1.23]
     http://tika.apache.org/
 - org.projectlombok:lombok [1.18.10 -> 1.18.12]
     https://projectlombok.org
 - org.springframework.boot:org.springframework.boot.gradle.plugin [2.1.6.RELEASE -> 2.2.4.RELEASE]
     https://projects.spring.io/spring-boot/#/spring-boot-parent/spring-boot-tools/org.springframework.boot.gradle.plugin
 - org.springframework.boot:spring-boot-devtools [2.1.6.RELEASE -> 2.2.4.RELEASE]
     https://projects.spring.io/spring-boot/#/spring-boot-parent/spring-boot-devtools
 - org.springframework.boot:spring-boot-gradle-plugin [2.1.6.RELEASE -> 2.2.4.RELEASE]
     https://projects.spring.io/spring-boot/#/spring-boot-parent/spring-boot-tools/spring-boot-gradle-plugin
 - org.springframework.boot:spring-boot-starter-test [2.1.6.RELEASE -> 2.2.4.RELEASE]
     https://projects.spring.io/spring-boot/#/spring-boot-parent/spring-boot-starters/spring-boot-starter-test
 - org.springframework.boot:spring-boot-starter-webflux [2.1.6.RELEASE -> 2.2.4.RELEASE]
     https://projects.spring.io/spring-boot/#/spring-boot-parent/spring-boot-starters/spring-boot-starter-webflux
 - org.zeroturnaround:zt-exec [1.10 -> 1.11]
     https://github.com/zeroturnaround/zt-exec
 - org.zeroturnaround:zt-zip [1.13 -> 1.14]
     https://github.com/zeroturnaround/zt-zip

Gradle release-candidate updates:
 - Gradle: [5.4.1 -> 6.2]

https://docs.gradle.org/current/userguide/upgrading_version_5.html

Document JavaOpts setting to set Gatling Xmx and pass env variables.

Provide more explanation regarding the JavaOpts parameter set when configuring a test run/debug.

-Xmx1G -Dfoo=bar

https://gatling.io/docs/3.0/cookbook/passing_parameters/

Administration - hosts table improvements

• Refresh button
• Add nodes dialog (return a list of available nodes), the user can select the ones he wants to add
• Rename button
• Detach button
• Add doc + openApi

HAR editor

Like:

http://www.softwareishard.com/blog/har-12-spec/
http://www.softwareishard.com/har/viewer/

Canceling a HAR Upload first step opens the second dialog

Uploading a HAR is divided in two steps (each done in a dialog):

1. Uploading the HAR file,
2. Converting it to a Gatling .scala file.

The second dialog is opened even if the first one is canceled.

Split the kraken-executor backend

Split the executor project into:

• docker-executor
• command-executor

Remove the docker-compose Spring controller and call the commands controller directly from the admin UI.

The DockerService will have to use the CommandClient to

• pull docker images
• purge the docker system

Testing tools support

• Lighthouse https://developers.google.com/web/tools/lighthouse
• Postman (Postman to HAR or direct execution of postman collections?)
• Protractor / Selenium How to handle metrics and results?
  • http://www.seleniumframework.com/advanced-tutorial/quick-performance-metrics/
  • https://www.npmjs.com/package/protractor-perf

Update README.md files for both frontend applications

They currently do not provide any useful information.

User load displays max value instead of sum

Use select field(active) sum() instead

Gatling script generator

UI like Postman / Insomnia.
It generates a scala script (Simulation) that Gatling can run.
Data stored in a yaml file.

Users management

• Analysis
  • Grafana integration
    • https://grafana.com/docs/grafana/latest/auth/generic-oauth/
    • https://community.grafana.com/t/setting-sso-for-grafana-via-keycloak-with-multiple-realms/16252
  • Influxdb integration
    • Oauth?
• Storage
  • One folder with defaults for each application (applications/gatling, applications/mkdocs, applications/hugo)
  • One folder for every user (users/user-id)
• Admin ui
  • Display applications default folders
  • Secure access to admin UI => check user role == ADMIN
• Gatling UI
  • Display hosts capacity/availability more clearly
  • Add a "files" panel that displays configuration files
• Runtime
  • Check host access/owner before executing a task
  • Add public/private host state (userId on host?)
• Keycloak
  • Webhook/listener
    • https://github.com/keycloak/keycloak-quickstarts/tree/latest/event-listener-sysout
    • https://github.com/keycloak/keycloak-quickstarts/tree/latest/event-store-mem
• Documentation
  • openapi update https://swagger.io/docs/specification/authentication/
  • curl examples update
• Security
  • Token decoder => validate the token https://auth0.com/docs/tokens/guides/validate-jwts
• SSE
  • Fix frontend configuration

Add buttons in the README.md files to open te embedded documentation

May be done by sending window events?

Right click on the files tree throws an exception

Happens when opening the contextual menu for the root node:

ERROR TypeError: Cannot read property 'path' of undefined

Kraken IDE not displaying in Mozilla Firefox

Describe the bug
After starting the containers, IDE is not getting properly displaying in Mozilla Firefox browser. Please check the screenshot.

To Reproduce
Steps to reproduce the behavior:

1. Launch Gatling IDE or Administrator UI in the latest version of Firefox

Expected behavior
IDE should launch properly.

Screenshots
Screenshot has been attached.

Handle multiple injectors for load tests

Not necessarily useful for debugging.

Solution 1 (OctoPerf like): an agent is started on each injector, it can start Gatling (internally or as a sibling Docker image ?).

Solution 2: use docker swarm or kubernetes

Compatibility with a simple Docker installation (no multi injectors) should be kept.

Git integration

Allow for scripts versioning using a Git integration.

Chose from:

• An integration in the Storage backend,
• A dedicated set of tasks to pull from/push to Git.

An integration directly in the storage backend might be easier to automate but harder to implement.

Where to store Git related information:

• In the user metadata in KeyCloak,
• In a .git folder (or another folder/file) along with the project files.

KeyCloak metadata would allow having this information directly on the connected user. But it will be complicated once we handle user groups.

Are there any security concerns with storing this info into the project files?

What about results data (influxdb) and dashboards (grafana)? It might be easier to save reports exported as pdf files.

Kraken cannot be installed in folder with special characters

For example, installing it in 'Téléchargements' will fail.

Embedded documentation points the the wrong page

In both frontend application, the embedded documentation points to the OctoPerf documentation.

It should point to kraken.octoper.com.

Also, actions made in the application (like starting a load test or opening a file editor), should update the documentation page.

Add keyboard shortcuts for the Result panel

All selectable tables

• Handle the focus on the Result table tabindex=0 (it is currently handled for storage trees, should we handle it at the tab level ?)
• Up to select the previous item
• Down to select the next item
• CTRL + Menu item click on delete to skip confirmation dialog

Result table

• SUPPR to remove the selected result
• CTRL + SUPPR to remove the selected result (no confirmation dialog)
• ENTER to open the selected result Grafana report (if available)

Debug table

• ENTER to open the comparison dialog
• Add loading state

Task table

• SUPPR to cancel/remove the selected task
• CTRL + SUPPR to cancel/remove the selected task (no confirmation dialog)

Add confirmation dialog to the test result deletion action

This operation cannot be reverted and removes quite a lot of data. Can be frustrating to unexpectedly remove a load test result and all associated data.

CTRL + click to skip the confirmation dialog?