joakimkistowski / http-load-generator Goto Github PK

Script: 5 requests per second ==> all 5 sent at once each second

If the HTTP-Load-Generator is called in Director mode without the argument for the power meter IPs (--power), it fails with a NullPointerException.

The DirectorCommand class initializes the powerAddresses as null. The variable is passed to the Director's executeDirector method. executeDirector calls split on the passed variable without null check, causing a NullPointerException in line 82 of the Director.java.

The call resulting in the error is:
java -jar httploadgenerator.jar director --ip 10.1.3.37 --load static.csv -o log.csv --lua LU.lua --power "" --timeout 5000 --threads 256
The load intensity is constant for 180s at 1350 and the LUA script only calls a single REST endpoint.

Incase no name for the outputfile is provided, the name of the lua script and the name of the csv file with the load could be used.

Starting the director with the loadgenerator running leads to errors on some configurations
Jun 22, 2018 3:29:15 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator acceptDirector INFO: Waiting for director. Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator acceptDirector INFO: Director connected. Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.ArrivalRateTupleLoadGenerator readLoadProfile INFO: Receiving 7230 Arrival Rates. Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.ArrivalRateTupleLoadGenerator readLoadProfile INFO: Received 7230 Arrival Rate Tuples Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator waitAndListen SEVERE: Read Failed Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator waitAndListen SEVERE: Unknown Command: --[[ Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator waitAndListen SEVERE: Unknown Command: petsupply_browse profile. Sends cyclical requests to webui instance or front-end loadbalancer. Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator waitAndListen SEVERE: Unknown Command: Browses store and updates shopping carts. Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator waitAndListen SEVERE: Unknown Command: Does not perform any actions that would change the database. Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator waitAndListen SEVERE: Unknown Command: --]] Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator waitAndListen SEVERE: Unknown Command: Jun 22, 2018 3:30:00 PM tools.descartes.dlim.httploadgenerator.generator.AbstractLoadGenerator waitAndListen SEVERE: Unknown Command: --[[ ...
It seems the loadgenerator fails to read the IRunnerConstants.SCRIPT_SEND_KEY proceeding the lua script and thus fails to read in the lua file.
The exact command with different loadgenerator addresses works on another configuration. The binary was download again but the error is still happening. Both loadgenerator machines run the same openjdk version "1.8.0_171" java version.

After receiving too many failed transactions, the load generator does not terminate without sending additional requests

I've successfully installed TeaStore and done some benchmark tests using HTTP-Load-Generator; no problems. However, I came accross a peculiar case where I couldn't understand the mechanism of accumulated transactions.

In the following scenario, I've delibaretly decreased the capacity of teastore-webui to the extend that TeaStore is still functional but successfull transactions are generally lower than the Load Intensity. In my tests, Director does not report Failed or Dropped transactions. But I assume that the transactions should start accumulate and I should observe some failed or dropped transactions after some time. But I couldn't find evidence (number of increasing threads, increasing memory etc) of this accumulation. In other words: where exactly are those "waiting" transactions are waiting? I checked "netstat -pantul": the number of connections is always 128. I checked the stack trace of java process in the generator: it stays fixed around 1000. The number of threads ("ps -eLf") of the generator is also not increasing.

Director output
java -jar httploadgenerator.jar director --ip 10.27.44.100 --load constantLoad_100tps.csv --lua teastore_browse.lua Here is the output.

Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director executeDirector
WARNING: No power measurements
Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director executeDirector
INFO: Load Generator Thread Count set to 128
Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director executeDirector
INFO: URL connection timout set to 0 ms
Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director executeDirector
INFO: Using Lua Script: teastore_browse.lua
Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director process
INFO: Read 86401 Arrival Rate Tuples
Load Generator sent: ok
Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director process
INFO: Arrival Rates sent to Load Generator(s).
Load Generator sent: ok
Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director process
INFO: Thread Count sent to Load Generator(s): 128
Load Generator sent: ok
Load Generator sent: ok
Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director process
INFO: Contents of script sent to Load Generator: teastore_browse.lua
Jan 03, 2022 10:03:35 AM tools.descartes.dlim.httploadgenerator.runner.Director process
INFO: Starting Load Generation
Beginning Run @1641193415722(03.01.2022;10:03:35722)
Target Time = 0.5; Load Intensity = 100.0; #Success = 14; #Failed = 0; #Dropped = 0
Target Time = 1.5; Load Intensity = 100.0; #Success = 184; #Failed = 0; #Dropped = 0
Target Time = 2.5; Load Intensity = 100.0; #Success = 101; #Failed = 0; #Dropped = 0
Target Time = 3.5; Load Intensity = 100.0; #Success = 81; #Failed = 0; #Dropped = 0
Target Time = 4.5; Load Intensity = 100.0; #Success = 67; #Failed = 0; #Dropped = 0
Target Time = 5.5; Load Intensity = 100.0; #Success = 55; #Failed = 0; #Dropped = 0
Target Time = 6.5; Load Intensity = 100.0; #Success = 26; #Failed = 0; #Dropped = 0
 ...
Target Time = 6586.5; Load Intensity = 100.0; #Success = 80; #Failed = 0; #Dropped = 0
Target Time = 6587.5; Load Intensity = 100.0; #Success = 113; #Failed = 0; #Dropped = 0
Target Time = 6588.5; Load Intensity = 100.0; #Success = 91; #Failed = 0; #Dropped = 0
Target Time = 6589.5; Load Intensity = 100.0; #Success = 62; #Failed = 0; #Dropped = 0
Target Time = 6590.5; Load Intensity = 100.0; #Success = 99; #Failed = 0; #Dropped = 0
Target Time = 6591.5; Load Intensity = 100.0; #Success = 75; #Failed = 0; #Dropped = 0
Target Time = 6592.5; Load Intensity = 100.0; #Success = 95; #Failed = 0; #Dropped = 0
Target Time = 6593.5; Load Intensity = 100.0; #Success = 101; #Failed = 0; #Dropped = 0
Target Time = 6594.5; Load Intensity = 100.0; #Success = 87; #Failed = 0; #Dropped = 0
Target Time = 6595.5; Load Intensity = 100.0; #Success = 80; #Failed = 0; #Dropped = 0
Target Time = 6596.5; Load Intensity = 100.0; #Success = 107; #Failed = 0; #Dropped = 0

Director output (visualized)
I've also visualized the previous output. "Waiting" refers to the cumulated sum of (load intensity-success-failed-dropped). As you see the successfull transactions oscillate around 100 but it is generally below zero. There is no failed or dropped transactions reported. So there should be some transactions waiting somewhere. And this "waiting" transactions are increasing linearly with time.

Director Side

Generator Side
java -jar httploadgenerator.jar loadgenerator

constantLoad_100tps.csv

0.5,100
1.5,100
2.5,100
3.5,100
...
86397.5,100
86398.5,100
86399.5,100
86400.5,100

teastore_browse.lua
I've only changed prefix = "http://teastore.apps.company/tools.descartes.teastore.webui/"

teastore-webui deployment
Here is the output of oc get deployment/teastore-webui -o yaml:
Please note that I deliberately set CATALINA_OPTS=-Xmx300M, replicas=1, and cpu limit to 500m to reduce the capacity of the teastore-webui container.

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "719"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"name":"teastore-webui","namespace":"teastore"},"spec":{"replicas":2,"selector":{"matchLabels":{"app":"teastore","run":"teastore-webui"}},"template":{"metadata":{"labels":{"app":"teastore","run":"teastore-webui"}},"spec":{"containers":[{"env":[{"name":"USE_POD_IP","value":"true"},{"name":"REGISTRY_HOST","value":"teastore-registry"},{"name":"RABBITMQ_HOST","value":"teastore-kieker-rabbitmq"},{"name":"CATALINA_OPTS","value":"-Xmx500M"}],"image":"default-route-openshift-image-registry.apps.company/teastore/teastore-webui","name":"teastore-webui","ports":[{"containerPort":8080}],"resources":{"limits":{"cpu":"500m"}}}]}}}}
  creationTimestamp: "2021-07-25T17:54:56Z"
  generation: 796
  name: teastore-webui
  namespace: teastore
  resourceVersion: "65487249"
  selfLink: /apis/apps/v1/namespaces/teastore/deployments/teastore-webui
  uid: d8fa2d7d-673e-4e79-91ec-f7739c2a6a0a
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: teastore
      run: teastore-webui
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: teastore
        run: teastore-webui
    spec:
      containers:
      - env:
        - name: USE_POD_IP
          value: "true"
        - name: REGISTRY_HOST
          value: teastore-registry
        - name: RABBITMQ_HOST
          value: teastore-kieker-rabbitmq
        - name: CATALINA_OPTS
          value: -Xmx300M
        image: default-route-openshift-image-registry.apps.company/teastore/teastore-webui
        imagePullPolicy: Always
        name: teastore-webui
        ports:
        - containerPort: 8080
          protocol: TCP
        resources:
          limits:
            cpu: 500m
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2021-12-04T20:43:55Z"
    lastUpdateTime: "2021-12-04T20:43:55Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2021-07-25T17:54:56Z"
    lastUpdateTime: "2021-12-04T21:00:15Z"
    message: ReplicaSet "teastore-webui-6b5b98b79f" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 796
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1`

Versions

• OpenShift 4.7 with CoreOS

Hello,
I used your HTTP-Load-Generator with Tea-Store benchmark and it was easy to write the script since they provide examples.

Now I am trying to write scripts for another application and I am facing a problem. When I want to send POST request, it fails to send it. I checked their code and I think the reason is the Content-Type.
In the examples provided by you, you use url-encoded format to send the payload. The application ,that I want to generate load for, uses json format which means that I have to specify the Content-Type in the header request as application/json instead of application/x-www-form-urlencoded.

Is there any way to change or name the content-type for POST requests in your generator?

Trying to start director with
java -jar httploadgenerator.jar director --ip 10.1.3.23,10.1.3.16 ...
with loadgenerators delimited by comma as required, results in
Jun 22, 2018 3:19:45 PM tools.descartes.dlim.httploadgenerator.runner.Director executeDirector WARNING: No power measurements Jun 22, 2018 3:19:45 PM tools.descartes.dlim.httploadgenerator.runner.Director executeDirector INFO: Load Generator Thread Count set to 256 Jun 22, 2018 3:19:45 PM tools.descartes.dlim.httploadgenerator.runner.Director executeDirector INFO: URL connection timout set to 5000 ms Jun 22, 2018 3:19:45 PM tools.descartes.dlim.httploadgenerator.runner.Director executeDirector INFO: Using Lua Script: lua_tstore_script.lua Jun 22, 2018 3:19:46 PM tools.descartes.dlim.httploadgenerator.runner.LoadGeneratorCommunicator <init> SEVERE: Could not connect to LoadGenerator. Jun 22, 2018 3:19:46 PM tools.descartes.dlim.httploadgenerator.runner.Director process INFO: Read 7230 Arrival Rate Tuples Exception in thread "main" java.lang.NullPointerException at tools.descartes.dlim.httploadgenerator.runner.LoadGeneratorCommunicator.sendArrivalRates(LoadGeneratorCommunicator.java:102) at tools.descartes.dlim.httploadgenerator.runner.Director.lambda$process$1(Director.java:166) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291) at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401) at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734) at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583) at tools.descartes.dlim.httploadgenerator.runner.Director.process(Director.java:166) at tools.descartes.dlim.httploadgenerator.runner.Director.executeDirector(Director.java:108) at tools.descartes.dlim.httploadgenerator.runner.cli.DirectorCommand.run(DirectorCommand.java:113) at tools.descartes.dlim.httploadgenerator.runner.Main.main(Main.java:87)

When specifying a warmup rate of for example 500 with two load drivers, the resulting load intensity during warmup is 1000.

Exception in thread "main" java.lang.NullPointerException
at tools.descartes.dlim.httploadgenerator.runner.Director.executeDirector(Director.java:82)
at tools.descartes.dlim.httploadgenerator.runner.cli.DirectorCommand.run(DirectorCommand.java:114)
at tools.descartes.dlim.httploadgenerator.runner.Main.main(Main.java:87)

Avg Response Time for all the entries is 0 in the output log if Timeout flag is set during the execution of HTTP Load Generator