leonadato / xk6-output-statsd

k6 extension to output real-time test metrics using StatsD.

xk6-output-statsd

Provides support for real-time output of k6 test metrics to a StatsD service.

Direct support for the StatsD metrics collector will be removed from k6 in a future release. For those seeking to continue using StatsD, the functionality may be compiled into custom k6 binaries as an xk6 extension.

Build

To build a k6 binary with this extension, first ensure you have the prerequisites:

• Go toolchain
• Git
• xk6

1. Build with xk6:

xk6 build --with github.com/LeonAdato/xk6-output-statsd

This will result in a k6 binary in the current directory.

2. Run with the just build `k6:

./k6 run -o output-statsd <script.js>

Options

The xk6-output-statsd extension supports the following configuration options:

Environment Variable	Default	Description
K6_STATSD_ADDR	localhost:8125	Address of the statsd service, currently only UDP is supported.
K6_STATSD_NAMESPACE	k6.	The namespace used as a prefix for all the metric names.
K6_STATSD_PUSH_INTERVAL	1s	Configure how often data batches are sent.
K6_STATSD_BUFFER_SIZE	20	The buffer size.
K6_STATSD_ENABLE_TAGS	false	If true enables sending tags. Versions prior to statsd v0.9.0 did not support tags.
K6_STATSD_TAG_BLOCKLIST	vu,iter,url	Comma-separated list of tags that should not be sent to statsd.

Options may be specified as system environment variables as with the following:

K6_STATSD_ADDR=localhost:8125 K6_STATSD_ENABLE_TAGS=true ./k6 run -o output-statsd examples/simple.js

Testing Locally

This repository includes a docker-compose.yml file that starts Graphite backed by StatsD which can be used for testing the xk6-output-statsd extension.

⚠️ This environment is intended for testing only and should not be used for production purposes.

1. Start the docker compose environment.

   docker compose up -d

   Output should appear similar to the following:

   ✔ Network xk6-output-statsd_default       Created               ...    0.0s
   ✔ Container xk6-output-statsd-graphite-1  Started               ...    0.3s

2. Use your custom k6 binary to run a k6 test script sending metrics to your StatsD service started ing the previous step.

   ./k6 run -o output-statsd examples/ramping-vus.js

3. Open the Graphite application by accessing http://localhost:80/ to view metrics from your test similar to the following:

xk6-output-statsd's Issues

Using an outdated version of k6

The current dependency is v0.45.1 with the latest version of k6 being v0.50.0

Is this going to be updated at any point? Between xk6, this extension and recent changes, k6 browser based tests are broken for me. (though I'm not suggesting an update with fix this)

Update modules for transfer of ownership

Ownership of the xk6-output-statsd extension has changed from @javaducky to @LeonAdato, just need to update some meta files given the change.

Metrics not delivering to CloudWatch: The parameter ... must be shorter than 1025 characters.

Hi,

I'm using K6 with this extension to delivery metrics to AWS CloudWatch Metrics via the CloudWatch agent.

I noticed that many of the metrics aren't delivery to CloudWatch (obvious when looking at the data on CloudWatch). I tested locally using Graphite, and here it is clear that all metrics are delivering.

I inspected the logs on the CloudWatch agent, and found that many requests are failing with HTTP 400 Bad Request errors. The specific errors look as follows:

<ErrorResponse xmlns=""http://monitoring.amazonaws.com/doc/2010-08-01/"">"
  <Error>
    <Type>Sender</Type>
    <Code>InvalidParameterValue</Code>
    <Message>The parameter MetricData.member.13.Dimensions.member.4.Value must be shorter than 1025 characters.</Message>
  </Error>
  <RequestId>12345</RequestId>
</ErrorResponse>

I realize that this is very specific to CloudWatch, but this extension is the officially recommended approach to delivery metrics to CloudWatch, and thus it feels most appropriate to report the issue here.

By setting the "aws_sdk_log_level" to "LogDebugWithHTTPBody" in the CloudWatch agent config, I am able to see the GZIP compressed Post body content for the failing requests, but I haven't been able to successfully decompress any of these, so I can't see the offending metric value.

Any suggestions regarding how I can further debug this, or potentially solve the issue?

Feature : provide a Prometheus dashboard

Hello,

I would like to have a Prometheus dashboard :)

k8s network errors

Hello! I'm use extenson in k6-operator with prometheus-statsd-exporter. In exporter running HPA. When test launched, everything works fine, but when resources run out and k8s creates additional exporters pods, traffic is not distributed to them. Apparently due to address caching. After deleting the exporters starting pod, I see a sending error in the logs. After this, traffic begins to be distributed among several pods, but not all.

datadog can not path filter like url path or status

Does the repo support the function about metrics filter by url path or status ？

Feature : provide a Docker image

Hello, could you please provide a Docker image ?

Porting a know issues from the k6 repository

What?

There are a couple of opened issues in the k6 repository related to the StatsD output:

• grafana/k6#2980
• grafana/k6#2819
• grafana/k6#2776
• grafana/k6#2549
• grafana/k6#2527
• grafana/k6#2470
• grafana/k6#2044

Since we likely won't work on them, keeping in mind the grafana/k6#2982 it's good to port (not sure of transfer could work) to this repository since currently this is the primarily recommended repository for the k6's statsd output.

@LeonAdato, do you think that makes sense? If so, I could port the mentioned issues to the repo and close them in the k6 reposory