`influxdb2` keeps accumulating background threads about riemann

[jarpy]

Ah! Thanks for explaining. I was making assumptions based on my own experience, not yours. Sorry for the trouble.

[jarpy]

It's not super-obvious, but the docs for influxdb2 suggest that:

For performance, you should wrap this stream with batch or an asynchronous queue.

You can use the async queue to call influxdb2 only once. Below is a sketch based on how I do this for Logstash. Note that I have not tested this sketch (sorry), but I think the same principles will apply for your use-case.

;; Establish a dedicated queue and thread(s) for sending metrics to InfluxDB.
(let [influx-queue (async-queue!
                    :influx-output-queue {:queue-size 1000 :core-pool-size 1 :max-pool-size 1}
                    (influxdb2 {:host "influxdb.example.com"
                                :organization "riemann"
                                :bucket "riemann"
                                :token "riemann"}))]

  ;; We've now defined the queue. Importantly, we've done it only once, in the
  ;; let-binding. Now we can define a stream function that closes over the queue
  ;; so that we can send many events to the single queue instance.
  (defn send-to-influx
    "A stream for sending events to influxdb2 via an async queue."
    [& children]
    (fn [event]
      (call-rescue event (conj children influx-queue)))))

[node13h]

@jarpy thanks, but I've already got it wrapped in async-queue! once. The VisualVM screenshots above were taken while influxdb2 was wrapped in async-queue!.
The problem is not in influxdb2, but in post-data which keeps creating a new instances of WriteApi on each invocation.

My config:

influxdb-batch-sender (batch 100 1
    (async-queue! :agg {:queue-size 1000
                        :core-pool-size 2
                        :max-pool-size 16}
                  (influxdb2 influxdb-creds)))

EDIT: formatting, and clarifications.