Clear gauges after flush...? about statsd

[recursify]

Similar... counters sum values within the same flush interval. They also write a value of "0" if no data is received within a flush interval. This is reasonable for a true counter, as zero new "things" were counted in that interval. The "Keep Last Value" function doesn't work when you start writing zeros in for no data (it expects None values).

Gauges and counters flush something to graphite, regardless if data was received for that stat (last value seen and zero, respectively). I'm looking for a dumber, more direct access to graphite.... if no data is seen for an interval, I don't want statsd to write anything.

My argument for clearing gauges after the flush interval is that it seems like you can get all the "keep and hold last value" functionality through graphite, so why do it in statsd?

You're also losing information by writing the last value to graphite. Take the example of logging CPU utilization. If a machine suddenly dies and stops sending metrics, you would have no idea. By only writing values when data is received, you could determine (ie: through the graphite json API) "this machine hasn't updated it's CPU utilization in a while... something is up".

[mrtazz]

Wouldn't this make them practically identical to counters?

[recursify]

Although I'm only considering the Graphite backend... not sure if other backends provide the "keep last value" functionality.

[Dieterbe]

if I understand this correctly, this can be paraphrased as:

• only submit the gauge value once, on each update, because that's really the only thing needed. (with the backends we have now)
• after submitting, reset (forget about) the gauge value, because given the above, statsd doesn't need to remember the value.

I agree with the above (and personally I wouldn't worry about other backends that are "dumb", we don't even know which other backends we would support, let alone how well they work. we can always add a "cache_gauges" or "explicit_gauges" config option that reintroduces this behavior if we ever need it. In fact, the backend plugin could cache and explicitly resubmit the values itself)

That said, I would like to point to #84 , I think your gauge behavior is the same as the "raw" type in that one. and "raw" is probably the better name for this kind of thing.

[recursify]

That's correct. And yes, it looks like the "raw" type is what I'm describing and is probably a more suitable name.

[Dieterbe]

I've thought a bit about it further and realized gauge is the better name. "raw" merely denotes the implementation detail for the graphite backend, if we would have this "raw" type and no "gauge" type, and we would then want to implement this type for a new backend where the implementation is a bit more involved than simple passing on the value (such as periodically writing the same value if the backend doesn't support "keep last value") the name "raw" would not make sense anymore. So I think the better thing to do is keep the "gauge" name and just implement it like how you described, and forget about the "raw" type because it's basically a gauge implemented properly but with a worse (backend-specific) name

[mrzor]

Is there a way to achieve this without using @recursify branch?

The librato backend would also benefit from this improvement, an example of another backend which is not "dumb" about its (possibly reset) inputs.

[brookemckim]

Would love to use this for graphing deploys and using drawAsInfinite. Right now I am using a counter which will graph no deploy as 0. I would rather have it be null.

[draco2003]

Closing this issue since I believe you can get this functionality by enabling the recent deleteGauges config option.

[recursify]

Great, thanks!

[obfuscurity]

TIL that this was the default behavior in StatsD gauges, and it made me 😿. I'm probably not stating anything that hasn't been said before, but to me, gauges represent a rate of change at a point in time. It makes zero sense imho for that to be artificially replicated in the next flush. Not only does this misrepresent what's actually being reported (or not being reported) to the collector, it completely misleads people into believing their running sum is greater than it actually is (e.g. with Graphite's integral).

I don't expect this will change soon; nor should it. If there's one thing I hate almost as much as bad data, it's introducing behavioral regressions for anyone who understands this is the Way Things Are™ and uses it appropriately. Perhaps this should change, but with a major release and blinky tags highlighting it in the release notes.

My $0.02.