Buffer repeated messages about monolog

[Seldaek]

I tend to think that if you have a high traffic site you're probably
better off just logging errors to files, then using another monitoring
tool to graph errors and email you in case the error rate spikes or
something.

Achieving this feature in monolog would make logging stuff a lot slower
IMO (though granted this could only be done in case an error actually
triggers, so hopefully not happen on most requests). Now if someone has
a sane idea and wants to do it, as long as it's optional I am not really
against it, but I am not sure the "sane" requirement can be met :)

[Seldaek]

https://github.com/Seldaek/monolog/blob/master/src/Monolog/Handler/DeduplicationHandler.php fixes it

[stof]

related to #34

[Seldaek]

This isn't a trivial problem because those messages span multiple requests, so one needs to filter those across requests.

[maraspin]

I second helix opinion.

Jordi, what do you think about the idea of some sort of "cache", with a key related to message, and value consisting on a count of occurrences, and the datetime when message was written first?

Such cachefile could be loaded at service startup and updated upon "cache expiration", accordingly to config settings (IE write to this handler only after N hours have passed since last written event). It would prevent much logging, and especially overwhelming situations (lots of file descriptors open, many e-mail connection attempts, ...) on high traffic websites.

What do you think?

[staabm]

couldn't this work, like the FingersCrossedHandler works?

[stof]

@staabm No, because the goal is to avoid repeating alert emails for separate web requests.

@Seldaek I also think using a tool like LogStash or LogEntries is a better way to achieve it than implementing this in Monolog

[maraspin]

@Seldaek Good points indeed. What I like about Monolog is being flexible, so that it can be used both in dev and production env. Hor scaling helps w/ load, so that having a local logger is not a concern (for now). And there are no additional tools to add; it can be used both on complex (= high traffic) architectures, and small websites. That helps a lot internally, with maintenance.

Don't know if you'd consider my idea sane enough, though. ;-) I was thinking about having tuples MSG_ID => (time,msg,severity) with MSG_ID being a key generated from monolog_channel_id + handler_id + message_content, with the aim of being unique. Then, within abstractHandler there'd be a ignore_within_mins variable. If null (default value) it'd work as it is now. Otherwise it would load proper key from "cache" (json, sqlite, redis, memcache, ...) and evaluate time last same message was sent against ignore_within_mins variable. If not matched, process message. Else return.

The idea is to have a setIgnoreCache method, accepting the threshold (minutes) and a cacheWriterHanddler, which could deal like a key/value reader (through a simple getKey/putKey interface) with the aforementioned caching systems. So, if a cacheWriteHandler is set within an handler, the aforementioned is taken into account, before message is considered. Otherwise it is not, and thinkgs continue to work as they do now. What do you think?

@stof Thanks for the hint about logstash. Didn't know it, I will definitely give it a look!

[Seldaek]

@maraspin sounds more or less ok from what I understood. I'd be happy to have a PSR Cache interface before doing this though.

[maraspin]

Thank you, @Seldaek