icedraco / heimon Goto Github PK

There should be more details regarding the specific connection that triggered a BUG alert (or any alert whatsoever, in fact):

• Buffer the regular "log" messages for that one connection
• Save the result from that one connection
• Dump all into the alert log for future analysis
• Take care to avoid dumping irrelevant-to-that-cycle log info

Right now there's some sort of hackery going on...
URL: https://docs.python.org/3/howto/logging.html

Example (from daily maintenance that lags the entire system temporarily):
[Sun Apr 10 05:00:08 2016][ALERT!] Heimdall check cycle failed: Timed out during AUTH stage
[Sun Apr 10 05:00:16 2016][ALERT!] Heimdall check cycle failed: Timed out during AUTH stage
[Sun Apr 10 05:00:25 2016][ALERT!] Heimdall check cycle failed: Timed out during AUTH stage
[Sun Apr 10 05:00:33 2016][ALERT!] Heimdall check cycle failed: Timed out during AUTH stage
[Sun Apr 10 05:00:41 2016][ALERT!] Heimdall check cycle failed: Timed out during AUTH stage
[Sun Apr 10 05:00:50 2016][ALERT!] Heimdall check cycle failed: Timed out during AUTH stage
[Sun Apr 10 05:00:58 2016][ALERT!] Heimdall check cycle failed: Timed out during AUTH stage

While the case of missing heimdalls does not repeat itself this way (due to "freshly_missing_threshold" value), all the other alerts from an on-going problem - will. Depending on the reporting mechanism, this can become very annoying and not useful!

There should be an option to avoid frequently-repeating alerts of the same kind on one hand, but on the other hand: long-lasting problems should be brought up again after a while (a la freshly_missing_threshold).

Obtaining data from the server...

DragonroarState
AuthState
ClosedState
Testing result...
[Sat Apr 9 00:08:18 2016][ALERT!] <class 'heimon.tests.TestAllComponentsPresent'>/BUG: 'heimdall'
component is missing from result!
Sleeping (2 secs)