Email alerts are overly active about meteor-apm-server

[vblagomir]

Unfortunately I do not use Slack. The alerts are being sent every 10 seconds repeatedly. In the log it kept on showing "sendTriggered" and seem never "sendCleared".

[lmachens]

I never setup mailing (Slack alerts should work fine).
Do you get multiple alerts at the same time or are there some minutes between?
You can check console.log which should show you if a send is triggered or cleared:

meteor-apm-server/server/alertsman/server/index.js

Lines 62 to 78 in e02f924

	if (!armed && checkedResult.success) {
	// We don't need to wait until the trigger sends
	// to mark the alert as armed.
	messenger.sendTriggered(alert, checkedResult);
	await alertsStore.setArmed(alert, true);
	console.log('sendTriggered');
	return;
	}
	if (armed && !checkedResult.success) {
	// We don't need to wait until the trigger sends
	// to mark the alert as cleared.
	messenger.sendCleared(alert, checkedResult);
	await alertsStore.setArmed(alert, false);
	console.log('sendCleared');
	return;
	}

[afrokick]

Me too, but i use slack. Every 10 seconds. I'll try to find steps to reproduce.

[markreid]

I've also had this one. Alert triggered and then flooded a slack channel until I removed it. Didn't really have time to investigate it though, just had to leave it off.

[afrokick]

@vblagomir @markreid do you use oplog? I dont. I think AlertsManager cannot receive updates from DB without Oplog.

[vblagomir]

I have initially wrote 'Yes', but then thought that depending on how the whole setup has been done, oplog might have been NOT enabled. This might be the issue indeed.

[afrokick]

Yes.

Could you please try to run monitoring with DEBUG env var?
DEBUG=alertsman:altersStore meteor

You should see the message like this
Mon, 15 Jul 2019 14:08:20 GMT alertsman:altersStore reset and load 1 alerts in console after run.

[vblagomir]

I have edited my 'Yes'. And because of workload the earliest time I could check this would be in couple of months.

[afrokick]

I have edited my 'Yes'. And because of workload the earliest time I could check this would be in couple of months.

Ok, it is the reason of issue. Without Oplog it doesn't work.

@lmachens did you start to work on item 'Direct db access of alertsman' from ToDo ? Maybe I can help with it

[afrokick]

@lmachens Done afrokick#1

[vblagomir]

Ok, it is the reason of issue. Without Oplog it doesn't work.

I hope this is the reason, if after checking it appears not the case, I will reopen it then. Thank you for your effort investigating!