Muliple alerts of monitoring service for the same check result

[Hudratronium]

ISSUE TYPE

• Bug Report

COMPONENT NAME

- Virtual router
- monitoring VR service

CLOUDSTACK VERSION

4.15.2

CONFIGURATION

• Advanced networking
• custome offering for redundant VPC without services dns / dhcp / userdata

OS / ENVIRONMENT

• Ubuntu server 20.04

SUMMARY

multiple alerts are sended (up to 90 in my setup) each single time the monitoring service detects a missing service. All alerts have the exact same timestemp.

STEPS TO REPRODUCE

- create a VPC / shared network with out providing service userdata / dns / dhcp
- wait for the first iteration of monitoring service

EXPECTED RESULTS

- if the monitoring service detects a missing service / service failure one alert should be sendet to the management server

ACTUAL RESULTS

- sending huge numbers of alerts to the management-server, which are related all to the timestemp and check of the monitoring service.
- Flooding of Alert-Overview with the same alert

[Hudratronium]

@weizhouapache
as you suggested here a new issue

[DaanHoogland]

@Hudratronium can you give some clues as to which services are provided in the VPC and what the alerts are about.
I created a vpc without dns/dhcp/userdata and am trying to reproduce, but to no avail so far.

[Hudratronium]

Service Offerings of the VPC:

• Static NAT
• Portforwarding
• VPN
• NetworkACL
• SouceNAT

Checks that failed (that's why i disabled them manually):

• dhcp check
• dns check

If i remember correctly, this was the case for both routers - as i deployed a redundant VPC.

[DaanHoogland]

@Hudratronium I've now a redundant VPC running with the same services but no alerts. The only difference is that I am running the latest, i.e. 4.16.0. If you have a change to reproduce it in a test environment, please try 4.16. I'll try out a 4.15.2 as well to see if I get any health check failures.

[Hudratronium]

Hi @DaanHoogland
i enabled the healthchecks on my 4.16 environment. Up till now no "false" alerts or anything.
Result for dhcp check: "All 0 VMs are present ind dhcphosts.txt" <- Which seems logial to me as the dhcp service is not deployed. :-D Same result for the dns check.

So up till now this seems working for me under 4.16. Lovely!

[DaanHoogland]

ok, closing now @Hudratronium . please reopen when you have further issues