[Fault type]: The user has deleted the subscription service configuration, deleted all alarm subscriptions, and did not stop receiving and sending emails.
[Key words]: Subscription service configuration and alarm subscription
[Applicable version]: FusionCompute R3 or later
[Symptom]: After the subscription service configuration is deleted and all alarm subscriptions are deleted, a large number of emails are received for about three hours.
[Solution]:
1、 Check the email before and after the subscription service configuration and alarm subscription. It is found that the email receiving frequency is 10 minutes. After the email is received, a large number of emails are received. It is found that the contents of the mail are not real-time alarms but two months ago. After that, the mail alarm content is approaching the actual time.
2、 Based on the preceding symptoms,
One packet is received every 10 minutes. This is the same as the flow control configured by the customer.
The received email content is not real-time alarms. It is estimated that alarm emails are queued and sent in the first in first out (FIFO) mode.
After the deletion, emails may be accumulated in the sending queue. After the configuration is deleted, traffic control is not performed, and alarms in the queue are sent to the customer in a moment.
3、 Check the environment based on the code and customer alarm list,
The code implementation principle is as follows:
Every 100 ms, insert the alarm generated in real time into the end of the alarm email table, send alarm emails to the SMTP mail server according to the flow control, and delete the alarms that are successfully sent from this table. Then the email server sends an alarm email to the customer's email box.
Viewing Historical FC Alarms of a Customer
About 14230, and a large number of alarms are generated in a short period of time (most of which are VM memory /CPU usage exceeding the threshold).
In conclusion, the conclusion is as follows: The alarm generation speed is higher than the alarm email sending speed. As a result, a large number of alarms are accumulated in the sending list. As the number of alarms increases, the email content delay becomes longer and older. Once the configuration is deleted, flow control is deleted. The alarms to be sent are sent to the SMTP server in milliseconds. After the SMTP server sends the message to the customer, the delay is three hours.
4、 Solution
To monitor major alarms in real time, you are advised to set the email sending interval to a small value or do not configure it. You can select different users for different alarms and select the alarm severity. You are advised to add critical or major alarms.
