This post is regarding MZooKeeper Heap memory usage exceeding the threshold. Let's see below how to tackle this issue.
Description
The system checks the heap memory usage of the MZooKeeper service every 60 seconds. The alarm is generated when the heap memory usage of a MZooKeeper instance exceeds the threshold (95% of the maximum memory).
The alarm is cleared when the memory usage is less than the threshold.
Possible causes
The heap memory of the MZooKeeper instance is overused or the heap memory is inappropriately allocated.
Solution
1. Log in to FusionInsight Manager.
2. On the FusionInsight Manager portal, On the displayed interface, click the drop-down button of ALM-43904 MZooKeeper Heap Memory Usage Exceeds the Threshold and find the IP address of the instance for which the alarm is reported in Location Information.
3. On the FusionInsight Manager portal, choose Cluster > Name of the desired cluster > Service > MZooKeeper > Instance > Mquorumpeer(the IP address checked). Click the drop-down menu in the upper right corner of Chart, choose Customize > Memory, and select MZooKeeper Heap And Direct Buffer Resource Percentage. Check the heap memory usage.
4. Check whether the used heap memory of MZooKeeper reaches 95% of the maximum heap memory specified for MZooKeeper.
If yes, go to 5.
If no, go to 8.
5. On the FusionInsight Manager portal, choose Cluster > Name of the desired cluster > Service > MZooKeeper > Configuration > All Configurations > Mquorumpeer > System. Increase the value of -Xmx in GC_OPTS as required.
6. Save the configuration and restart the MZooKeeper service.
7. Check whether the alarm is cleared.
If yes, no further action is required.
If no, go to 8.
8. On the FusionInsight Manager portal, choose O&M > Log > Download.
9. Select MZooKeeper of the cluster to be operated from the Service.
10. Click http://10.88.194.32:7088/idp-edit-service/editor/image/25541599350/A-1_1_en-us_image_0168911587.png in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
11. Contact technical support and send the collected logs.
