Issue Description:
When customer tried the face search feature on the IVS3800 no results appear, instead he receives the below.
Beside that the below alarm were received.
Handling process:
1- After RDE checked the alarms, he found that an exception occurs when the search table is written, the heartbeat of the TServer process is lost, and the MCS process is offline.
2- We checked the major alarms on the CSP as below.
We found the below alarms.
Major 5004 Insufficient Disk Space use_rate over 85% csp 5/4/2021 13:58
Major 308703407 Retrieving data table write exception tableName=dynamic_ ivs3800-01 5/24/2021 3:49
Major 308521513 harddisk full disk sub area = /Snapshot ivs3800-01 5/25/2021 8:51
Major 6807 Threshold-Controlled conditions Met A_VA_D_ivs entity name=Physical Memory Usage: >=90 csp 6/2/2021 21:11
3- We checked furtherly regarding alarm 308703407.
4- After analyzing the major alarms, RDE confirmed that the above alarm indicates that the retrieval table fails to be written. The cause is that the MCS-0003 container is overloaded and the feature data is not saved to the database. After restarting the process, it is now restored. Currently, the MCS version is OptiX OSN 3800 C50 SPC306, which has a known memory leak problem.
Problem can be solved by either of the below points.
1. Periodically check the memory usage of the MCS container. When the memory usage reaches 80%, restart the TSERVER process of the MCS.
2. Install patch 319.
5- Regarding the shown alarm ALM-308703514 D Chip AI Core Software Error
It is confirmed that in case that MCS and VA has shared resources, If memory leakage occurs in the MCS, the VA resources may not be used. As a result, this alarm is generated
