Hi, Community!
I have a case about SNMP access to storage array share with you.
Description
When network management software (NMS) such as Zabbix, snmpwalk, snmpget, eService, and MIB Browser accesses a storage array through the SNMP protocol, the storage array does not respond to the access request.
Symptom
Zabbix and eService use SNMP to monitor storage devices. The storage information cannot be queried by running the snmpwalk command. The message "Timeout: No Response from XXX" is displayed.
Causes
1. Immediate cause: Thread deadlock occurs when thread concurrency in the third-party SNMP library causes a lock error.
2. Root cause: In the third-party SNMP library, multiple threads are used to process SNMP requests. When a lock is added, the lock is released repeatedly. As a result, the MIB tree is unlocked during update and deadlock is caused by concurrent illegal memory accesses.
Fault Locating
1. Capture packets on the storage array to check whether the storage array responds.
In the minisystem, run the tcpdump.sh -i any port 161 command. If the customer has changed the SNMP port, specify port 161 as the port.
The preceding figure shows that port 161 of the storage array does not respond.
2. Export the log packet and check whether SNMP logs are printed. If the SNMP is not suspended, consecutive logs are generated. The log directories are as follows:
As shown in the preceding log screenshot, if the last entry is generated a long time later than the current time, the process is interrupted.
Solution
If the storage system is an OceanStor converged storage system, upgrade it to V500R007C30SPH109 or V300R006C50SPH109.
If the storage system is an OceanStor Dorado storage system, upgrade it to V300R002C10SPC100SPH110.
Workarounds
Run the change snmp status status=stop command.
Run the change snmp status status=start command to restart the SNMP service. The fault is temporarily rectified, but may recur when the customer queries frequently.
Check After Recovery
Run the mib browser, zabbix, or snmpwalk command to access the storage array. The fault is rectified if access succeeds.
We warmly welcome you to enjoy our community!
