S5300 V200R001C00SPC300

Problem:

On our newly deployed S5300 we observed huge amount of OSPF and VRRP flapping. We got the media checked. There was not problem in media. At first it seemed very strange that protocols were randomly flapping on their own.

Description and Solution:

When we did troubleshooting we find out that Like Junipers Protect RE, Huawei also has routing engine protection mechanism to protect it routing engine called MPU. All the traffic that is being carried to MPU has rate limit. You can see it by issuing, display cpu-defend applied all  

We found out that we where using too aggressive VRRP and OSPF hello and dead timer and number of protocol adjacency were too many for switch. When due to this reason protocol control packets are dropped you can see following message in logbuffer.

%%01DEFD/4/CPCAR_DROP_MPU(l)[1]:Rate of packets to cpu exceeded the CPCAR limit on the MPU. (Protocol=ospf, ExceededPacketCount=0747)

Then you can see the dropped traffic by issue following command. 

display cpu-defend statistics

You can see the current rate limit of protocol control packets by issuing following command. 

display cpu-defend applied all 

You can change the applied rate limits by using custom cpu-defend policy and changing rate limit for particular protocol. But check with GTAC as well because allowing too many packets to MPU can cause high cpu utilization and system crash.

Solution is to relax protocol timers, increase cpu-defend rate limit of protocols or use bfd. But remember S5300 does not support echo based BFD, bfd packets will also go via MPU.