| Product Name | Version |
| RTN 980 | V100R006C10SPC100 |
| S5720/S5700 | V200R007C00SPC500 |
The client reported that, during the service testing process of S5700/S5720 switch carried by RTN equipments,
when the IF connection of RTN equipment change from 256QAM to DOWN,
the service load could be discarded according to the priority queue,
connection status recover from DOWN to QPSK~64QAM, service failed constantly.
Handling Processstep 1.Test on site CTB, IF connection recover from DOWN to 16QAM, service failed constantly, issue reproduced.
step 2.Analysis the performance statistics of these devices, concluded the traffic by direction CTB to VPK is normal, but the switch in site VPK did not send message to RTN device.
step 3.Diagnosed the reason of switch in site VPK did not send message to RTN, found that the switch did not learn the MAC address on the remote site. The ARP records ages every 20 minute and the switch will reinitiate the learning process. Switch in VPK send an ARP request, but it never received response message from the remote site, so the switch in VPK can not renew the ARP table.
step 4.After analyzing NMS configuration of the RTN device, found the trusted type of the ports on RTN device is set to “IP-DSCP”, therefore determine the Qos priority of messages by the DSCP information. BUT the ARP messages is Layer 2 messages, it lack of “IP-DSCP” segment, so the RTN device made it the lowest priority.
step 5.By the meantime, on transmitting direction of the IF link of RTN980 of site CTB is constantly jammed, ARP messages treated as lowest priority was discarded first, therefore the remote side VPK can’t learn the MAC of the switch in CTB.
step 6.By put a static ARP record on the switch of VPK, the service was recovered. By change the Qos configuration of the switch, put CVLAN priority information in both service messages and protocol messages, and change the configuration on RTN devices, make it determine message priority by CVLAN priority information, the problem did not showed again in many test, confirmed the solution is effective.
step 7.Tested again on site NPO which was performed normally, the problem could not be reproduced. After diagnosis, data shows every time the IF link is down, the connection between RTN and switch flash down. The ARP records was emptied by then. After link recovered, no message can be transmitted normally because of empty ARP record, so the IF link was not jammed, ARP protocol can perform normally, and the aging time of ARP records is renewed to 20 minute, so the problem of site CTB/MTP cannot be reproduced here.
step 8.Diagnosed the reason of flash down of the link between RTN and switch, found that the RTN was configured with time source, both of the port of EG2D board (connected to the switch) and prot of IF link is in the time source list. When the IF link is down, triggered a time source switching, by then, the port of EG2D board restarted automatically, resulted in the RTN-Switch link flash down.
step 9.Deleted the time source record of the RTN device in site NPO, the link flash down problem did not show up again. Performed tests following the steps took on site CTB/MTP, that problem could be reproduced.
----End<?xml:namespace prefix = "o" />
Root Cause1.Cause of service fail when IF link changed from DOWN to 64QAM:
The RTN device was configured to determine message priority by IP-DSCP value,
but the ARP protocol messages don’t have a DSCP value, and was processed as
lowest priority by the RTN. When the IF link works in QPSK-64QAM status, the BE
queen was constantly jammed, caused the ARP protocol messages was discarded by
priority, then the switch can’t get the MAC address of the other side, led to
the service failing.
2.Reason why the site NPO performed normal:
RTN device in NPO was configured to have ports on EG2D board and ports of IF
link in the time source list. When the IF link was down, it triggered a switch
of time source, and ports of EG2D would restart automatically, the switch
emptied ARP table after detected the link was down. By this time, the IF link
was not jammed, therefore the ARP protocol can works normally, and service
traffic would not be affected.
SolutionConfigure the switches with static ARP record.
SuggestionsChange the Qos policy of the switch, make the service messages and protocol messages carrying CVLAN priority information.
By the meantime, change the DS trust type of RTN devices, determine message priority by CVLAN priority information of the messages.
