Hello, everyone. Today, I'd like to share a case about the Ethernet loop because of offshore E-LAN configuration. I hope it will be helpful for you.
Issue Description
1.1 Software Version:
NMS version: U2000 V1R5C00SPC603, WebLCT V1R6C00SPC200.
Site-A: RTN950 with CSH ver 5.76.03.42.
Site-B: RTN980 with CSHN ver 5.83.03.42.
1.2 Topology Service configuration:
For network topology and rack layout please check the below diagram, All RTN has been configured with E-LAN service to access & aggregate 3G traffic (Iub interface) coming from Node-B and transfer it to SDH network (OSN).
Also for RTN management, the InBand DCN has been configured then through the SDH network the whole traffic is aggregated to NE40 which filters the Iub traffic to RNC & M2000 and sends the RTN DCN to U2000.
1.3 EPLAN service configuration:
At the Site-1,2,3: all the services were configured only on the Vertical ISX2 and the Horizontal ISX2 was not a member of the EPLAN service, details service configuration for all sites are below:
Site1-NE1: EPLAN members [ EM6F-S1-P1 (VLAN:1000,1001) , ISX2-S3 (VLAN:1000,1001) ]
Site2-NE1: EPLAN members [ EM6F-S1-P3 (VLAN:1000) , ISX2-S3 (VLAN:1000,1001), ISX2-S7 (VLAN:1000,1001) ]
Site3-NE1: EPLAN members [ EM6F-S1-P1 (VLAN:1001) , EM6F-S1-P3 (VLAN:1000), ISX2-S3 (VLAN:1000,1001), ISX2-S7 (VLAN:1000,1001) ]
Site3-NE2: EPLAN members [ EM4F-P3 (VLAN:1001) , ISU2-S3 (VLAN:1001)].
Site4-NE1: EPLAN members [ EM6F-S1-P2 (VLAN:1000,1001) , ISU2-S3 (VLAN:1000,1001)].
Site4-NE2: EPLAN members [ EM6F-S1-P1 (VLAN:1000,1001), EM6F-S1-P2 (VLAN:1001), ISU2-S3 (VLAN:1000), ISU2-S4 (VLAN:1000) ].
Site4-NE3: EPLAN members [ EM4F-P3 (VLAN:1001), ISU2-S3 (VLAN:1001)].
Site5-NE1,Site6-NE1: EPLAN members [ EM4F-P1 (VLAN:1000) , ISU2-S3 (VLAN:1000)].
Site7-NE1,Site8-NE1: EPLAN members [ EM4F-P1 (VLAN:1001) , ISU2-S3 (VLAN:1001)].
All the services were configured without split-horizon group.
Figure 1: Network Topology
Figure 2: Rack Layout
.jpg)
Alarm Information
PORT_EXC_TRAFFIC
Handling Process
Check the RMON performance at OSN & RTN we found huge traffic & unexpected.
Obviously a broadcast storm happened which increased the RMON readings.
Check the EPLAN service configuration and modify it only to configure the VLAN on the needed ports and delete the VLANs from unrequired ports.
Check Split Horizon group, it was not configured, after adding the ISX2 to split Horizon Group and modify the VLAN’s on the EPLAN members the problem solved.
Below is the detailed configuration:
Site1-NE1:
EPLAN members [EM6F-S1-P1 (VLAN:1000,1001) , ISX2-S3 (VLAN:1000), ISX2-S4 (VLAN:1001) ].
Add Split Horizon Group [ISX2-S3, ISX2-S4]
Site2-NE1:
EPLAN members [ EM6F-S1-P3 (VLAN:1000), ISX2-S3 (VLAN:1000), ISX2-S7 (VLAN:1000), ISX2-S4 (VLAN:1001), ISX2-S8 (VLAN:1001) ].
Add Split Horizon Group [EM6F-S1-P3, ISX2-S7, ISX2-S8]
Site3-NE1:
EPLAN members [ EM6F-S1-P1 (VLAN:1001) , EM6F-S1-P3 (VLAN:1000), ISX2-S3 (VLAN:1000), ISX2-S7 (VLAN:1000,1001), ISX2-S4 (VLAN:1001) ].
Add Split Horizon Group [EM6F-S1-P1, EM6F-S1-P3, ISU2-S7].
Root Cause
Customer engineers want to distribute the traffic between Vertical & Horizontal ISX2 on the XPIC links.
So at Site2-NE1 & Site3-NE1, they add the ISX2 of the horizontal ISX2 to the EPLAN service & they add both VLANs (1000 & 1001).
Site1-NE1: EPLAN members [ EM6F-S1-P1 (VLAN:1000,1001) , ISX2-S3 (VLAN:1000,1001), ISX2-S4 (VLAN:1000,1001) ]
Site2-NE1: EPLAN members [ EM6F-S1-P3 (VLAN:1000) , ISX2-S3 (VLAN:1000,1001), ISX2-S7 (VLAN:1000,1001), ISX2-S4 (VLAN:1000,1001), ISX2-S8 (VLAN:1000,1001) ]
Site3-NE1: EPLAN members [ EM6F-S1-P1 (VLAN:1001) , EM6F-S1-P3 (VLAN:1000), ISX2-S3 (VLAN:1000,1001), ISX2-S7 (VLAN:1000,1001), ISX2-S4 (VLAN:1000,1001) ]
They change the configuration for Site3-NE1 then Site2-NE1, once they finish the configuration they noticed that huge traffic is passing the ISX2 cards & all ISX2 at Site2 & Site3 reports PORT_EXC_TRAFFIC warning, then a few seconds later all the 3G traffic is down.
We checked the RMON data for the Ethernet ports at Site2 & 3 we found around 250 Mbps traffic even the 3G NodeB’s still in the testing (Not commercially launch yet).
Suggestions
For EPLAN Service the split-horizon group must be configured carefully since it helps to stop Ethernet Loops which and avoid service interruption.
Configuring the VLAN should be accurate and based on design requirements, adding unwanted VLANs at different ports may lead to unwanted communication between NEs and this may lead to an Ethernet loop also it will decrease the bandwidth utilization and affect the actual customer capacity.
Thank you for browsing.
