Troubleshooting ERPS

Network failureoccurs in an ERPS ring network (single ring or intersecting ring).

  Possible Causes

1) Configurations are incorrect, forexample, the device is not configured with the RPL owner node or device isconfigured with RPL owner nodes.

2) Operations are incorrect, for example, incorrectoperations cause loops.

3) The ERPS ring cannot be stabilized tothe idle state.

Troubleshooting Procedure

Step 1: Check the ERPS configurations ofeach device.

Step 2: Check operations on the ERPS ringnetwork that may cause loops.

Step 3: Check whether the ERPS ring networkcan recover to the idle state.

Step1: Run the show run command to display the ERPSconfigurations of each device.

ERPS configuration tips:

1. Each ring has only one RPL owner node.

2. The RPL-port command is notrequired for non RPL-Owner nodes.

                              

ERPS configurations of [S86E-2]

erpsenable

erpsraps-vlan 4001

ring-port west GigabitEthernet 0/3 eastGigabitEthernet 0/4

rpl-port west rpl-owner

state enable

interfaceGigabitEthernet 0/3

switchport mode trunk

rldp port bidirection-detect shutdown-port

!

interfaceGigabitEthernet 0/4

switchport mode trunk

rldp port bidirection-detect shutdown-port

ERPS configurations of [S86E-3]

erpsenable

erpsraps-vlan 4001

ring-port west GigabitEthernet 0/3 eastGigabitEthernet 0/4  

rpl-port east rpl-owner

state enable

interfaceGigabitEthernet 0/3

switchport mode trunk

rldp port bidirection-detect shutdown-port

!

interfaceGigabitEthernet 0/4

switchport mode trunk

rldp port bidirection-detect shutdown-port

Run the show run command. If itis found that both devices are configured with the RPL owner node, the RPLowner node in the ERPS ring network will be Blocked, which will causedisconnection of the ring network. See the following topology.

2: Enable debug erps packet of the devicesto check receiving and sending of ERPS packets.

If you cannot loginto all devices to collect ERPS configurations due to network disconnection,you can run the debug erps packetcommand to check receiving and sending of ERPS packets on device S86E-2 (or S86E-4)on which RPL owner is deployed. You can check whether the MAC addresses in theR-APS (NR, RB) packets received in the same RAPS-VLAN are the same as the MACaddress of the device. If not, it indicates that one ERPS ring network isconfigured with two RPL owner nodes.

Debugging operations cause risks (The worst case is to restartthe switch for recovery). Perform debugging only after informing customers ofthe risks and get them accepted. It is recommended debugging at low-trafficperiods (Be more cautious when dealing with core switches). If packet capturingis also required for troubleshooting, remember to collect information bydebugging and packet capturing at the same time.

S86E-2# debugerps packet

22:23:34:35:  %7: [ERPS-PKT]:Send erps packet R-APS(NR, RB)at 198567506, raps-vlan 4001 status 0x80

22:23:34:35:  %ERPS-5-TOPOLOGY_CHANGE: Topology changed forR-APS VLAN 4001, The ring changed state from protection to idle

22:23:34:35:  %7: [ERPS-PKT]:Send erps packet R-APS(NR, RB)at 198567509, raps-vlan 4001 status 0x80     

22:23:34:35:  %7: [ERPS-PKT]:Send erps packet R-APS(NR, RB)at 198567510, raps-vlan 4001 status 0x80   

22:23:34:39:  %7: [ERPS-PKT]:Recv packet at 198567995, len= 60

22:23:34:39:  %7: [ERPS-PKT]:Recv erps packet R-APS(NR) at198567995, raps-vlan 4001 ifx 4 status 0x0 node-id 001a.a917.de42

22:23:34:39:  %7: [ERPS-PKT]:Recv packet at 198567995, len= 60

22:23:34:39:  %7: [ERPS-PKT]:Recv erps packet R-APS(NR) at198567995, raps-vlan 4001 ifx 3 status 0x0 node-id 001a.a917.de42   //Receive (NR,RB) packets from another RPLowner node. The MAC address implies that the packets are sent by device S86E-3,indicating that there is another ERPS RPL owner node in the network, namely,the network is configured with two RPL owner nodes.

S86E-2# nodebug all

All possibledebugging has been turned off

TheERPS status of each device is as follows:

Device S86E-2

S86E-2#sho erps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

RingStatus                 : Enabled

WestPort                   : Gi0/3        (Blocking)  

EastPort                   : Gi0/4        (Forwarding)

RPLPort                    : West Port

ProtectedVLANs             : ALL

RPLOwner                   : Enabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : idle

Device S86E-3

S86E-3#shoerps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

RingStatus                 : Enabled

WestPort                   : Gi0/3       (Forwarding)

EastPort                   : Gi0/4       (Blocking)  

RPLPort                    : East Port

ProtectedVLANs             : ALL

RPLOwner                   : Enabled

HoldoffTime                : 0 milliseconds

Guard Time                  : 500 milliseconds

WTRTime                    : 5 minutes

CurrentRing State          : idle

Device S86E-4

S86E-4#shoerps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

RingStatus                 : Enabled

WestPort                   : Gi0/3        (Forwarding)

EastPort                   : Gi0/4        (Forwarding)

RPLPort                    : None

ProtectedVLANs             : ALL

RPLOwner                   : Disabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : idle

Thepreceding information shows that the Gi0/3 interface of device S86E-2 and theGi0/4 interface of device S86E-3 are blocked, which causes that device S86E-2cannot access device S86E-3.

3.Delete the RPL owner configuration from S86E-3,wait for 2 minutes (the WTR time period of S8600E devices is 2 minutes duringwhich the ERPS convergence completes.), and then run the show erps command to display the ERPS status. After the ring statusbecomes stable (idle state), only the Gi0/3 interface of device S86E-2 is inthe Block state.

Theprocedure for deleting RPL owner configurations is as follows:

       1. Shut down an ERPS interface.

       2. Disable ERPS of ring 4001.

       3. Delete the RPL owner configuration.

       4. Enable ERPS of ring 4001.

       5. Enable the ERPS interface that isshut down previously.

Example:Modify the ERPS configurations of device S86E-3. The configurations are asfollows:

SS86E-3(config)#intgi 0/4

SS86E-3(config-if)#shutdonwn

SS86E-3(config)#erps raps-vlan 4001

SS86E-3(config-erps4001)#no state enable   

SS86E-3(config-erps4001)#no rpl-port

SS86E-3(config)#intgi 0/4

SS86E-3(config-if)#noshutdonwn

The ERPS status of device S86E-2 is as follows:

SS86E_2#show erps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

RingStatus                 : Enabled

West Port                   : Gi0/3        (Block)

East Port                   : Gi0/4        (Forwarding)

RPL Port                    : None

ProtectedVLANs             : ALL

RPLOwner                   : Disabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : idle

The ERPS status of device S86E-4 is as follows:

SS86E_4#show erps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

Ring Status                 : Enabled

West Port                   : Gi0/3        (Forwarding)

East Port                   : Gi0/4        (Forwarding)

RPL Port                    : West

ProtectedVLANs             : ALL

RPLOwner                   : Disabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : idle

The ERPS status of device S86E-4 is as follows:

SS86E_4#show erps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

RingStatus                 : Enabled

West Port                   : Gi0/3        (Forwarding)

East Port                   : Gi0/4        (Forwarding)

RPL Port                    : None

ProtectedVLANs             : ALL

RPLOwner                   : Disabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : idle

Step2: Check whether the ERPS interface is modified duringthe operations. If the ERPS interface is directly modified without being shutdown, convergence will fail and then a loop will occur.

The topology is as follows:

The problem operation procedures are as follows:

Scenario1: The topology is shown in the preceding figure.

1. S86E-2is an RPL owner node in the ERPS ring. The ERPS interfaces on S86E-2 should beGi0/3 (up) and Gi0/4 (up). However, the ERPS interfaces are configured as Gi0/3(up) and Gi0/6 (down) incorrectly.

S86E-2(config)#erpsraps-vlan 4010

S86E-2(config-erps4010)#no state enable

S86E-2(config-erps4010)#ring-port west gi0/3 east gi0/6

S86E-2(config-erps4010)#ring-port west rpl-owner

S86E-2(config-erps4010)#state enable

2. Since theGi0/6 interface of S86E-2 is in the down state, the Gi0/3 interface sends an SFpacket. Then, S86E-2 enters the protection state and needs toreceive an NR packet to trigger the WRT timer and recover to the idle state.

3. On device S86E-2, modify the ERPSinterface from Gi0/6 to Gi0/4. However, this configuration does not shut downthe ERPS interface Gi0/3 (or Gi0/4).

S86E-2(config)#erpsraps-vlan 4010

S86E-2(config-erps4010)#no state enable   //Stop sending SFpackets.

S86E-2(config-erps4010)#ring-port west gi0/3 east gi0/4   //Configure the ERPS interface.

S86E-2(config-erps4010)#ring-port west rpl-owner  //EnableERPS without triggering NR packets, which causes that the WTR timer cannot beenabled, the ERPS control packets in the entire network are lost, ERPSconvergence fails and then a loop occurs. ERPS is in the protection state andthe ERPS interface is in the (Link Normal) forwarding state.

S86E-2(config-erps4010)#state enable

4. The ERPS ring state modified through thepreceding operations is as follows:

S86E-2#shoerps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4010

RingStatus                 : Enabled

WestPort                   : Gi0/3        (Forwarding)

EastPort                   : Gi0/4        (Forwarding)

RPLPort                    : West Port

ProtectedVLANs             : ALL

RPLOwner                   : Enabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : protection

S86E-2#debug erp pa    //debug erps packet is enabled, but no packet is received, the WTR timerof ERPS is lost, and the ERPS control plane of loses effect.

[Scenario 2] The topology is shown in thepreceding figure. The ERPS interfaces on S86E-4 should be Gi0/3 (up) and Gi0/4(up). However, the ERPS interfaces are configured as Gi0/3(up) and Gi0/6 (down)incorrectly.

S86E-4(config)#erpsraps-vlan 4010

S86E-4(config-erps4010)#no state enable

S86E-4(config-erps4010)#ring-port west gi0/3 east gi0/6

S86E-4(config-erps4010)#state enable

S86E-4(config-erps4010)#00:00:20:23: %ERPS-5-PORT_STATE_CHANGE: Port GigabitEthernet 0/6 on R-APS VLAN 4010has been set to forwarding state.

00:00:20:23:  %ERPS-5-PORT_STATE_CHANGE: PortGigabitEthernet 0/3 on R-APS VLAN 4010 has been set to forwarding state.

S86E-4#showerps

ERPS Information

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4010

RingStatus                 : Enabled

WestPort                   : Gi0/3        (Link Normal)

EastPort                   : Gi0/6        (Link Failure)  //A down interface is associated. In the ERPSring network, only the Gi0/6 interface is in the down state and the Gi0/3interface sends SF packets.

RPLPort                    : None

ProtectedVLANs             : ALL

RPLOwner                   : Disabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : protection

2) The interface Gi0/6 is replaced by theinterface Gi0/4 when the ERPS interface (Gi0/3 or Gi0/4) is not shut down. Thiscauses that the WTR timer is lost, ERPS control packets are lost, ERPSconvergence fails, and a loop occurs. The ERPS is in the protection state andthe ERPS interface is in the (Link Normal) forwarding state.

S86E-2#config

S86E-2(config)#erpsraps-vlan 4010

S86E-2(config-erps4010)#no state enable

S86E-2(config-erps4010)#ring-port west gi0/3 east gi0/4  //Make the modification directly without shutting down the ERPSinterface gi0/3 or gi0/4.

S86E-2(config-erps4010)#state enable

S86E-2#shoerps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4010

RingStatus                 : Enabled

WestPort                   : Gi0/3        (Link Normal)

EastPort                   : Gi0/4        (Link Normal)

RPLPort                    : None

ProtectedVLANs             : ALL

RPLOwner                   : Disabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : protection

If you are sure that the loop occurring onthe ERPS ring network is caused by preceding operations, rectify the fault byusing the following configurations:

int gi 0/3    //On the faulty device, shut down the ERPS interface.

shutdown

erps raps-vlan 4010

no state enable

ring-port west gi0/7 east gi0/8  //Change the ERPS interface.

ring-port west rpl-owner

state enable

int gi 0/3  //After completing the preceding configurations, enable the interface.

no shutdown

Step 3: Check ERPS status that cannot beconverged to the idle state in the network.

1. Run show commands to check the devicestatus in the ERPS ring network and check whether the ERPS status cannot beconverged to idle.

S86E-2#shoerps

ERPS Information

GlobalStatus               : Enabled

Link monitoredby           : Not Oam

-------------------------------------------

R-APS VLAN                  : 4010

Ring Status                 : Enabled

West Port                   : Gi0/3        (Blocking)  

East Port                   : Gi0/4        (Forwarding)

RPL Port                    : West Port

ProtectedVLANs             : ALL

RPL Owner                   : Enabled

Holdoff Time                : 0 milliseconds

Guard Time                  : 500 milliseconds

WTR Time                    : 2 minutes

CurrentRing State          : protection   //The device keeps the protection state allthe time.

2. Check whether all devices in the ERPS ring network areenabled with ERPS.

Asshown in the following topology, S86E-3 is not enabled with ERPS.

1.After ERPS is correctly configured for S86E-2 and S86E-4, check the ERPS statusthrough debug erps packets on S86E-2. Since S86E-3 is not enabled with ERPS,RLDP of S86E-2 and S86E-4 detects a loop, which will disable the Gi0/3interface of each device.

Device S86E-2

S86E-2#sho rldp

rldpstate         : enable

rldphello interval: 3

rldpmax hello     : 6

rldplocal bridge  : 00d0.f822.33b9

-----------------------------------

GigabitEthernet0/3

portstate      : error

neighborbridge : 001a.a917.de42

neighborport   : GigabitEthernet 0/3

bidirectiondetect information :

     action: shutdown-port

state : error

GigabitEthernet0/4

portstate      : normal

neighborbridge : 00d0.f822.33e6

neighborport   : GigabitEthernet 0/4

bidirectiondetect information :

     action: shutdown-port

     state : normal

S86E-2#debug erps packet

21:22:42:32:  %7: [ERPS-PKT]:Send erps packet R-APS(SF) at189615201, raps-vlan 4001 status 0x0

21:22:42:36:  %7: [ERPS-PKT]:Recv packet at 189615682, len= 60

21:22:42:36:  %7: [ERPS-PKT]:Recv erps packet R-APS(SF) at189615682, raps-vlan 4001 ifx 4 status 0x0 node-id 00d0.f822.33e6

21:22:42:37:  %7: [ERPS-PKT]:Send erps packet R-APS(SF) at189615701, raps-vlan 4001 status 0x0

21:22:42:41:  %7: [ERPS-PKT]:Recv packet at 189616182, len= 60

//Sincethe Gi0/3 interface is in the error state detected by RLDP, the device sends SFpackets to notify the peer device. (When the node link is down, the node sendsthis packet to notify other nodes.)

S86E-2#no debug all

S86E-2#sho erps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

RingStatus                 : Enabled

WestPort                   : Gi0/3        (Link Failure)            //Since this interface is detectedas error by RLDP, link failure is displayed.

EastPort                   : Gi0/4        (Forwarding)

RPLPort                    : West Port

ProtectedVLANs             : ALL

RPLOwner                   : Enabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : protection

Device S86E-4

S86E-4#sho rldp

rldpstate         : enable

rldphello interval: 3

rldpmax hello     : 2

rldplocal bridge  : 00d0.f822.33e6

-----------------------------------

GigabitEthernet0/4

portstate      : normal

neighborbridge : 00d0.f822.33b9

neighborport   : GigabitEthernet 0/4

bidirectiondetect information :

     action: shutdown-port

     state : normal

loopdetect information        :

     action: shutdown-port

     state : normal

GigabitEthernet0/3

portstate      : error

neighborbridge : 001a.a917.de42

neighborport   : GigabitEthernet 0/4

bidirectiondetect information :

     action: shutdown-port

     state : error

loopdetect information        :

     action: shutdown-port

     state : normal

S86E-4#debug erps packet

07:01:23:57:  %7: [ERPS-PKT]:Send erps packet R-APS(SF) at60983722, raps-vlan 4001 status 0x0

07:01:24:01:  %7: [ERPS-PKT]:Recv packet at 60984141, len =60

07:01:24:01:  %7: [ERPS-PKT]:Recv erps packet R-APS(SF) at60984141, raps-vlan 4001 ifx 4 status 0x0 node-id 00d0.f822.33b9

07:01:24:02:  %7: [ERPS-PKT]:Send erps packet R-APS(SF) at60984222, raps-vlan 4001 status 0x0

S86E-4#sho erps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

RingStatus                 : Enabled

WestPort                   : Gi0/3        (Link Failure)

EastPort                   : Gi0/4        (Forwarding)

RPLPort                    : None

ProtectedVLANs             : ALL

RPLOwner                   : Disabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTRTime                    : 2 minutes

CurrentRing State          : protection

Note: In the ERPS ring network, S86E-3 not enabled with ERPS cantransparently transmit ERPS packets. However, RLDP takes effect first in thisprocess; therefore, the Gi0/3 interfaces of S86E-2 and S86E-4 are always in theRLDP error state, and the corresponding interfaces in the ERPS ring network arealways in the link failure state. When the entire ERPSring network is faulty and then the faulty is rectified, the ERPS cannotrecover to the idle state but the interconnection of the entire network isnormal without any loop.

Solution:Configure the erps command fordevices not enabled with the ERPS function.

Step3: Check whether any link connection in the network isdown. When a physical link in the ERPS ring network is down, the link failurestate is kept all the time.

When a faulty link is recovered, the ERPSstate can be recovered to idle. If the faulty link is always in the down state,ERPS state cannot be recovered to idle, which is a normal behavior of ERPS.

Thetopology is as follows:

S86E-3#shoerps

ERPS Information

GlobalStatus               : Enabled

Link monitoredby           : Not Oam

-------------------------------------------

R-APS VLAN                  : 4001

Ring Status                 : Enabled

West Port                   : Gi0/3       (Forwarding)

East Port                   : Gi0/4       (Link Failure)

RPL Port                    : None

ProtectedVLANs             : ALL

RPL Owner                   : Disabled

Holdoff Time                : 0 milliseconds

Guard Time                  : 500 milliseconds

WTR Time                    : 5 minutes

Current RingState          : protection

S86E-2#22:23:22:29:  %ERPS-5-PORT_STATE_CHANGE: PortGigabitEthernet 0/3 on R-APS VLAN 4001 has been set to forwarding state.

22:23:22:29:  %ERPS-5-TOPOLOGY_CHANGE: Topology changed forR-APS VLAN 4001, The ring changed state from idle to protection

S86E-2#debugerps pack

S86E-2#22:23:23:39:  %7: [ERPS-PKT]:Recv packet at 198501930, len= 60

22:23:23:39:  %7: [ERPS-PKT]:Recv erps packet R-APS(SF) at198501930, raps-vlan 4001 ifx 3 status 0x0 node-id 001a.a917.de42   

22:23:23:39:  %7: [ERPS-PKT]:Recv packet at 198501943, len= 60

22:23:23:39:  %7: [ERPS-PKT]:Recv erps packet R-APS(SF) at198501943, raps-vlan 4001 ifx 4 status 0x0 node-id 00d0.f822.33e6

//SF packets arecontinuously received from S86E-3 and S86E-4. The ERPS is in the protectionstate.

S86E-2#nodebug all

All possibledebugging has been turned off

S86E-2#shoerps

ERPSInformation

GlobalStatus               : Enabled

Linkmonitored by           : Not Oam

-------------------------------------------

R-APSVLAN                  : 4001

RingStatus                 : Enabled

WestPort                   : Gi0/3        (Forwarding)

EastPort                   : Gi0/4        (Forwarding)

RPLPort                    : West Port

ProtectedVLANs             : ALL

RPLOwner                   : Enabled

HoldoffTime                : 0 milliseconds

GuardTime                  : 500 milliseconds

WTR Time                    : 2 minutes

CurrentRing State          : protection

Solution:Recover the faulty link. If it is confirmed that an interface is shut down(which can be recovered by the noshutdown command), check whether the interface is shut down due to cableproblems (which can be solved by cable replacement).

Step4: Fault Information Collection

If the fault persists after the precedingoperations are performed, collect the following fault information and dial thenumber 4008-111-000 for help.

showversion

showrun

showlog

showcpu

showinterface description

showinterface status

showinterface counter

showmac-address-table

showerps

showerps global

debugerps packet

Collect the logs during troubleshooting.