Troubleshooting packet forwarding fails

The routing table contains routeinformation but packet forwarding fails.

Possible Causes

1) Switch configurations problems, for example, security policies are configured.

2) The network environment problems, for example, packet loss occurs, ARP learning is abnormal and ICMP destination is unreachable.

Troubleshooting Procedure
Step1: Check whether the environment is normal.

Check the network topology, including thedevice interfaces and IP addresses.

Step2: Check whether route entries are normal.

1.Check a route along the forwarding path hop by hop, and check whether the local end has a route reachable to the peer end and whether the peer end has a return route.

Ruijie#show ip route | include X.X.X.X    or  show ip route X.X.X.X (X.X.X.X indicates thedestination network segment)

If the route configurations are incorrect,modify the route configurations, and check whether the route forwarding is reachable.

2.Check whether a gateway address is configured on a PC NIC card. If not, configure the gateway address on the PC.

Step3: Check whether massive packets are lost.

Run show interfaces counters command to display the interface error counters like FCS, collisionsand alignments. You can check whether the value of the FCS Errors increases orderly. If yes, it indicates packet loss occurs due to a hardware fault. If conditions permit,replace the interface and network cable to check whether the fault isrectified. Along the forwarding path, check whether the next device is faultyby using the same method.

Step4: Check the running status of interfaces.

Run show interfaces [interface-id] command to check the physical status, speed,duplex mode, auto-negotiation status of the interfaces to make sure that their physical status is up, and the speed, duplex mode and auto-negotiation status are consistent at the two ends.

For copper port, it works well with the 10M/100Mnegotiation speed; but it works abnormally with the 1000M negotiation speed. Checkwhether the network cable is normal. If not, replace the network cable.

Step5: Check whether security policies are configured.

Check whether the ACL, binding and other security functions are configured. If yes, disable the corresponding configurations and then test whether the links can be successfully pinged.

Step6: Check whether the ping failure is caused by a largedelay in link transmission.

Using ping command to test the networkreachability

Ruijie# ping ip-address [ length bytes ] [ ntimes times ] [ timeout seconds ]

If the ping succeeds after the delay isincreased (specifying the timeout parameterfor the ping command enables the setting of ping delay), it indicates that the forwarding path is normal and that the fault may be caused by link congestion.

Step7: Reduce the fault locating scope.

Ping the next hop one by one to check the link between which two devices is disconnected. Perform ingress or egress packet capturing for a target device to check whether the packets that failed to be forwarded reach the target device. This step is critical because it can determine where the packets are lost and locate two faulty devices.

Step8: Check whether ARP entries are learned.

Ruijie#show arp | incX.X.X.X

Check ARP entries on the faulty devices. If ARP entries of the peer endare not learned, it indicates that the fault occurs in ARP packets.

For a PC, run the following commands in the “command prompt” window:

>ipconfig /all

>arp -a

Check whether the PC has learned the ARP entries of the gateway.

If no ARP entries are learned, go to the next step to check whether packets are normally received and sent.

Step9: Check whether ARP packets or ICMP packets areunreachable.

Perform packet capturing on the faulty devices and PC to check the receiving and sending of ARP and ICMP packets.

Enable the debugging function for ARP and ICMP on the faulty devices tocheck whether the ARP and ICMP packets are normally sent to the CPUs.

Ruijie#debug arp +ACL

Disable the preceding debugging function.

Ruijie#undeug all

Note: This command outputs huge information; therefore an ACL must be configured for filtering. In addition, it is recommended to use a standard ACL, and define the source and destination IP segments to debug ARP request and reply packets respectively.

Debugging operations cause risks (The worst case is to restart the switch for recovery). Perform debugging only after informing customers ofthe risks and get them accepted. It is recommended debugging at low-traffic periods (Be more cautious when dealing with core switches). If packet capturingis also required for troubleshooting, remember to collect information by debugging and packet capturing at the same time.

Step10. Fault Information Collection

Collect log information (pay attention to the time switch and the time accuracy) for fault analysis.

[Device debugging information, configuration, hardware and software versions, device logs and operation logs]

Collecting basic information

show version

show version slots

show run

show log

show ip interface brief

show interface status

show interface counter sum

show interface counter  rate

show interfaces counters errors

show interfaces counters

show arp counter

show arp

show arp detail

show mac-address-table

show mac-address-table counter

show ip route

show ip route count

show cpu