ARP Resolution process in Juniper EX switches
When an ip packet which requires a L3 lookup hits an interface it will be subjected to L3 lookup provided Destination Mac address is the Mac address of the switch.
Let us take an example of a ip packet destined to 10.1.1.10 which is not resolved yet and falls under the subnet 10.1.1.0/24, this is a directly connected subnet on the EX.
The packet matches the following route entry:
20.1.1/24 Resolve 1337 RT-ifl 66 ge-0/0/16.0 ifl 66
This entry will be installed when the L3 interface is created on the switch.
Due to the above lookup, the packet will be trapped to the CPU (SFI) for resolving the destination.Such packets are rate-limited (1000 pps).
SFI queues the packets and informs the pfem to resolve it.
PFE queues this request and informs the kernel to resolve the ip destination.The route corresponding to this ip will be kept on HOLD in the PFE.
10.1.1.10 Hold 1346 RT-ifl 66 ifl 131079
A timer is also associated with this route
kernel then resolves this ip, by broadcasting ARP request in the broadcast domain.
If a reply is not received the hold timer is expired and the HOLD route is deleted.PFE informs SFI about the failure and packet is dropped by SFI.
If the reply is received by the Kernel, Kernel informs pfe to install the unicase route ,PFE does that and informs SFI who inturn will forward the packet.
10.1.1.10 10.1.1.10 Unicast 1346 RT-ifl 66 ge-0/0/16.0 ifl 66
Hold NH is added when arp request is sent.
If the arp reply is not received, this times out and the NH is deleted:
root@EX-lab:RE:0% rtsockmon -t sender flag type op [18:16:48] kernel P route add inet 10.1.1.10 tid=0 plen=32 type=dest flags=0x0 nh=hold nhflags=0x1 nhidx=1346 rt_nhiflist = 0 altfwdnhidx=0 filtidx=0 [18:16:48] kernel P nexthop add inet FCH-A672-LANIERLD435.vtc.csc.com nh=hold flags=0x1 idx=1346 ifidx=66 filteridx=0 [18:16:51] kernel P route delete inet 10.1.1.10 tid=0 plen=32 type=dest flags=0x180 nh=hold nhflags=0x1 nhidx=1346 rt_nhiflist = 0 altfwdnhidx=0 filtidx=0 [18:16:51] kernel P nexthop delete inet FCH-A672-LANIERLD435.vtc.csc.com nh=hold flags=0x5 idx=1346 ifidx=66 filteridx=0
Packets are Soft-dropped by pfe, till the Hold NH is present
PFEM0(/dev/ttyp0)# show halp-rt route ip rtt-index 0 prefix 10.1.1.10 p 32 Route Type Paths RtIdx Rpf SipSa Row:Col:Row:Col -------------------- ---- ----- ----- --- ----- --------------- 10.1.1.10 ECMP 0 25 No No 29:0:0:0 Dev0 (RtIdx: 25) ----------------- Command : Soft Drop CpuCode : 0 <<<<<<<<<< Soft Drop AppSpCpuCodeEn : 0 UcSipFiltEna : 0 TtlDecEna : 0 TtlOptChkBypass: 0 IngressMirror : 0 QoSProfileEn : 0 QoSProfileIndex : 0 QoSPrecedence : 0 ModifyUp : 0 ModifyDscp : 0 CounterSet : 1 ArpBc2Cpu : 0 SipAccessLevel : 0 DipAccessLevel : 0 IcmpRedirExpnMirr : 0 MtuProfileIdx : 1 Ipv6ScopeCheckEn : 0 Ipv6DstSiteId : 0 NhTnnl : 0 NhTnnlIdx : 0 NhVlan : 0 NhIf(dev/port) : 0/0 NhArpIdx : 0 Device: 0 ArpEntry Idx 0 : ff:ff:ff:ff:ff:ff Hit/Miss : N Dev1 (RtIdx: 25) ----------------- Command : Soft Drop CpuCode : 0 <<<<<<<<<< Soft Drop AppSpCpuCodeEn : 0 UcSipFiltEna : 0 TtlDecEna : 0 TtlOptChkBypass: 0 IngressMirror : 0 QoSProfileEn : 0 QoSProfileIndex : 0 QoSPrecedence : 0 ModifyUp : 0 ModifyDscp : 0 CounterSet : 1 ArpBc2Cpu : 0 SipAccessLevel : 0 DipAccessLevel : 0 IcmpRedirExpnMirr : 0 MtuProfileIdx : 1 Ipv6ScopeCheckEn : 0 Ipv6DstSiteId : 0 NhTnnl : 0 NhTnnlIdx : 0 NhVlan : 0 NhIf(dev/port) : 0/0 NhArpIdx : 0 Device: 1 ArpEntry Idx 0 : ff:ff:ff:ff:ff:ff Hit/Miss : N
The below commands can be useful in debugging arp issues:
root@EX-lab:RE:0% netstat -p arp arp: 49697 datagrams received 4729 ARP requests received 1898 ARP replies received <<<<< No of ARP relpies 13989 resolution requests received 0 unrestricted proxy requests 0 restricted proxy requests 0 received proxy requests 0 unrestricted proxy requests not proxied 0 restricted-proxy requests not proxied 0 with bogus interface 0 with incorrect length 0 for non-IP protocol 0 with unsupported op code 0 with bad protocol address length 0 with bad hardware address length 0 with multicast source address 0 with multicast target address 0 with my own hardware address 950 for an address not on the interface 0 with a broadcast source address 0 with source address duplicate to mine 42120 which were not for me 0 packets discarded waiting for resolution 1 packets sent after waiting for resolution 70928 ARP requests sent <<<<< No of ARP requests 4729 ARP replies sent 0 requests for memory denied 0 requests dropped on entry 0 requests dropped during retry 0 requests dropped due to interface deletion 0 requests on unnumbered interfaces 0 new requests on unnumbered interfaces 0 replies for from unnumbered interfaces 0 requests on unnumbered interface with non-subnetted donor 0 replies from unnumbered interface with non-subnetted donor
PFEM0(/dev/ttyp0)# show halp-nh statistics
Shim Layer Next Hop Statistics:
Nh Type: Hold
Inst Req: 1
UnInst Req: 1
root@EX-lab:0% rtsockmon -t
PFEM0(/dev/ttyp0)# show nhdb management all
PFEM0(/dev/ttyp0)# show halp-rt route ip rtt-index 0 prefix 10.1.1.10 p 32