This article describes the issue of users experiencing traffic loss, when ECMP is used on redundant links during the NS/NA process.
Topology:
-------------------------------------- 2001:10:2::/64, fe80::10:2:0:0/64 ge-2/1/0 - ::2 ge-/2/0/0 - ::1 +------------------+---------------+----------+ | Logical LR1 | | Master | +------------------+---------------+----------+ ge-6/1/0 - ::2 ge-6/0/0 - ::1 2001:10:1::/64, fe80::10:1:0:0/64 lo0.2 - 2001:192:168:2::1/128 lo0.1 - 2001:192:168:1::1/128
- The master and LR1 are on the same router, on which LR1 is a logical-system.
- All interfaces are running OSPF3 and the master is conducting ECMP.
- The ge-6/0/0 interface is disabled.
The master is continuously sending ICMP packet to LR1’s lo0.2 2001:192:168:2::1; every second. ge-6/0/0 is enabled via the following procedure:
----------------------------------- lab@barutan-re0# run clear ipv6 neighbors fe80::10:2:0:2 00:05:85:68:1a:3b deleted fe80::10:2:0:1 00:05:85:68:19:fc deleted 2001:10:2::1 00:05:85:68:19:fc deleted 2001:10:2::2 00:05:85:68:1a:3b deleted fe80::10:1:0:2 00:05:85:68:1e:33 deleted fe80::10:1:0:1 00:05:85:68:1d:f4 deleted 2001:10:1::1 00:05:85:68:1d:f4 deleted lab@barutan-re0# set interfaces ge-6/0/0 enable lab@barutan-re0# commit commit complete -----------------------------------
The output of the ping indicates that the master did not hear an ICMP response from LR1 (from 15:41:33 to 15:41:43:
ICMP:
2012/08/20 15:41:30 ping result OK 2012/08/20 15:41:31 ping result OK 2012/08/20 15:41:32 ping result OK ; ge-6/0/0 enabled 2012/08/20 15:41:33 ping result NG 2012/08/20 15:41:34 ping result NG 2012/08/20 15:41:35 ping result NG 2012/08/20 15:41:37 ping result NG 2012/08/20 15:41:38 ping result NG 2012/08/20 15:41:39 ping result NG 2012/08/20 15:41:40 ping result NG 2012/08/20 15:41:41 ping result NG 2012/08/20 15:41:42 ping result NG 2012/08/20 15:41:43 ping result NG 2012/08/20 15:41:44 ping result OK 2012/08/20 15:41:45 ping result OK 2012/08/20 15:41:46 ping result OK 2012/08/20 15:41:47 ping result OK 2012/08/20 15:41:49 ping result OK 2012/08/20 15:41:48 ping result OK 2012/08/20 15:41:50 ping result OK
The packet capture on ge-2/0/0 indicates that no packets are traversing on this interface from 15:41:33 to 15:41:36:
ge-2/0/0:
15:41:30.668401 Out IP6 2001:10:2::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:30.669005 In IP6 2001:192:168:2::1 > 2001:10:2::1: ICMP6, echo reply, seq 0, length 16 15:41:31.735979 Out IP6 2001:10:2::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:31.736596 In IP6 2001:192:168:2::1 > 2001:10:2::1: ICMP6, echo reply, seq 0, length 16 15:41:32.799911 Out IP6 2001:10:2::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:32.800488 In IP6 2001:192:168:2::1 > 2001:10:2::1: ICMP6, echo reply, seq 0, length 16 15:41:37.064614 Out IP6 2001:192:168:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:40.269934 Out IP6 2001:192:168:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:43.458779 Out IP6 2001:192:168:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:44.521339 Out IP6 2001:10:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:46.894579 Out IP6 2001:10:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:47.957522 Out IP6 2001:10:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:50.085271 Out IP6 2001:10:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16
The newly enabled ge-6/0/0 interface does not traverse the ICMP packet, until it completes DAD and NA / NS, which takes place from 15:41:33 to 15:41:44:
ge-6/0/0:
15:41:33.253844 Out IP6 2001:192:168:1::1 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has fe80::10:1:0:2, length 32 15:41:33.259055 In IP6 2001:192:168:2::1 > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::10:1:0:1, length 32 15:41:34.770282 Out IP6 2001:192:168:1::1 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has fe80::10:1:0:2, length 32 15:41:34.770747 In IP6 2001:192:168:2::1 > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::10:1:0:1, length 32 15:41:34.818265 Out IP6 :: > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has 2001:10:1::1, length 24 15:41:34.818289 Out IP6 :: > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::10:1:0:1, length 24 15:41:35.770037 Out IP6 2001:192:168:1::1 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has fe80::10:1:0:2, length 32 15:41:35.770554 In IP6 2001:192:168:2::1 > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::10:1:0:1, length 32 15:41:37.817534 Out IP6 :: > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has 2001:10:1::1, length 24 15:41:37.817553 Out IP6 :: > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::10:1:0:1, length 24 15:41:40.816802 Out IP6 :: > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has 2001:10:1::1, length 24 15:41:40.816827 Out IP6 :: > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::10:1:0:1, length 24 15:41:41.768559 Out IP6 2001:192:168:1::1 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has fe80::10:1:0:2, length 32 15:41:41.768925 In IP6 2001:192:168:2::1 > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::10:1:0:1, length 32 15:41:43.816066 Out IP6 2001:10:1::1 > ff02::1: ICMP6, neighbor advertisment, tgt is 2001:10:1::1, length 32 15:41:43.816083 Out IP6 fe80::10:1:0:1 > ff02::1: ICMP6, neighbor advertisment, tgt is fe80::10:1:0:1, length 32 15:41:43.816786 In IP6 2001:10:1::2 > ff02::1: ICMP6, neighbor advertisment, tgt is 2001:10:1::2, length 32 15:41:43.816789 In IP6 fe80::10:1:0:2 > ff02::1: ICMP6, neighbor advertisment, tgt is fe80::10:1:0:2, length 32 15:41:44.768144 In IP6 fe80::10:1:0:2 > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has 2001:10:1::1, length 32 15:41:44.768167 Out IP6 fe80::10:1:0:1 > fe80::10:1:0:2: ICMP6, neighbor advertisment, tgt is 2001:10:1::1, length 32 15:41:44.768713 In IP6 2001:192:168:2::1 > 2001:10:1::1: ICMP6, echo reply, seq 0, length 16 15:41:45.831235 Out IP6 2001:10:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:45.831933 In IP6 2001:192:168:2::1 > 2001:10:1::1: ICMP6, echo reply, seq 0, length 16 15:41:46.895316 In IP6 2001:192:168:2::1 > 2001:10:1::1: ICMP6, echo reply, seq 0, length 16 15:41:47.958289 In IP6 2001:192:168:2::1 > 2001:10:1::1: ICMP6, echo reply, seq 0, length 16 15:41:49.021643 Out IP6 2001:10:1::1 > 2001:192:168:2::1: ICMP6, echo request, seq 0, length 16 15:41:49.022281 In IP6 2001:192:168:2::1 > 2001:10:1::1: ICMP6, echo reply, seq 0, length 16 15:41:50.085907 In IP6 2001:192:168:2::1 > 2001:10:1::1: ICMP6, echo reply, seq 0, length 16
The master router is actually sending ICMP packets from 15:41:33 to 15:41:36. However, it is not being displayed on tcpdump, as it is being dropped; before it reaches the interface that is still in the process of DAD.
ECMP sends out traffic from the interface, without considering whether its state is in hold or reject. The workaround is to configure a static NDP entry for the relevant link-local address disable DAD.