CCNP Switch: Multilayer Switching

CCNP Switch: Multilayer Switching

InterVLAN Routing

Recall that a Layer 2 network is defined as a broadcast domain. A Layer 2 network also can exist as a VLAN inside one or more switches. VLANs essentially are isolated from each other so that packets in one VLAN cannot cross into another VLAN.

To transport packets between VLANs, you must use a Layer 3 device. Traditionally, this has been a router’s function. The router must have a physical or logical connection to each VLAN so that it can forward packets between them. This is known as interVLAN routing.

InterVLAN routing can be performed by an external router that connects to each of the VLANs on a switch. Separate physical connections can be used, or the router can access each of the VLANs through a single trunk link. Part A of Figure 13-1 illustrates this concept. The external router also can connect to the switch through a single trunk link, carrying all the necessary VLANs, as illustrated in Part B of Figure 13-1. Part B illustrates what commonly is referred to as a “router on a stick” or a “one-armed router” because the router needs only a single interface to do its job.

Finally, Part C of Figure 13-1 shows how the routing and switching functions can be combined into one device: a multilayer switch. No external router is needed.

Figure 13-1 Examples of InterVLAN Routing Connections
CCNP Switch Multilayer Switchingdfig13.1

Types of Interfaces

Multilayer switches can perform both Layer 2 switching and interVLAN routing, as appropriate. Layer 2 switching occurs between interfaces that are assigned to Layer 2 VLANs or Layer 2 trunks. Layer 3 switching can occur between any type of interface, as long as the interface can have a Layer 3 address assigned to it.

As with a router, a multilayer switch can assign a Layer 3 address to a physical interface. It also can assign a Layer 3 address to a logical interface that represents an entire VLAN. This is known as a switched virtual interface (SVI).

Configuring InterVLAN Routing

InterVLAN routing first requires that routing be enabled for the Layer 3 protocol. In addition, you must configure static routes or a dynamic routing protocol. These topics are covered fully in the BSCI course.

Because a multilayer switch supports many different types of interfaces for Layer 2 or Layer 3 switching, you must define each interface on a switch that will be used. By default, every switch port on platforms such as the Catalyst 2950, 3560, or 4500 is a Layer 2 interface, whereas every switch port on a Catalyst 6500 (native IOS) is a Layer 3 interface. If another type or mode is needed, you must explicitly configure it.

A port is either in Layer 2 or Layer 3 mode, depending on the use of the switchport configuration command. You can display a port’s current mode with the following command:

If the Switchport: line is shown as enabled, the port is in Layer 2 mode. If this line is shown as disabled, as in the following example, the port is in Layer 3 mode:

Figure 13-2 shows how the different types of interface modes can be used within a single switch.

Layer 2 Port Configuration

By default, all switch ports on Catalyst 2950, 3560, and 4500 platforms operate in Layer 2 mode. If you need to reconfigure a port for Layer 2 functionality, use the following command sequence:

Figure 13-2 Catalyst Switch with Various Types of Ports
CCNP Switch Multilayer Switchingdfig13.2

The switchport command puts the port in Layer 2 mode. Then you can use other switchport command keywords to configure trunking, access VLANs, and so on. Figure 13-2 shows several Layer 2 ports, each assigned to a specific VLAN. A Layer 2 port also can act as a trunk, transporting multiple VLANs.

Layer 3 Port Configuration

Physical switch ports also can operate as Layer 3 interfaces, where a Layer 3 network address is assigned and routing can occur. Figure 13-2 shows an example of this. By default, all switch ports on the Catalyst 6500 (native IOS) platform operate in the Layer 3 mode. For Layer 3 functionality, you must explicitly configure switch ports with the following command sequence:

The no switchport command takes the port out of Layer 2 operation. You then can assign a network address to the port, as you would to a router interface.

NOTEKeep in mind that a Layer 3 port assigns a network address to one specific physical interface. If using EtherChannel, it, too, can become a Layer 3 port. In that case, the network address is assigned to the port-channel interface, not to the individual physical links within the channel.

SVI Port Configuration

On a multilayer switch, you also can enable Layer 3 functionality for an entire VLAN on the switch. This allows a network address to be assigned to a logical interface: that of the VLAN itself.
This is useful when the switch has many ports assigned to a common VLAN, and routing is needed in and out of that VLAN.

Figure 13-2 shows how an IP address is applied to the switched virtual interface (SVI) called VLAN 10. Notice that the SVI itself has no physical connection to the outside world; to reach the outside, VLAN 10 must extend through a Layer 2 port or trunk.

The logical Layer 3 interface is known as an SVI. However, when it is configured, it uses the much more intuitive interface name vlan vlan-id, as if the VLAN itself is a physical interface. First, define or identify the VLAN interface; then assign any Layer 3 functionality to it with the following configuration commands:

The VLAN must be defined and active on the switch before the SVI can be used. Make sure the new VLAN interface also is enabled with the no shutdown interface-configuration command.

NOTEThe VLAN and the SVI are configured separately, even though they interoperate. Creating or configuring the SVI doesn’t create or configure the VLAN; you still must define each one independently. As an example, the following commands show how VLAN 100 is created and then defined as a Layer 3 SVI:

Multilayer Switching with CEF

Catalyst switches can use several methods to forward packets based on Layer 3 and Layer 4 information. The current generation of Catalyst multilayer switches uses the efficient Cisco Express Forwarding (CEF) method. This section describes the evolution of multilayer switching and discusses CEF in detail. Although CEF is easy to configure and use, the underlying switching mechanisms are more involved and should be understood.

Traditional MLS Overview

Multilayer switching began as a dual effort between a route processor (RP) and a switching engine (SE). The basic idea is to “route once and switch many.” The RP receives the first packet of a new traffic flow between two hosts, as usual. A routing decision is made, and the packet is forwarded toward the destination.

To participate in multilayer switching, the SE must know the identity of each RP. The SE then can listen in to the first packet going to the router and also going away from the router. If the SE can switch the packet in both directions, it can learn a “shortcut path” so that subsequent packets of the same flow can be switched directly to the destination port without passing through the RP.

This technique also is known as NetFlow switching or route cache switching. Traditionally, NetFlow switching was performed on Cisco hardware, such as the Catalyst 6000 Supervisor 1/1a and Multilayer Switch Feature Card (MSFC), Catalyst 5500 with a Route Switch Module (RSM), Route Switch Feature Card (RSFC), or external router. Basically, the hardware consisted of an independent RP component and a NetFlow-capable SE component.

CEF Overview

NetFlow switching has given way to a more efficient form of multilayer switching: Cisco Express Forwarding (CEF). Cisco developed CEF for its line of routers, offering high-performance packet forwarding through the use of dynamic lookup tables.
CEF also has been carried over to the Catalyst switching platforms. The following platforms all perform CEF in hardware:

  • Catalyst 6500 Supervisor 720 (with an integrated MSFC3)
  • Catalyst 6500 Supervisor 2/MSFC2 combination
  • Catalyst 4500 Supervisor III, IV, and V
  • Fixed-configuration switches, such as the Catalyst 3750, 3560, 3550, and 2950

CEF runs by default, taking advantage of the specialized hardware.

A CEF-based multilayer switch consists of two basic functional blocks, as shown in Figure 13-3:
The Layer 3 Engine is involved in building routing information that the Layer 3 Forwarding Engine can use to switch packets in hardware.

Figure 13-3 Packet Flow Through a CEF-Based Multilayer Switch
CCNP Switch Multilayer Switchingdfig13.3

Forwarding Information Base

The Layer 3 engine (essentially a router) maintains routing information, whether from static routes or dynamic routing protocols. Basically, the routing table is reformatted into an ordered list with the most specific route first, for each IP destination subnet in the table. The new format is called a Forwarding Information Base (FIB) and contains routing or forwarding information that the network prefix can reference.

In other words, a route to might be contained in the FIB along with routes to 24 and, if those exist. Notice that these examples are increasingly more specific subnets, as designated by the longer subnet masks. In the FIB, these would be ordered with the most specific, or longest match, first, followed by less specific subnets. When the switch receives a packet, it easily can examine the destination address and find the longest-match destination route entry in the FIB.
The FIB also contains the next-hop address for each entry. When a longest-match entry is found in the FIB, the Layer 3 next-hop address is found, too.

You might be surprised to know that the FIB also contains host route (subnet mask entries. These normally are not found in the routing table unless they are advertised or manually configured. Host routes are maintained in the FIB for the most efficient routing lookup to directly connected or adjacent hosts.

As with a routing table, the FIB is dynamic in nature. When the Layer 3 engine sees a change in the routing topology, it sends an update to the FIB. Any time the routing table receives a change to a route prefix or the next-hop address, the FIB receives the same change. Also, if a next-hop address is changed or aged out of the Address Resolution Protocol (ARP) table, the FIB must reflect the same change.

You can display FIB table entries related to a specific interface or VLAN with the following form of the show ip cef command:

The FIB entries corresponding to the VLAN 101 switched virtual interface might be shown as demonstrated in Example 13-1.

Example 13-1 Displaying FIB Table Entries for a Specified VLAN

You also can view FIB entries by specifying an IP prefix address and mask, using the following form of the show ip cef command:

The output in Example 13-2 displays any subnet within that is known by the switch, regardless of the prefix or mask length. Normally, only an exact match of the IP prefix and mask will be displayed if it exists in the CEF table. To see other longer match entries, you can add the longer-prefixes keyword.

Example 13-2 Displaying FIB Table Entries for a Specified IP Prefix Address/Mask

Notice that the first three entries are the same ones listed in Example 13-1. Other subnets also are displayed, along with their next-hop router addresses and switch interfaces. You can add the detail keyword to see more information about each FIB table entry for CEF, as demonstrated in Example 13-3.

Example 13-3 Displaying Detailed CEF Entry Information

The version number describes the number of times the CEF entry has been updated since the table was generated. The epoch number denotes the number of times the CEF table has been flushed and regenerated as a whole. The subnet has two next-hop router addresses, so the local switch is using per-destination load sharing between the two routers.

After the FIB is built, packets can be forwarded along the bottom dashed path in Figure 13-3. This follows the hardware switching process, in which no “expensive” or time-consuming operations are needed. At times, however, a packet cannot be switched in hardware, according to the FIB. Packets then are marked as “CEF punt” and immediately are sent to the Layer 3 engine for further processing, as shown in the top dashed path in Figure 13-3. Here are some of the conditions that cause this:

  • An entry cannot be located in the FIB.
  • The FIB table is full.
  • The IP Time To Live (TTL) has expired.
  • The maximum transmission unit (MTU) is exceeded, and the packet must be fragmented.
  • An Internet Control Message Protocol (ICMP) redirect is involved.
  • The encapsulation type is not supported.
  • Packets are tunneled, requiring a compression or encryption operation.
  • An access list with the log option is triggered.
  • A Network Address Translation (NAT) operation must be performed (except on the Catalyst 6500 Supervisor 720, which can handle NAT in hardware).

CEF operations can be handled on a single hardware platform, such as the Catalyst 3560 switch. The FIB is generated and contained centrally in the switch. CEF also can be optimized through the use of specialized forwarding hardware, using the following techniques:

  • Accelerated CEF (aCEF)—CEF is distributed across multiple Layer 3 forwarding engines, typically located on Catalyst 6500 line cards. These engines do not have the capability to store and use the entire FIB, so only a portion of the FIB is downloaded to them at any time. This functions as an FIB “cache,” containing entries that are likely to be used again. If FIB entries are not found in the cache, requests are sent to the Layer 3 engine for more FIB information.
    The net result is that CEF is accelerated on the line cards, but not necessarily at a sustained wire-speed rate.
  • Distributed CEF (dCEF)—CEF can be distributed completely among multiple Layer 3 forwarding engines for even greater performance. Because the FIB is self-contained for complete Layer 3 forwarding, it can be replicated across any number of independent Layer 3 forwarding engines. The Catalyst 6500 has line cards that support dCEF, each with its own FIB table and forwarding engine. A central Layer 3 engine (the MSFC3, for example) maintains the routing table and generates the FIB, which then dynamically is downloaded in full to each of the line cards.

Adjacency Table

A router normally maintains a routing table containing Layer 3 network and next-hop information, and an ARP table containing Layer 3 to Layer 2 address mapping. These tables are kept independently.

Recall that the FIB keeps the Layer 3 next-hop address for each entry. To streamline packet forwarding even more, the FIB has corresponding Layer 2 information for every next-hop entry. This portion of the FIB is called the adjacency table, consisting of the MAC addresses of nodes that can be reached in a single Layer 2 hop.

You can display the adjacency table’s contents with the following command:

As an example, the total number of adjacencies known on each physical or VLAN interface can be displayed with the show adjacency summary command, as demonstrated in Example 13-4.

Example 13-4 Displaying the Total Number of Known Adjacencies

Adjacencies are kept for each next-hop router and each host that is connected directly to the local switch. You can see more detailed information about the adjacencies by using the detail keyword, as demonstrated in Example 13-5.

Example 13-5 Displaying Detailed Information About Adjacencies

Notice that the adjacency entries include both the IP address (Layer 3) and the MAC address (Layer 2) of the directly attached host. The MAC address could be shown as the first six octets of the long string of hex digits (as shaded in the previous output) or on a line by itself. The remainder of the string of hex digits contains the MAC address of the Layer 3 engine’s interface (six octets, corresponding to the Vlan99 interface in the example) and the EtherType value (two octets, where 0800 denotes IP).

The adjacency table information is built from the ARP table. Example 13-5 shows adjacency with the age of its ARP entry. As a next-hop address receives a valid ARP entry, the adjacency table is updated. If an ARP entry does not exist, the FIB entry is marked as “CEF glean.” This means that the Layer 3 forwarding engine can’t forward the packet in hardware because of the missing Layer 2 next-hop address. The packet is sent to the Layer 3 engine so that it can generate an ARP request and receive an ARP reply. This is known as the “CEF glean” state, in which the Layer 3 engine must glean the next-hop destination’s MAC address.

The glean state can be demonstrated in several ways, as demonstrated in Example 13-6.

Example 13-6 Displaying Adjacencies in the CEF Glean State

Notice that the FIB entry for directly connected host is present but listed in the glean state. The show ip arp command shows that there is no valid ARP entry for the IP address. During the time that an FIB entry is in the CEF glean state waiting for the ARP resolution, subsequent packets to that host immediately are dropped so that the input queues do not fill and the Layer 3 engine does not become too busy worrying about the need for duplicate ARP requests. This is called ARP throttling or throttling adjacency. If an ARP reply is not received in 2 seconds, the throttling is released so that another ARP request can be triggered. Otherwise, after an ARP reply is received, the throttling is released, the FIB entry can be completed, and packets can be forwarded completely in hardware. The adjacency table also can contain other types of entries so that packets can be handled efficiently. For example, you might see the following adjacency types listed:

  • Null adjacency—Used to switch packets destined for the null interface. The null interface always is defined on a router or switch; it represents a logical interface that silently absorbs packets without actually forwarding them.
  • Drop adjacency—Used to switch packets that can’t be forwarded normally. In effect, these packets are dropped without being forwarded. Packets can be dropped because of an encapsulation failure, an unresolved address, an unsupported protocol, no valid route present, no valid adjacency, or a checksum error. You can gauge drop adjacency activity with the following command:
  • Discard adjacency—Used when packets must be discarded because of an access list or other policy action.
  • Punt adjacency—Used when packets must be sent to the Layer 3 engine for further processing. You can gauge the CEF punt activity by looking at the various punt adjacency reasons listed by the show cef not-cef-switched command:
    The reasons shown are as follows:

    • An incomplete adjacency (No_adj)
    • An incomplete ARP resolution (No_encap)
    • Unsupported packet features (Unsupp’ted)
    • ICMP redirect (Redirect)
    • Layer 3 engine interfaces (Receive)
    • IP options present (Options)
    • Access list evaluation failure (Access)
    • Fragmentation failure (Frag)

The Receive reason includes packets destined for IP addresses that are assigned to interfaces on the Layer 3 engine, IP network addresses, and IP broadcast addresses.

Packet Rewrite

When a multilayer switch finds valid entries in the FIB and adjacency tables, a packet is almost ready to be forwarded. One step remains: The packet header information must be rewritten. Keep in mind that multilayer switching occurs as quick table lookups, to find the next-hop address and the outbound switch port. The packet is untouched and still has the original destination MAC address of the switch itself. The IP header also must be adjusted, as if a traditional router had done the forwarding.

The switch has an additional functional block that performs a packet rewrite in real time. The packet rewrite engine (shown in Figure 13-3) makes the following changes to the packet just before forwarding:

  • Layer 2 destination address—Changed to the next-hop device’s MAC address
  • Layer 2 source address—Changed to the outbound Layer 3 switch interface’s MAC address
  • Layer 3 IP Time To Live (TTL)—Decremented by one because one router hop has just occurred
  • Layer 3 IP checksum—Recalculated to include changes to the IP header
  • Layer 2 frame checksum—Recalculated to include changes to the Layer 2 and Layer 3 headers

A traditional router normally would make the same changes to each packet. The multilayer switch must act as if a traditional router were being used, making identical changes. However, the multilayer switch can do this very efficiently with dedicated packet-rewrite hardware and address information obtained from table lookups.

Configuring CEF

CEF is enabled on all CEF-capable Catalyst switches by default. In fact, the Catalyst 6500 (with a Supervisor 720 and its integrated MSFC3, or a Supervisor 2 and MSFC2 combination) runs CEF inherently, so CEF never can be disabled.

NOTESwitches such as the Catalyst 3750 and 4500 run CEF by default, but you can disable CEF on a per-interface basis. You can use the no ip route-cache cef and no ip cef interface configuration commands to disable CEF on the Catalyst 3750 and 4500, respectively. You always should keep CEF enabled whenever possible, except when you need to disable it for debugging purposes.

Fallback Bridging

For protocols that CEF can’t route or switch, a technique known as fallback bridging is used. Example protocols are IPX and AppleTalk, which are routable but not supported by CEF, as well as SNA and LAT, which are not routable. To summarize fallback bridging operation, each SVI associated with a VLAN in which nonroutable protocols are being used is assigned to a bridge

group. Packets that cannot be routed from one VLAN to another are bridged transparently instead, as long as the two VLANs belong to the same bridge group.

NOTE Only the Catalyst 3560 and 3750 offer fallback bridging; it can CEF switch IP packets but no others. The Catalyst 4500 and 6500 (all Supervisor models running Cisco IOS Software) also can CEF-switch IP but can handle other routable protocols more slowly with their Layer 3 engines. Those two platforms have no need for fallback bridging.

Bridge groups used in fallback bridging do not interact with normal Layer 2 switching (also using bridging). They do use a special Spanning Tree Protocol to maintain loop-free fallback bridging, but these bridge protocol data units (BPDU) are not exchanged with other 802.1D, Rapid Spanning Tree Protocol (RSTP), or Multiple Spanning Tree (MST) BPDUs on VLANs. Instead, the VLAN-bridge STP is used, with one instance per fallback bridge group.

To configure fallback bridging, first decide which VLANs have traffic that CEF cannot route. Begin by enabling a fallback bridge group and its instance of the VLAN-bridge STP:

Next, for each VLAN SVI in which nonroutable traffic will be bridged, assign it to the appropriate bridge group:

You can configure up to 31 different fallback bridge groups on a switch. Although the VLAN bridge STP instance running on each bridge group does not interact with normal 802.1D STP, it does behave similarly. For example, you can configure the bridge priority, port priority and cost, Hello timer, Forward Delay timer, and Max Age timer. These parameters all should look familiar because they are used in the 802.1D STP. Instead of using the spanning-tree command to adjust the parameter values, you must adjust them according to the bridge group number with the bridgegroup bridge-group command keywords.

Verifying Multilayer Switching

The multilayer switching topics presented in this chapter are not difficult to configure; however, you might need to verify how a switch is forwarding packets. In particular, the following sections discuss the commands that you can use to verify the operation of InterVLAN routing, CEF, and fallback bridging.

Verifying InterVLAN Routing

To verify the configuration of a Layer 2 port, you can use the following EXEC command:

The output from this command displays the access VLAN or the trunking mode and native VLAN. The administrative modes reflect what has been configured for the port, while the operational modes show the port’s active status. You can use this same command to verify the configuration of a Layer 3 or routed port. In this case, you should see the switchport (Layer 2) mode disabled, as in Example 13-7.

Example 13-7 Verifying Configuration of a Layer 3 Switch Port

To see the physical interface’s status and counters, use the command without the switchport keyword. To see a summary listing of all interfaces, you can use the show interface status command.
To verify the configuration of an SVI, you can use the following EXEC command:

The VLAN interface should be up, with the line protocol also up. If this is not true, either the interface is disabled with the shutdown command or the VLAN itself has not been defined on the switch. Use the show vlan command to see a list of configured VLANs.
Example 13-8 shows the output produced from the show vlan command. Notice that each defined VLAN is shown, along with the switch ports that are assigned to it.

Example 13-8 Displaying a List of Configured VLANs

You also can display the IP-related information about a switch interface with the show ip interface command, as demonstrated in Example 13-9.

You can use the show ip interface brief command to see a summary listing of the interfaces involved in routing IP traffic, as demonstrated in Example 13-10.

Example 13-10 Displaying a Summary Listing of Interfaces Routing IP Traffic

Verifying CEF

CEF operation depends on the correct routing information being generated and downloaded to the Layer 3 forwarding engine hardware. This information is contained in the FIB and is maintained dynamically. To view the entire FIB, use the following EXEC command:

Example 13-11 shows sample output from this command.

Example 13-11 Displaying the FIB Contents for a Switch

On this switch, only VLAN 1 has been configured with the IP address Notice several things about the FIB for such a small configuration:

  •—An FIB entry has been reserved for the default route. No next hop is defined, so the entry is marked “receive” so that packets will be sent to the Layer 3 engine for further processing.
  •—The subnet assigned to the VLAN 1 interface is given its own entry. This is marked “attached” because it is connected directly to an SVI, VLAN 1.
  •—An FIB entry has been reserved for the exact network address. This is used to contain an adjacency for packets sent to the network address, if the network is not directly connected. In this case, there is no adjacency, and the entry is marked “receive.”
  •—An entry has been reserved for the VLAN 1 SVI’s IP address. Notice that this is a host route (/32). Packets destined for the VLAN 1 interface must be dealt with internally, so the entry is marked “receive.”
  •—This is an entry for a neighboring multilayer switch, found on the VLAN 1 interface. The next-hop field has been filled in with the same IP address, denoting that an adjacency is available.
  •—An FIB entry has been reserved for the subnet’s broadcast address. The route processor (Layer 3 engine) handles all directed broadcasts, so the entry is marked “receive.” To see complete FIB table information for a specific interface, use the following EXEC command:

Verifying Fallback Bridging

To verify the operation of fallback bridging, you can use the following EXEC commands:

The first command shows a summary of all active fallback bridge groups, along with their STP states. The second command displays the bridging table contents for a specific fallback bridge group.

About the author


Leave a Comment