Design 2: Maximizing Layer 3 with Catalyst 8500 Switching Routers
This section presents Design 2, an approach that relies on Catalyst 8500-style hardware-based routing (in other words, the 8500 is a switching router). Figure 17-6 illustrates Design 2.
Figure 17-6. Design 2 Network Diagram
Several differences from the physical layout used in Design 1 are important. First, the ATM core has been replaced with Gigabit Ethernet. Second, the Building 2 third floor has been replaced with a Catalyst 5509. However, both designs are similar in that a pair of redundant MDF devices is used in each basement with two riser links going to each IDF.
Design Discussion
Whereas Design 1 sought to blend Layer 2 and Layer 3 technology, Design 2 follows an approach that maximizes the Layer 3 content in the MDF/distribution layer switches. In doing so, this somewhat subtle change has a dramatic impact on the rest of the design.
The most important change created by this design is that all IDF VLANs are terminated at the MDF switch. In other words, users connected to different IDFs always fall in different VLANs. As discussed in Chapter 11, although it is possible to have a limited number of VLANs traverse a Catalyst 8500 using IRB, this is not a technique that you want to use many times throughout your campus (it is appropriate for one or two special-case VLANs). In other words, this style of Layer 3 switching is best used as a fast version of a normal routing.
The second most important change, a simplification of Spanning Tree, is discussed in the next section.
Spanning Tree
Although some view the loss of IDF-to-IDF VLANs as a downside to the approach taken in Design 2, it is important to offset this with the simplifications that hardware-based routing make possible. One of the most important simplifications involves the area of Layer 2 loops and the Spanning-Tree Protocol. In fact, hardware-based routing has completely eliminated the Layer 2 loops between the IDF and MDF switches. Whereas Design 1 used Layer 2 triangles, this design uses Layer 2 V’s.
Note
As was stressed in the discussion of Design 1, MLS can be used to build loop-free Layer 2 V’s. However, it is important to realize that switching routers such as the 8500 do this by default, whereas MLS (and routing switches) require you to manually prune certain VLANs from selected links. See the earlier section “Trunks” for more information.
Because this design removes all Layer 2 loops (at least the ones that are intentionally formed), some organizations have decided to completely disable Spanning Tree when using this approach. However, because it does not prevent unintentional loops on a single IDF switch (generally as the result of a cabling mistake), other network designers want to maintain a Spanning Tree security blanket on their IDF switches. However, it is important to recognize that even in the cases where Spanning Tree remains enabled (as it is in Design 2), the operation of the Spanning-Tree Protocol is dramatically simplified for a variety of reasons.
First, Root Bridge placement becomes a non-issue. Each IDF switch is not aware of any other switches and naturally elects itself as the Root Bridge.
- Tip
It can still be a good idea to lower the Bridge Priority in case someone plugs in another bridge some day.
In addition, Spanning Tree load balancing is not required (or, for that matter, possible).
Also, features such as UplinkFast and BackboneFast are no longer necessary for fast convergence.
Finally, the Spanning Tree network diameter has been reduced to the IDF switch itself. As a result, the Max Age and Forward Delay times can be aggressively tuned without concern. For example, Design 2 specifies a Max Age of 10 seconds and a Forward Delay of 7 seconds. Although somewhat more aggressive values can be used, these were chosen as a conservative compromise. As a result, failover performance where a loop exists is between 14 and 20 seconds. However, because the topology is loop free at Layer 2, there should be no Blocking ports during normal operation. As a result, IDF uplink failover performance is governed by HSRP, not Spanning Tree. Also as a result, the network can recover from uplink failures in as little as one second (assuming that the HSRP parameters are lowered).
- Tip
The Spanning-Tree Protocol does not affect failover performance in this network.
VLAN Design
Although the concept of a VLAN begins to blur (or fade) in this design, the IDF switches are configured with the same end user VLAN names as used in Design 1. However, notice that all of the VLANs use essentially the same numbers throughout this version of the design. The management VLAN in all switches is always VLAN 1 (even though they are different IP subnets). Similarly, the first end-user VLAN on an IDF switch is VLAN 2. If more than one VLAN is required on a given IDF switch, VLANs 3 and greater can be created.
Notice that this brings a completely different approach to user mobility than Design 1. Design 1 attempted to place all users in the same community of interest located within a single building in the same VLAN. In the case of Design 2, that is no longer possible without enabling IRB on the Catalyst 8500. Here, it is expected that users in the same community of interest may very well fall into different subnets. However, because DHCP is in use, IP addressing is transparent to the users. Furthermore, because the available Layer 3 bandwidth is so high with 8500 technology, the use of routing (Layer 3 switching) does not impair the network’s performance.
Note
Note that a similar case for Layer 3 performance can be made for the Catalyst 6000/6500. See Chapter 18 for more detail.
IP and IPX Addresses
Because Design 2 is less flat than Design 1, it requires more IP subnets (and IPX networks). For example, every link through the core is a separate subnet. Furthermore, every IDF uses a separate subnet as a management VLAN (remember, all VLAN terminate at the MDF switches). To avoid using an excessive number of address space, variable length subnet masking (VLSM) has been specified in Design 2.
Although in reality this is not a concern for most organizations using the Class A network such as network 10.0.0.0, it provides another benefit by making the subnets appear similar to the subnets used in Design 1. For example, whereas Design 1 uses a single backbone subnet of 10.250.250.0/24, Design 2 uses multiple 10.250.250.0/29 subnets. Just as Design 1 uses 10.1.10.0/24 and 10.2.20.0/24 for management VLANs, Design 2 uses multiple smaller subnets of 10.1.10.0/29 and 10.2.20.0/29.
As a result, Design 2 uses two subnet masks:
- /24 (255.255.255.0) for end-user segments
- /29 (255.255.255.248) for management VLANs, loopback addresses, and backbone links
Although it is possible to further optimize the address space utilization by using a /30 mask (255.255.255.252) for loopback interfaces and backbone links, a common mask was chosen for simplicity (furthermore, this one-bit optimization quickly reaches a point of diminishing returns when working with a Class A address!). Table 17-4 shows the IP subnets along with the corresponding IPX network numbers.
Table 17-4. IP Subnets and IPX Networks for Design 2
| Use | Description | Bldg | VLAN | Subnet | Mask | IPX Net |
| B1_Mgt | Cat-B1-1A SC0 | 1 | 1 | 10.1.10.8 | /29 | 0A010A08 |
| B1_Mgt | Cat-B1-2A SC0 | 1 | 1 | 10.1.10.16 | /29 | 0A010A10 |
| B1_Mgt | Cat-B1-3A SC0 | 1 | 1 | 10.1.10.24 | /29 | 0A010A18 |
| B1_Sales | End-user segment | 1 | 2 | 10.1.11.0 | /24 | 0A010B00 |
| B1_Mkting | End-user segment | 1 | 3 | 10.1.12.0 | /24 | 0A010C00 |
| B1_Eng | End-user segment | 1 | 2 | 10.1.13.0 | /24 | 0A010D00 |
| B1_Finance | End-user segment | 1 | 2 | 10.1.14.0 | /24 | 0A010E00 |
| B2_Mgt | Cat-B2-1A SC0 | 2 | 1 | 10.2.20.8 | /29 | 0A021408 |
| B2_Mgt | Cat-B2-2A SC0 | 2 | 1 | 10.2.20.16 | /29 | 0A021410 |
| B2_Mgt | Cat-B2-3A SC0 | 2 | 1 | 10.2.20.24 | /29 | 0A021418 |
| B2_Sales | End-user segment | 2 | 2 | 10.2.21.0 | /24 | 0A021500 |
| B2_Mkting | End-user segment | 2 | 3 | 10.2.22.0 | /24 | 0A021600 |
| B2_Eng | End-user segment | 2 | 2 | 10.2.23.0 | /24 | 0A021700 |
| B2_Finance | End-user segment | 2 | 2 | 10.2.24.0 | /24 | 0A021800 |
| Svr. Farm | Server Farm segment | Backbone | 100 | 10.100.100.0 | /24 | 0A646400 |
| Loopback | Cat-B1-0A | Backbone | N/A | 10.200.200.8 | /29 | 0AC8C808 |
| Loopback | Cat-B1-0B | Backbone | N/A | 10.200.200.16 | /29 | 0AC8C810 |
| Loopback | Cat-B2-0A | Backbone | N/A | 10.200.200.24 | /29 | 0AC8C818 |
| Loopback | Cat-B2-0B | Backbone | N/A | 10.200.200.32 | /29 | 0AC8C820 |
| Backbone | Cat-B1-0A to Cat-B1-0B | Backbone | N/A | 10.250.250.8 | /29 | 0AFAFA08 |
| Backbone | Cat-B1-0A to Cat-B2-0B | Backbone | N/A | 10.250.250.16 | /29 | 0AFAFA10 |
| Backbone | Cat-B1-0A to Cat-B2-0A | Backbone | N/A | 10.250.250.24 | /29 | 0AFAFA18 |
| Backbone | Cat-B1-0B to Cat-B2-0B | Backbone | N/A | 10.250.250.32 | /29 | 0AFAFA20 |
| Backbone | Cat-B1-0B to Cat-B2-0A | Backbone | N/A | 10.250.250.40 | /29 | 0AFAFA28 |
| Backbone | Cat-B2-0A to Cat-B2-0B | Backbone | N/A | 10.250.250.48 | /29 | 0AFAFA30 |
VTP
Given the Layer 3 nature of Design 2, VTP server mode has little meaning (8500s do not propagate VTP frames). Therefore, Design 2 calls for VTP transparent mode. Although not a requirement, the design also calls for a VTP domain name of Happy (unlike server and client modes, transparent mode does not require a VTP domain name).
As a result, each IDF switch must be individually configured with the list of VLANs it must handle. However, this is rarely a significant issue because each IDF switch usually only handles a small number of VLANs.
- Tip
If the VLAN configuration tasks are a concern (or, for that matter any other configuration task), consider using tools such as Perl and Expect. Both run on a wide variety of UNIX platforms as well as Windows NT.
Trunks
To present an alternative approach, Design 2 uses Fast EtherChannel links between the MDF and IDF switches. To provide adequate bandwidth in the core, Gigabit Ethernet links are used.
Server Farm
This design calls for a separate Server Farm building (a third building at the corporate headquarters campus will be used). The Server Farm could have easily been placed in Building 1 as it was with Design 1, however, an alternate approach was used for variety.
Configurations
This section presents the configurations for Design 2. As with Design 1, you see only one example of each type of device. First, you see configurations for and discussion of a Catalyst 5509 IDF switch, followed by configurations for and discussion of a Catalyst 8540 MDF switch.
IDF Supervisor Configuration
As with Design 1, this section is broken into two sections:
- The interactive configuration output
- The full configuration listing
Configuring an IDF Supervisor: Cat-B2-1A
As with the IDF switch in Design 1, begin by configuring the device VTP domain names as in Example 17-21.
Example 17-21 System Name and VTP Configuration
Console> (enable) set system name Cat-B2-1A System name set. Cat-B2-1A> (enable) set vtp domain Happy VTP domain Happy modified Cat-B2-1A> (enable)
Unlike Design 1, this design utilizes VTP transparent mode and requires only a single end-user VLAN for Cat-B2-1A as shown in Example 17-22.
Example 17-22 VTP and VLAN Configration
Cat-B2-1A> (enable) set vtp mode transparent VTP domain Happy modified Cat-B2-1A> (enable) Cat-B2-1A> (enable) set vlan 2 name Engineering Vlan 2 configuration successful Cat-B2-1A> (enable)
The SC0 interface also uses a different configuration under Design 2. First, the IP address and netmask are obviously different. Second, SC0 is left in VLAN 1, the default. Third, Design 2 calls for two default gateway addresses to be specified with the ip route command (this feature was first supported in Version 4.1 of Catalyst 5000 code). This can simplify the overall configuration and maintenance of the network by not requiring a separate HSRP group to be maintained for each management subnet/VLAN. Example 17-23 demonstrates these steps.
Example 17-23 IP Configuration
Cat-B2-1A> (enable) set interface sc0 1 10.2.20.11 255.255.255.248 Interface sc0 vlan set, IP address and netmask set. Cat-B2-1A> (enable) set ip route default 10.2.20.9 Route added. Cat-B2-1A> (enable) set ip route default 10.2.20.10 Route added. Cat-B2-1A> (enable) Next, configure the Spanning Tree parameters as in Example 17-24. Example 17-24 Spanning Tree Configuration Cat-B2-1A> (enable) set spantree root 1 dia 2 hello 2 VLAN 1 bridge priority set to 8192. VLAN 1 bridge max aging time set to 10. VLAN 1 bridge hello time set to 2. VLAN 1 bridge forward delay set to 7. Switch is now the root switch for active VLAN 1. Cat-B2-1A> (enable) Cat-B2-1A> (enable) set spantree root 2 dia 2 hello 2 VLAN 2 bridge priority set to 8192. VLAN 2 bridge max aging time set to 10. VLAN 2 bridge hello time set to 2. VLAN 2 bridge forward delay set to 7. Switch is now the root switch for active VLAN 2. Cat-B2-1A> (enable) Cat-B2-1A> (enable) set spantree portfast 4/1-24,5/1-24,6/1-24,7/1-24, 8/1-24 enable Warning: Spantree port fast start should only be enabled on ports connected to a single host. Connecting hubs, concentrators, switches, bridges, etc. to a fast start port can cause temporary Spanning Tree loops. Use with caution. Spantree ports 4/1-24,5/1-24,6/1-24,7/1-24,8/1-24 fast start enabled. Cat-B2-1A> (enable)
The first two commands (set spantree root) lower the Max Age and Forward Delay timers to 10 and 7 seconds, respectively. For consistency, this also forces the IDF switch to be the Root Bridge. (Although this is useful in the event that other switches or bridges have been cascaded off the IDF switch, in most situations this has no impact on the actual topology under Design 2.) Finally, PortFast is enabled on all of the end-user ports in slots 4–8.
Next, the trunk ports are configured as in Example 17-25.
Example 17-25 Port and Trunk Configuration
Cat-B2-1A> (enable) Cat-B2-1A> (enable) set port name 3/1-4 FEC link to Cat-B2-0A Port 3/1-4 name set. Cat-B2-1A> (enable) set port name 3/5-8 FEC link to Cat-B2-0B Port 3/5-8 name set. Cat-B2-1A> (enable) B2-MDF-02> (enable) set port channel 3/1-4 on Port(s) 3/1-4 channel mode set to on. B2-MDF-02> (enable) set port channel 3/5-8 on Port(s) 3/5-8 channel mode set to on. Cat-B2-1A> (enable) Cat-B2-1A> (enable) set trunk 3/1 on isl Port(s) 3/1-4 trunk mode set to on. Port(s) 3/1-4 trunk type set to isl. Cat-B2-1A> (enable) Cat-B2-1A> (enable) set trunk 3/5 on isl Port(s) 3/5-8 trunk mode set to on. Port(s) 3/5-8 trunk type set to isl. Cat-B2-1A> (enable)
As mentioned earlier, Design 2 uses Fast EtherChannel links from Cat-B2-1A and Cat-B2-2A to the MDF switches. For stability, these are hard-coded to the port channel on state. The resulting EtherChannel bundles are also hard-coded as ISL trunks. Also notice that although the set trunk command is only applied to a single port, the Catalyst automatically applies it to every port in the EtherChannel bundle.
The commands in Example 17-26 are very similar to those used in Example 17-16 of Design 1.
Example 17-26 Configuring SNMP, Password, Banner, System Information, DNS, IP Permit List, IGMP Snooping, Protocol Filtering, SNMP, and Syslog
Cat-B2-1A> (enable) set snmp community read-only lesspublic SNMP read-only community string set to 'lesspublic'. Cat-B2-1A> (enable) set snmp community read-write moreprivate SNMP read-write community string set to 'moreprivate'. Cat-B2-1A> (enable) set snmp community read-write-all mostprivate SNMP read-write-all community string set to 'mostprivate'. Cat-B2-1A> (enable) Cat-B2-1A> (enable) set password Enter old password: Enter new password: Retype new password: Password changed. Cat-B2-1A> (enable) Cat-B2-1A> (enable) set enablepass Enter old password: Enter new password: Retype new password: Password changed. Cat-B2-1A> (enable) Cat-B2-1A> (enable) Cat-B2-1A> (enable) set banner motd ~PRIVATE NETWORK -- HACKERS WILL BE SHOT!!~ MOTD banner set Cat-B2-1A> (enable) set system location Building 2 First Floor System location set. Cat-B2-1A> (enable) set system contact Joe x111 System contact set. Cat-B2-1A> (enable) Cat-B2-1A> (enable) set ip dns enable DNS is enabled Cat-B2-1A> (enable) set ip dns domain happy.com Default DNS domain name set to happy.com Cat-B2-1A> (enable) set ip dns server 10.100.100.42 10.100.100.42 added to DNS server table as primary server. Cat-B2-1A> (enable) set ip dns server 10.100.100.68 10.100.100.68 added to DNS server table as backup server. Cat-B2-1A> (enable) Cat-B2-1A> (enable) set ip permit enable IP permit list enabled. WARNING!! IP permit list has no entries. Cat-B2-1A> (enable) set ip permit 10.100.100.0 255.255.255.0 10.100.100.0 with mask 255.255.255.0 added to IP permit list. Cat-B2-1A> (enable) Cat-B2-1A> (enable) Cat-B2-1A> (enable) set igmp enable IGMP feature for IP multicast enabled Cat-B2-1A> (enable) Cat-B2-1A> (enable) set protocolfilter enable Protocol filtering enabled on this switch. Cat-B2-1A> (enable) Cat-B2-1A> (enable) Cat-B2-1A> (enable) set snmp trap 10.100.100.21 trapped SNMP trap receiver added. Cat-B2-1A> (enable) set snmp trap enable module SNMP module traps enabled. Cat-B2-1A> (enable) set snmp trap enable chassis SNMP chassis alarm traps enabled. Cat-B2-1A> (enable) set snmp trap enable bridge SNMP bridge traps enabled. Cat-B2-1A> (enable) set snmp trap enable auth SNMP authentication traps enabled. Cat-B2-1A> (enable) set snmp trap enable stpx SNMP STPX traps enabled. Cat-B2-1A> (enable) set snmp trap enable config SNMP CONFIG traps enabled. Cat-B2-1A> (enable) set port trap 3/1-8 enable Port 3/1-8 up/down trap enabled. Cat-B2-1A> (enable) Cat-B2-1A> (enable) Cat-B2-1A> (enable) set logging server enable System logging messages will be sent to the configured syslog servers. Cat-B2-1A> (enable) set logging server 10.100.100.21 10.100.100.21 added to System logging server table. Cat-B2-1A> (enable) Cat-B2-1A> (enable)
Full IDF Supervisor Listing: Cat-B2-1A
Example 17-27 presents the configuration code that results from the previous sequence of configuration steps.
Example 17-27 Full Catalyst Configuration
begin ! set password $1$FMFQ$HfZR5DUszVHIRhrz4h6V70 set enablepass $1$FMFQ$HfZR5DUszVHIRhrz4h6V70 set prompt Cat-B2-1A> set length 24 default set logout 20 set banner motd ^CPRIVATE NETWORK -- HACKERS WILL BE SHOT!!^C ! #system set system baud 9600 set system modem disable set system name Cat-B2-1A set system location Building 2 First Floor set system contact Joe x111 ! #snmp set snmp community read-only lesspublic set snmp community read-write moreprivate set snmp community read-write-all mostprivate set snmp rmon disable set snmp trap enable module set snmp trap enable chassis set snmp trap enable bridge set snmp trap disable repeater set snmp trap disable vtp set snmp trap enable auth set snmp trap disable ippermit set snmp trap disable vmps set snmp trap disable entity set snmp trap enable config set snmp trap enable stpx set snmp trap disable syslog set snmp extendedrmon vlanmode disable set snmp extendedrmon vlanagent disable set snmp extendedrmon enable set snmp trap 10.100.100.21 trapped ! #ip set interface sc0 1 10.2.20.11 255.255.255.248 10.2.10.2.20.15 set interface sc0 up set interface sl0 0.0.0.0 0.0.0.0 set interface sl0 up set arp agingtime 1200 set ip redirect enable set ip unreachable enable set ip fragmentation enable set ip route 0.0.0.0 10.2.20.9 1 set ip route 0.0.0.0 10.2.20.10 1 set ip alias default 0.0.0.0 ! #Command alias ! #vmps set vmps server retry 3 set vmps server reconfirminterval 60 set vmps tftpserver 0.0.0.0 vmps-config-database.1 set vmps state disable ! #dns set ip dns server 10.100.100.42 primary set ip dns server 10.100.100.68 set ip dns enable set ip dns domain happy.com ! #tacacs+ set tacacs attempts 3 set tacacs directedrequest disable set tacacs timeout 5 ! #authentication set authentication login tacacs disable console set authentication login tacacs disable telnet set authentication enable tacacs disable console set authentication enable tacacs disable telnet set authentication login local enable console set authentication login local enable telnet set authentication enable local enable console set authentication enable local enable telnet ! #bridge set bridge ipx snaptoether 8023raw set bridge ipx 8022toether 8023 set bridge ipx 8023rawtofddi snap ! #vtp set vtp domain Happy set vtp mode transparent set vtp v2 disable set vtp pruning disable set vtp pruneeligible 2-1000 clear vtp pruneeligible 1001-1005 set vlan 1 name default type ethernet mtu 1500 said 100001 state active set vlan 2 name Engineering type ethernet mtu 1500 said 100002 state active set vlan 1002 name fddi-default type fddi mtu 1500 said 101002 state active set vlan 1004 name fddinet-default type fddinet mtu 1500 said 101004 state active bridge 0x0 stp ieee set vlan 1005 name trnet-default type trbrf mtu 1500 said 101005 state active bridge 0x0 stp ibm set vlan 1003 name token-ring-default type trcrf mtu 1500 said 101003 state active parent 0 ring 0x0 mode srb aremaxhop 0 stemaxhop 0 ! #spantree #uplinkfast groups set spantree uplinkfast disable #backbonefast set spantree backbonefast disable set spantree enable all #vlan 1 set spantree fwddelay 7 1 set spantree hello 2 1 set spantree maxage 10 1 set spantree priority 8192 1 #vlan 2 set spantree fwddelay 7 2 set spantree hello 2 2 set spantree maxage 10 2 set spantree priority 8192 2 #vlan 1003 set spantree fwddelay 15 1003 set spantree hello 2 1003 set spantree maxage 20 1003 set spantree priority 32768 1003 set spantree portstate 1003 block 0 set spantree portcost 1003 62 set spantree portpri 1003 4 set spantree portfast 1003 disable #vlan 1005 set spantree fwddelay 15 1005 set spantree hello 2 1005 set spantree maxage 20 1005 set spantree priority 32768 1005 set spantree multicast-address 1005 ieee ! #cgmp set cgmp disable set cgmp leave disable ! #syslog set logging console enable set logging server enable set logging server 10.100.100.21 set logging level cdp 2 default set logging level mcast 2 default set logging level dtp 5 default set logging level dvlan 2 default set logging level earl 2 default set logging level fddi 2 default set logging level ip 2 default set logging level pruning 2 default set logging level snmp 2 default set logging level spantree 2 default set logging level sys 5 default set logging level tac 2 default set logging level tcp 2 default set logging level telnet 2 default set logging level tftp 2 default set logging level vtp 2 default set logging level vmps 2 default set logging level kernel 2 default set logging level filesys 2 default set logging level drip 2 default set logging level pagp 5 default set logging level mgmt 5 default set logging level mls 5 default set logging level protfilt 2 default set logging level security 2 default set logging server facility LOCAL7 set logging server severity 4 set logging buffer 500 set logging timestamp disable ! #ntp set ntp broadcastclient disable set ntp broadcastdelay 3000 set ntp client disable clear timezone set summertime disable ! #set boot command set boot config-register 0x10f set boot system flash bootflash:sup.bin ! #permit list set ip permit enable set ip permit 10.100.100.0 255.255.255.0 ! #drip set tokenring reduction enable set tokenring distrib-crf disable ! #igmp set igmp enable ! #protocolfilter set protocolfilter enable ! #mls set mls enable set mls flow destination set mls agingtime 256 set mls agingtime fast 0 0 set mls nde disable ! #standby ports set standbyports enable ! #module 1 : 2-port 10/100BaseTX Supervisor set module name 1 set vlan 1 1/1-2 set port channel 1/1-2 off set port channel 1/1-2 auto set port enable 1/1-2 set port level 1/1-2 normal set port speed 1/1-2 auto set port trap 1/1-2 disable set port name 1/1-2 set port security 1/1-2 disable set port broadcast 1/1-2 100% set port membership 1/1-2 static set port protocol 1/1-2 ip on set port protocol 1/1-2 ipx auto set cdp enable 1/1-2 set cdp interval 1/1-2 60 set trunk 1/1 auto isl 1-1005 set trunk 1/2 auto isl 1-1005 set spantree portfast 1/1-2 disable set spantree portcost 1/1-2 100 set spantree portpri 1/1-2 32 set spantree portvlanpri 1/1 0 set spantree portvlanpri 1/2 0 set spantree portvlancost 1/1 cost 99 set spantree portvlancost 1/2 cost 99 ! #module 2 : 2-port 10/100BaseTX Supervisor set module name 2 set vlan 1 2/1-2 set port channel 2/1-2 off set port channel 2/1-2 auto set port enable 2/1-2 set port level 2/1-2 normal set port speed 2/1-2 auto set port trap 2/1-2 disable set port name 2/1-2 set port security 2/1-2 disable set port broadcast 2/1-2 100% set port membership 2/1-2 static set port protocol 2/1-2 ip on set port protocol 2/1-2 ipx auto set cdp enable 2/1-2 set cdp interval 2/1-2 60 set trunk 2/1 auto isl 1-1005 set trunk 2/2 auto isl 1-1005 set spantree portfast 2/1-2 disable set spantree portcost 2/1-2 100 set spantree portpri 2/1-2 32 set spantree portvlanpri 2/1 0 set spantree portvlanpri 2/2 0 set spantree portvlancost 2/1 cost 99 set spantree portvlancost 2/2 cost 99 ! #module 3 : 12-port 10/100BaseTX Ethernet set module name 3 set module enable 3 set vlan 1 3/1-12 set port channel 3/1-4 off set port channel 3/5-8 off set port channel 3/9-12 off set port channel 3/1-4 on set port channel 3/5-8 on set port channel 3/9-12 auto set port enable 3/1-12 set port level 3/1-12 normal set port speed 3/1-12 100 set port duplex 3/1-12 full set port trap 3/1-8 enable set port trap 3/9-12 disable set port name 3/1 FEC link to Cat-B2-0A set port name 3/2 FEC link to Cat-B2-0A set port name 3/3 FEC link to Cat-B2-0A set port name 3/4 FEC link to Cat-B2-0A set port name 3/5 FEC link to Cat-B2-0B set port name 3/6 FEC link to Cat-B2-0B set port name 3/7 FEC link to Cat-B2-0B set port name 3/8 FEC link to Cat-B2-0B set port name 3/9-12 set port security 3/1-12 disable set port broadcast 3/1-12 0 set port membership 3/1-12 static set port protocol 3/1-12 ip on set port protocol 3/1-12 ipx auto set cdp enable 3/1-12 set cdp interval 3/1-12 60 set trunk 3/1 on isl 1-1005 set trunk 3/2 on isl 1-1005 set trunk 3/3 on isl 1-1005 set trunk 3/4 on isl 1-1005 set trunk 3/5 on isl 1-1005 set trunk 3/6 on isl 1-1005 set trunk 3/7 on isl 1-1005 set trunk 3/8 on isl 1-1005 set trunk 3/9 auto isl 1-1005 set trunk 3/10 auto isl 1-1005 set trunk 3/11 auto isl 1-1005 set trunk 3/12 auto isl 1-1005 set spantree portfast 3/1-12 disable set spantree portcost 3/1-12 19 set spantree portpri 3/1-12 32 set spantree portvlanpri 3/1 0 set spantree portvlanpri 3/2 0 set spantree portvlanpri 3/3 0 set spantree portvlanpri 3/4 0 set spantree portvlanpri 3/5 0 set spantree portvlanpri 3/6 0 set spantree portvlanpri 3/7 0 set spantree portvlanpri 3/8 0 set spantree portvlanpri 3/9 0 set spantree portvlanpri 3/10 0 set spantree portvlanpri 3/11 0 set spantree portvlanpri 3/12 0 set spantree portvlancost 3/1 cost 18 set spantree portvlancost 3/2 cost 18 set spantree portvlancost 3/3 cost 18 set spantree portvlancost 3/4 cost 18 set spantree portvlancost 3/5 cost 18 set spantree portvlancost 3/6 cost 18 set spantree portvlancost 3/7 cost 18 set spantree portvlancost 3/8 cost 18 set spantree portvlancost 3/9 cost 18 set spantree portvlancost 3/10 cost 18 set spantree portvlancost 3/11 cost 18 set spantree portvlancost 3/12 cost 18 ! #module 5 : 24-port 10/100BaseTX Ethernet set module name 5 set module enable 5 set vlan 2 5/1-24 set port enable 5/1-24 set port level 5/1-24 normal set port speed 5/1-24 auto set port trap 5/1-24 disable set port name 5/1-24 set port security 5/1-24 disable set port broadcast 5/1-24 0 set port membership 5/1-24 static set port protocol 5/1-24 ip on set port protocol 5/1-24 ipx auto set cdp enable 5/1-24 set cdp interval 5/1-24 60 set spantree portfast 5/1-24 enable set spantree portcost 5/1-24 100 set spantree portpri 5/1-24 32 ! #module 6 : 24-port 10/100BaseTX Ethernet set module name 6 set module enable 6 set vlan 2 6/1-24 set port enable 6/1-24 set port level 6/1-24 normal set port speed 6/1-24 auto set port trap 6/1-24 disable set port name 6/1-24 set port security 6/1-24 disable set port broadcast 6/1-24 0 set port membership 6/1-24 static set port protocol 6/1-24 ip on set port protocol 6/1-24 ipx auto set cdp enable 6/1-24 set cdp interval 6/1-24 60 set spantree portfast 6/1-24 enable set spantree portcost 6/1-24 100 set spantree portpri 6/1-24 32 ! #module 7 : 24-port 10/100BaseTX Ethernet set module name 7 set module enable 7 set vlan 2 7/1-24 set port enable 7/1-24 set port level 7/1-24 normal set port speed 7/1-24 auto set port trap 7/1-24 disable set port name 7/1-24 set port security 7/1-24 disable set port broadcast 7/1-24 0 set port membership 7/1-24 static set port protocol 7/1-24 ip on set port protocol 7/1-24 ipx auto set cdp enable 7/1-24 set cdp interval 7/1-24 60 set spantree portfast 7/1-24 enable set spantree portcost 7/1-24 100 set spantree portpri 7/1-24 32 ! #module 8 : 24-port 10/100BaseTX Ethernet set module name 8 set module enable 8 set vlan 2 8/1-24 set port enable 8/1-24 set port level 8/1-24 normal set port speed 8/1-24 auto set port trap 8/1-24 disable set port name 8/1-24 set port security 8/1-24 disable set port broadcast 8/1-24 0 set port membership 8/1-24 static set port protocol 8/1-24 ip on set port protocol 8/1-24 ipx auto set cdp enable 8/1-24 set cdp interval 8/1-24 60 set spantree portfast 8/1-24 enable set spantree portcost 8/1-24 100 set spantree portpri 8/1-24 32 ! #module 9 empty ! #switch port analyzer !set span 1 1/1 both inpkts disable set span disable ! #cam set cam agingtime 1,2,1003,1005 300 end
MDF Configuration: Cat-B2-0B
Example 17-28 presents the full configuration listing for Cat-B2-0B, an 8540 MDF switch. The chassis contains a 16-port 100BaseFX module in slot 0 and 2-port Gigabit Ethernet modules in slots 1 and 2. Because IOS-based router configurations are shorter (they only list non-default commands) and easier to read than XDI/CatOS-based Catalyst images, this section does not show a separate listing of the interactive command output.
Example 17-28 Full Catalyst 8540 Configuration
! no service pad service timestamps log datetime localtime service password-encryption ! hostname Cat-B2-0B ! logging buffered 4096 debugging logging console informational enable secret 5 $1$C3lJ$qVaCyxa7mpq2OXMzTHY7h1 ! clock timezone EST -5 clock summer-time EDT recurring redundancy main-cpu no sync config startup sync config running facility-alarm core-temperature major 53 facility-alarm core-temperature minor 45 ip subnet-zero ip domain-name happy.com ip name-server 10.100.100.42 ip name-server 10.100.100.68 ipx routing 0090.2149.2400 ! ! interface Loopback0 ip address 10.200.200.33 255.255.255.248 no ip directed-broadcast ! interface Port-channel1 description Link to Cat-B2-1A no ip address no ip directed-broadcast hold-queue 300 in ! interface Port-channel1.1 description Mgt VLAN: Cat-B2-1A SC0 encapsulation isl 1 ip address 10.2.20.9 255.255.255.248 no ip redirects no ip directed-broadcast ! interface Port-channel1.2 description User VLAN: Engineering encapsulation isl 2 ip address 10.2.23.4 255.255.255.0 ip helper-address 10.100.100.81 ip helper-address 10.100.100.33 no ip redirects no ip directed-broadcast ipx network 0A021700 standby 1 priority 100 standby 1 preempt standby 1 ip 10.2.23.1 standby 1 track GigabitEthernet1/0/0 7 standby 1 track GigabitEthernet1/0/1 7 standby 1 track GigabitEthernet2/0/0 7 standby 2 priority 110 standby 2 preempt standby 2 ip 10.2.23.2 standby 2 track GigabitEthernet1/0/0 7 standby 2 track GigabitEthernet1/0/1 7 standby 2 track GigabitEthernet2/0/0 7 ! interface Port-channel2 description Link to Cat-B2-2A no ip address no ip directed-broadcast hold-queue 300 in ! interface Port-channel2.1 description Mgt VLAN: Cat-B2-2A SC0 encapsulation isl 1 ip address 10.2.20.17 255.255.255.248 no ip redirects no ip directed-broadcast ! interface Port-channel2.2 description User VLAN: Finance encapsulation isl 2 ip address 10.2.24.4 255.255.255.0 ip helper-address 10.100.100.81 ip helper-address 10.100.100.33 no ip redirects no ip directed-broadcast ipx network 0A021800 standby 1 priority 100 standby 1 preempt standby 1 ip 10.2.24.1 standby 1 track GigabitEthernet1/0/0 7 standby 1 track GigabitEthernet1/0/1 7 standby 1 track GigabitEthernet2/0/0 7 standby 2 priority 110 standby 2 preempt standby 2 ip 10.2.24.2 standby 2 track GigabitEthernet1/0/0 7 standby 2 track GigabitEthernet1/0/1 7 standby 2 track GigabitEthernet2/0/0 7 ! interface Port-channel2.3 description User VLAN: Mkting encapsulation isl 3 ip address 10.2.22.4 255.255.255.0 ip helper-address 10.100.100.81 ip helper-address 10.100.100.33 no ip redirects no ip directed-broadcast ipx network 0A021600 standby 1 priority 100 standby 1 preempt standby 1 ip 10.2.22.1 standby 1 track GigabitEthernet1/0/0 7 standby 1 track GigabitEthernet1/0/1 7 standby 1 track GigabitEthernet2/0/0 7 standby 2 priority 110 standby 2 preempt standby 2 ip 10.2.22.2 standby 2 track GigabitEthernet1/0/0 7 standby 2 track GigabitEthernet1/0/1 7 standby 2 track GigabitEthernet2/0/0 7 ! interface Port-channel3 description Link to Cat-B2-3A no ip address no ip directed-broadcast hold-queue 300 in ! interface Port-channel3.1 description Mgt VLAN: Cat-B2-3A SC0 encapsulation isl 1 ip address 10.2.20.25 255.255.255.248 no ip redirects no ip directed-broadcast ! interface Port-channel3.2 description User VLAN: Sales encapsulation isl 2 ip address 10.2.21.4 255.255.255.0 ip helper-address 10.100.100.81 ip helper-address 10.100.100.33 no ip redirects no ip directed-broadcast ipx network 0A021500 standby 1 priority 100 standby 1 preempt standby 1 ip 10.2.21.1 standby 1 track GigabitEthernet1/0/0 7 standby 1 track GigabitEthernet1/0/1 7 standby 1 track GigabitEthernet2/0/0 7 standby 2 priority 110 standby 2 preempt standby 2 ip 10.2.21.2 standby 2 track GigabitEthernet1/0/0 7 standby 2 track GigabitEthernet1/0/1 7 standby 2 track GigabitEthernet2/0/0 7 ! interface FastEthernet0/0/0 no ip address no ip directed-broadcast channel-group 1 ! interface FastEthernet0/0/1 no ip address no ip directed-broadcast channel-group 1 ! interface FastEthernet0/0/2 no ip address no ip directed-broadcast channel-group 1 ! interface FastEthernet0/0/3 no ip address no ip directed-broadcast channel-group 1 ! interface FastEthernet0/0/4 no ip address no ip directed-broadcast channel-group 2 ! interface FastEthernet0/0/5 no ip address no ip directed-broadcast channel-group 2 ! interface FastEthernet0/0/6 no ip address no ip directed-broadcast channel-group 2 ! interface FastEthernet0/0/7 no ip address no ip directed-broadcast channel-group 2 ! interface FastEthernet0/0/8 no ip address no ip directed-broadcast channel-group 3 ! interface FastEthernet0/0/9 no ip address no ip directed-broadcast channel-group 3 ! interface FastEthernet0/0/10 no ip address no ip directed-broadcast channel-group 3 ! interface FastEthernet0/0/11 no ip address no ip directed-broadcast channel-group 3 ! interface FastEthernet0/0/12 no ip address no ip directed-broadcast shutdown ! interface FastEthernet0/0/13 no ip address no ip directed-broadcast shutdown ! interface FastEthernet0/0/14 no ip address no ip directed-broadcast shutdown ! interface FastEthernet0/0/15 no ip address no ip directed-broadcast shutdown ! interface GigabitEthernet1/0/0 description Gigabit link: Cat-B1-0A to Cat-B2-0B ip address 10.250.250.18 255.255.255.248 no ip directed-broadcast ipx network 0AFAFA10 no negotiation auto ! interface GigabitEthernet1/0/1 description Gigabit link: Cat-B1-0B to Cat-B2-0B ip address 10.250.250.34 255.255.255.248 no ip directed-broadcast ipx network 0AFAFA20 no negotiation auto ! interface GigabitEthernet2/0/0 description Gigabit link: Cat-B2-0A to Cat-B2-0B ip address 10.250.250.50 255.255.255.248 no ip directed-broadcast ipx network 0AFAFA30 no negotiation auto ! interface GigabitEthernet2/0/1 description Gigabit link: Server Farm ip address 10.100.100.4 255.255.255.0 no ip redirects no ip directed-broadcast ipx network 0A646400 no negotiation auto ! interface Ethernet0 no ip address no ip directed-broadcast ! router eigrp 131 passive-interface Port-channel1.1 passive-interface Port-channel1.2 passive-interface Port-channel2.1 passive-interface Port-channel2.2 passive-interface Port-channel2.3 passive-interface Port-channel3.1 passive-interface Port-channel3.2 network 10.0.0.0 ! ip classless no ip forward-protocol udp netbios-ns no ip forward-protocol udp netbios-dgm ! logging 10.100.100.21 access-list 1 permit 10.100.100.0 0.0.0.255 snmp-server community lesspublic RO snmp-server community moreprivate RW snmp-server host 10.100.100.21 trapped snmp-server location Building 2 MDF snmp-server contact Joe x111 snmp-server enable traps config banner motd ^CPRIVATE NETWORK -- HACKERS WILL BE SHOT!!^C ! ! line con 0 password 7 055A545C transport input none line aux 0 password 7 055A545C line vty 0 4 access-class 1 in password 7 055A545C login ! end
Three logical port-channel interfaces are configured to handle the links to the three IDF switches. Because the EtherChannels are using ISL encapsulation to trunk multiple VLANs to the IDFs, each port-channel is then configured with multiple subinterfaces, one for each IDF VLAN. For example, interface port-channel 2 is used to connect to Cat-B2-2A on the second floor. Subinterface port-channel 2.1 is created for the management VLAN, 2.2 for the Finance VLAN, and 2.3 for the Marketing VLAN. Each subinterface is configured with an encapsulation isl statement and the appropriate IP and IPX Layer 3 information.
The subinterfaces supporting end-user traffic are also configured with two HSRP groups. As explained in Chapter 11, HSRP load balancing should be employed in designs where a single end-user VLAN is used on each IDF and there are no Layer 2 loops (making Spanning Tree load balancing impossible). To enable HSRP load balancing, a technique called Multigroup HSRP (MHSRP) is used. Under MHSRP, two (or more) HSRP groups are created for every subnet.
By having each MDF device be the active HSRP peer for one of the two HSRP groups, load balancing can be achieved. For example, Design 2 calls for two HSRP groups per end-user subnet (as mentioned earlier, the management VLANs use multiple default gateways instead). The first HSRP group uses .1 in the fourth octet of the IP address, and the second group uses .2. By making Cat-B2-0A the active peer for the first group and Cat-B2-0B the active peer for the second group, both router ports can be active at the same time.
Note
Note that the recommendation to use MHSRP is predicated upon the fact that a single VLAN is being used on the IDF switches (as discussed in Chapter 11, this is often done to facilitate ease of network management). If you are using multiple VLANs on the IDFs, you can simply alternate active HSRP peers between the VLANs. See Chapter 11 for more information and configuration examples.
The catch with this approach is finding some technique to have half of the end stations use the .1 default gateway address and the other half use .2. Chapter 11 suggests using DHCP for this purpose. For example, Happy Homes is planning to deploy two DHCP servers (from the ip helper-address statements, we can determine that the IP addresses are 10.100.100.33 and 10.100.100.81). All leases issued by the first DHCP server, 10.100.100.33, specify .1 as the default gateway.
On the other hand, all leases issued by the second DHCP server, 10.100.100.81, specify .2 as a default gateway. To help ensure a fairly random distribution of leases between two DHCP, the order of the ip helper-address statements can be inverted between the two MDF switches. For example, the configuration for Cat-B2-0B shows 10.100.100.81 as the first ip helper-address on every end-user subinterface. On the other MDF switch, Cat-B2-0A, 10.100.100.33 should be listed first.
Further down in the configuration, the actual Fast Ethernet ports are shown. Notice that these do not contain any direct configuration statements (the entire configuration is done on the logical port-channel interface). The only statement added to each interface is a channel-group command that includes the physical interface in the appropriate logical port-channel interface.
Because the Gigabit Ethernet interfaces are not using EtherChannel, the configuration is placed directly on the interface itself. Each interface receives an IP address and an IPX network statement. Because these interfaces do not connect to any end stations, HSRP and IP helper addresses are not necessary.
The remaining configuration commands set up the same management features discussed in the earlier configurations.
Design Alternatives
As with Design 1, hundreds of permutations are possible for Design 2. This section briefly discusses some of the more common alternatives.
First, as shown in Figure 17-5, Design 2 calls for a pair of 8500s for the server farm. Figure 17-7 illustrates a potential layout for the server farm under Design 2.
Figure 17-7. Detail of Server Farm for Design 2
In this plan, a pair of Catalyst 6500 switches are directly connected to the backbone via Cat-B1-0B and Cat-B2-0B. By using the Catalyst 6500’s MSFC Native IOS Mode, you can leverage the capability of these devices to simultaneous behave as both routing switches and switching routers (see Chapter 18 for more information on this capability). This gives you the flexibility to provide Layer 2 connectivity within the server farm while also utilizing Layer 3 to reach the backbone. In essence, the server farm becomes a miniature version of one of the buildings, but all contained within a pair of devices (the 6500s are acting like MDF and IDF devices at the same time).
As an alternative, some organizations have used the design shown in Figure 17-8.
Figure 17-8. Layer 2 Server Farm Design
In this example, the Layer 2 Catalysts (in this case, 4003s) have been directly connected to the existing 8540s, Cat-B1-0B and Cat-B2-0B. The advantage of this approach is that it saves the expense of two Layer 3 switches and potentially removes one router hop from the typical end-user data path.
Unfortunately, this design is susceptible to the same default gateway issues discussed earlier in association with directly connecting servers to the LANE cloud in Design 1. As a result, it can actually add router hops by unnecessarily forwarding traffic to the wrong building. (You can run HSRP, but all traffic is directed to the active peer. MHSRP can be used, but it is generally less effective with servers than end users because of their extremely high bandwidth consumption.) If you do implement this design, consider running a routing protocol on your servers.
However, potentially the most serious problem involves IP addressing and link failures. Consider the case of where the Gigabit Ethernet link between the 4000s fails—both 8500s continue trying to send all traffic destined to the server farm subnet out their rightmost port. For example, Cat-B2-0B still tries to reach servers connected to Server Farm A by sending the traffic first to Server Farm B. And if the link between Server Farm B and Server Farm A is down, the traffic obviously never reaches its destination. This is a classic case of the discontinuous subnet problem.
- Tip
Look for potential discontinuous subnets in your network. This can be especially important in mission-critical areas of your network such as a server farm.
Probably the most common modification to Design 2 entails using a Layer 2 core rather than directly connecting the MDF switches to each other with a full or partial mesh of Gigabit Ethernet links. Although the approach used in Design 2 is fine for smaller networks, a Layer 2 core is more scalable for several reasons:
- It is easier to add distribution blocks.
- It is easier to upgrade access bandwidth to one building block (simply upgrade the links to the Layer 2 core versus upgrading all the meshed bandwidth).
- Routing protocol peering is reduced from the distribution layer to the core.
The most common implementation is to use a pair of Layer 2 switches for redundancy (however, be careful to remove all Layer 2 loops in the core).
A third potential modification to Design 2 involves VLAN numbering. Notice that Design 2 uses the pattern-based VLAN numbering scheme discussed in Chapter 15. Because designs with a strong Layer 3 switching component effectively nullify the concept of VLANs being globally-unique broadcast domains, this approach is appropriate for designs such as Design 2. However, some organizations prefer to maintain globally-unique VLAN numbers even when utilizing Layer 3 switching. In this case, every subnet is mapped to a unique VLAN number. See Chapter 15 for more information on pattern-based versus globally-unique VLAN numbering schemes.
Finally, another option is to deploy Gigabit EtherChannel within the core and server farm. By offering considerably more available bandwidth, this can provide additional room for growth with the Happy Homes campus.