Troubleshooting VMWare ESX/ESXi NIC teaming issues and Juniper EX Switches

This article provides assistance to Juniper employees and support engineers, when troubleshooting VMware related issues that are reported on an EX switch.

When configuring Link aggregation between an EX switch and an ESX device, you might encounter a few issues, such as link flap or duplicate packets. This article helps in avoiding some of these issues. The troubleshooting check list provides below can assist with faster issue detection and resolution.

On the EX switch that is connected with the ESX/ESXi server, which is configured with NIC Teaming, you might experience the following issues:

  • traffic not passing correctly
  • link flaps
  • intermittent disconnects
  • duplicate packets generated between them

This article helps in resolving such issues by presenting a check list of Do’s and Don’ts on both the EX switch and ESX server.
NIC teaming allows users to group two or more physical NICs into a single logical network device, known as a bond. When a logical NIC is configured, the virtual machine is not aware of the underlying physical NICs. The packets being sent to the logical NIC are dispatched to one of the physical NICs in the bond and packets that arrive at any of the physical NICs are automatically directed to the appropriate logical NIC.

Note: NIC Teaming in VMware ESX Server supports the IEEE 802.3ad static link aggregation standard.

Do’s on the EX switch:

  • Enable the 802.3ad static link aggregation on EX switches.
  • The ESX/ESXi host supports NIC teaming only on a single physical switch or a single virtual switch.

Do’s on the ESX/ESXi server:

  • The only load balancing option for the vSwitch or vDistributed switch that can be used with link aggregation is IP Hash.
  • The virtual switch must have its load balancing method set to Route based on IP hash.

Don’ts on the EX switch:

  • Do not configure LACP on the EX side.
  • Do not change the default minimum-link parameter for link aggregation on EX switches, unless any specific design requirement is necessary.

Don’ts on the ESX/ESXi server:

  • Do not use beacon probing with IP Hash load balancing.
  • Do not configure standby or unused uplinks with IP Hash load balancing.
  • VMware supports only one LAG per vSwitch or vNetwork Distributed Switch (vDS).
  • Enabling route based on IP hash, without 802.3ad aggregation, or vice-versa disrupts networking; so you must first make the changes to the virtual switch. This way, the service console is not available, but the physical switch management interface is; so you can enable aggregation on the ports that are involved to restore networking.
  • Do not mix gigabit and fast Ethernet NIC’s. On ESX, you can mix Gigabit and Fast Ethernet NICs in the same teaming bond. However. 802.3ad requires the same speed interfaces to be part of a link aggregation.
  • The ESX server 2 NIC Teaming will also work with regular switches, without 802.3ad lag, and it will still support the outbound load balancing and fail-over. However, performance is not guaranteed, if users deploy enterprise switches without link aggregation features; primarily as the inbound traffic may come from unpredictable ports.
  • Do not enable the NIC Teaming failback feature, if 802.3ad is not used on the connected Ethernet-switch. The NIC Teaming failback feature poses a risk of black holing traffic for a few seconds, when a linkup is present on the LAN switching side; but the port does not go into the forwarding mode right away.

About the author


Leave a Comment