Traffic passing through the VSX cluster is lost during a cluster failure on Standby member
When a cable is disconnected from a cluster member / switch port is shut down, that member stops receiving CCP packets from the peer members. By design, at this point, the "disconnected" member starts the probing process - send a series of ARP Requests to all IP addresses in the network connected to the problematic interface (for more details about the probing, refer to sk93306 - ATRG: ClusterXL - Chapter 'Cluster Control Protocol (CCP)').
In VSX cluster, the Standby member sends the probing ARP Requests packets with the following Layer 2 and Layer 3 addresses:
- Destination MAC address = FF:FF:FF:FF:FF:FF
- Source MAC address = 00:00:00:00:fwha_mac_magic:ID_of_Source_Member
- Source IP address = Cluster VIP
- Destination IP address = Broadcast IP address for this subnet
As a result, the surrounding switches / routers / hosts might update their ARP tables to associate the Cluster VIP (Layer 3 address) with MAC address of Standby member (Layer 2 address). This will cause the traffic to flow towards the Standby member, which by design does not process any traffic.
This outage continues until the Gratuitous ARP Requests packets arrive from the Active member with its Source MAC address (by design, these GARP packets are sent periodically), thus restoring the correct association of Cluster VIP with MAC address of Active member.