Support Center > Search Results > SecureKnowledge Details
Communication problems with ClusterXL clusters connected to Cisco ACI Technical Level
Symptoms
  • Security Group Management interfaces (Example: eth1-Mgmt4, magg0) are intermittently inaccessible. Policy installation most likely fails during this time.
  • Communication issues with ClusterXL members.
    Traffic captures indicate the packets are forwarded to the wrong member.
    For example - when trying to access the Standby member, packets are forwarded to the Active member.
  • Cluster status monitoring commands like 'cphaprob stat' do not show any problematic status
  • Active traffic may be forwarded to the wrong cluster member
  • Cisco ACI Switch logs (Cisco End Point logs) show endless "attached / detached" state transitions regarding the Management IP for each SGM Mgmt interface MAC.
  • Cisco ACI switch environment was recently upgraded to version 4.2(4i) or newer.
Cause
Cisco ACI has several proprietary features which cause problems with Check Point clusters.
These features are "Endpoint Dataplane Learning", "COOP Endpoint Dampening", and "Rogue Endpoint Detection". 
Before explaining these features, we will review some key points about Check Point clusters.

Check Point Clustering

Most switches learn information about hosts connected to the network by listening to ARP requests and replies. Check Point clustering relies on this behavior to ensure traffic is always sent to the Active cluster member by using GARP.

Key points about ClusterXL Clusters:

  • All clustered interfaces have a Virtual IP (VIP) which is shared by all cluster members.
  • Any cluster member may transmit information on a clustered interface using the shared VIP and their own unique MAC address.
  • Traffic destined for the shared VIP must be forwarded to the Active cluster member.

Key points about Chassis and Maestro Security Groups:

  • The IP configured on the Management port is shared between all Security Group members.
  • Only the SMO replies to ARP requests received on the Management port.
    All other SGMs drop the ARP request.
  • Any SGM in the Security Group may transmit information using the Management port, the shared Management IP, and their own unique MAC address.
  • Traffic received on the Management port must have the destination MAC address of the current SMO.

Cisco Endpoint Learning Features

Cisco ACI does not behave like most switches as explained above. 
The switches still listen to ARP requests and replies to learn about the network.
With "Dataplane Endpoint Learning", the Cisco switches also learn about the network from the source IP and source MAC information in regular network traffic.

Because any Cluster member or SGM can transmit traffic using their own MAC and a shared IP address, the Cisco ACI switch thinks that the shared IP is constantly "moving" to different MACs.
If the IP-MAC association changes too frequently, the IP is considered "misbehaving" or "rogue".

Once the shared IP address is considered "misbehaving" or "rogue", Cisco features like "Rogue Endpoint Detection" or "COOP Endpoint Dampening" disable updates and freeze the current IP-MAC association. If the frozen MAC does not belong to the currently active Cluster member or SMO SGM, then there will be traffic issues.

Solution
Note: To view this solution you need to Sign In .