R77.30 cluster member might go Down after disabling CoreXL Dynamic Dispatcher only on one member

Support Center > Search Results > SecureKnowledge Details

Technical Level

Solution ID	sk108856
Technical Level
Product	ClusterXL, Cluster - 3rd party, CoreXL
Version	R77.30 (EOL)
OS	Gaia
Platform / Model	All
Date Created	10-Dec-2015
Last Modified	01-Jun-2017

Symptoms

After disabling CoreXL Dynamic Dispatcher (per sk105261) on a cluster member and rebooting this cluster member, output of "cphaprob state" command shows that its state is "Down".
Output of "cphaprob list" command shows that Critical Device "Synchronization" reports its state as "problem".
Full Sync never succeeds, and this member remains in "Down" state.
Rebooting the problematic cluster member, or manually forcing the Full Sync does not help.

Cause

In R77.20 and lower versions, traffic distribution between CoreXL FW instances is statically based on Source IP addresses, Destination IP addresses, and the IP 'Protocol' type.

CoreXL Dynamic Dispatcher was introduced in R77.30 - rather than statically assigning new connections to a CoreXL FW instance based on packet's IP addresses and IP protocol (static hash function), the new dynamic assignment mechanism is based on the utilization of CPU cores, on which the CoreXL FW instances are running.

The following is an explanation about packet flows when a connection is opened between two cluster members.

The first packet flow describes the flow when a connection is opened while CoreXL Dynamic Dispatcher is disabled on both cluster members (default).
The second packet flow describes the flow when Full Sync is initiated from a cluster member with disabled CoreXL Dynamic Dispatcher to a cluster member with enabled CoreXL Dynamic Dispatcher.

A local connection between cluster members is recorded on both cluster members.

The cluster member that opens the connection (Member_A) records the following in its Connections Table:

Member_A -> Member_B outbound
Symbolic Link Member_B -> Member_A inbound

The cluster member that receives the connection (Member_B) records the following in its Connections Table:

Member_A -> Member_B inbound
Symbolic Link Member_B -> Member_A outbound

When the connection is recorded on the same CoreXL FW instance on the same cluster members, the connection entry is synchronized to the peer cluster member. Thus, each cluster member now has 2 connection entries and 2 symbolic links.

When the connection is recorded on different CoreXL FW instances, after the Full Sync, the cluster member uses the latest CoreXL FW instance. Thus, the connection is dropped, because the cluster member that initiated the connection now uses the connection entry "Member_A -> Member_B inbound" and a "Symbolic Link Member_B -> Member_A outbound".

Solution

Note: To view this solution you need to Sign In .