Dynamic Balancing (Dynamic Split) is a performance-enhancing daemon that balances the load between CoreXL SNDs and CoreXL Firewalls. It dynamically changes the split between CoreXL SNDs and CoreXL Firewalls and does not require a reboot or cause an outage.
Each Check Point Security Gateway's CPU belongs to one of two groups, each of which performs a different task:
Firewall Instance
SND (with the exception of a single CPU running FWD in large User-Space appliances).
The distribution of jobs across a Security Gateway's CPUs is referred to as the Security Gateway's split. As the distribution of work across these groups depends on your security policy and traffic, we highly recommend that you configure your split to fit your specific needs.
CoreXL's Out-of-the-Box Dynamic Balancing performs a dynamic change of the split. It monitors your system and makes changes as needed.
Starting in R81, this feature is on by default.
Note: If you manually changed CoreXL or Multi-Queue affinity settings on the old version, the feature is not on by default on the upgraded version.
Key Features
Out-of-the-box optimization
A flexible split to suit your profile
System requirements
Supported Platforms: Dynamic Balancing is supported on Check Point Appliances.
Supported Models: Dynamic Balancing is supported on these models:
All models in these series: 7000, 15000, 16000, 23000, 26000 and 28000
5000 series: 5600, 5800 and 5900
6000 series: 6200T, 6400, 6500, 6600, 6700, 6800 and 6900
3000 series: 3100, 3200, 3600 and 3800
Notes:
On models with fewer than eight cores, you must enable a GNAT port allocation feature. Refer to sk165153. If Dynamic Balancing was enabled before or is on by default, post-enabling GNAT and the required reboot, Dynamic Balancing automatically starts running.
R80.40 is the last supported version for 2200 / 4000 / 12000 / 13000 / 21000 appliances.
Supported Versions: Check Point R80.40 with Jumbo Hotfix Take 25 and higher
Supported Configurations: Security Gateway (Kernel or USFW), Stand-Alone, VSX, and Maestro.
Maestro support is for R81.20 and higher versions. Dynamic Balancing is on by default in Maestro R81.20 and higher versions (including Maestro VSX and Maestro mixed appliances).
VSX support is for:
R80.40 Jumbo Hotfix Take 126
R81 with Jumbo Hotfix Take 58
R81.10 and higher versions
Note: Starting in R81.20, VSX support is on by default.
Supported features: IPv6, Management Data Plane Separation (MDPS), Bridge mode
1. In Expert mode, run dynamic_balancing -o enable
In Maestro Expert mode, run g_dynamic_balancing -o enable
In Clish/gClish, run set dynamic-balancing state enable
2. You can receive a prompt if one of these requirements is not met.
a. On non-VSX environments, if your current CoreXL split is not the default, you are asked to change the number of instances (not applicable to Maestro mixed appliances).
In Expert mode, Enter y
In Clish (non-interactive), run the command set dynamic-balancing state enable set_default_fw_instances
b. On VSX environments, if your current FWK affinity is not the same on all VSs, you are asked to set it to default. Refer to the Performance Tuning R81 Administration Guide and refer to this section: CoreXL > CoreXL Commands > fw ctl affinity > Running the fw ctl affinity -s command in VSX Mode.
3. Reboot your appliance. In Maestro, reboot each Security Gateway Member (SGM) gradually. Important note for cluster environments: You must configure all cluster members in the same way. In High Availability mode, start with the Standby member. For more information, refer to the ClusterXL R81.10 Administration Guide (section: ClusterXL Requirements and Compatibility > Software Requirements for Cluster Members).
Syntax
Expert: dynamic_balancing -o { enable | disable | stop | start } In Maestro Expert mode: use g_dynamic_balancing to run the command on all SGMs. Clish/gClish: set dynamic-balancing state { enable | disable | stop | start } Note: You can use start/stop commands on specific SGMs with this gClish command: set dynamic-balancing state {start | stop} member_ids <members IDs>
Action
Description
enable
Starts Dynamic Balancing from the Security Gateway’s default split.
disable
Returns the Security Gateway to its default configuration.
stop
Stops Dynamic Balancing from making split changes. Survives reboot. The CoreXL split is restored after the reboot to the same configuration when the command was executed.
start
Starts Dynamic Balancing after it is stopped by a user.
Verify the Dynamic Split Status in CPView under the SysInfo tab:
Monitor Dynamic Balancing through CPView under the CPU tab:
Check the Dynamic Balancing status:
Expert: dynamic_balancing –p Maestro Expert mode: g_dynamic_balancing -p Clish/gClish: show dynamic-balancing state Output: Dynamic Balancing is currently On/Off
This feature introduces improvements in Elephant Flow resource utilization (refer to sk164215 - How to Detect and Handle Heavy Connections). Dynamic Balancing detects Elephant Flows and dedicates available resources to the Firewall instance that handles the connection. This feature is on by default.
How it works:
Upon detecting an Elephant Flow, Dynamic Balancing stops the Firewall instance that handles the connection, and its logical sibling instance (on SMT HyperThreading environments).
New connections are not dispatched to either one of the Firewall instances,
The Elephant Flow gradually benefits from the free CPU processing power of the entire physical core.
Upon the completion of the Elephant Flow or high overall system utilization, Dynamic Balancing removes the Super Instance by starting the two instances back.
The CPU tab in CPView shows the logical sibling core as CoreXL_FW_RESERVED.
In R81.20 and higher, alongside the regular CoreXL split decisions, Dynamic Balancing is responsible for HyperFlow cores activation (refer to sk178070 - HyperFlow in R81.20 and higher).
How it works:
When it detects an Elephant Flow, Dynamic Balancing allocates cores for the HyperFlow’s Parallel Processing Engine (PPE) threads at the expense of Firewall or SND cores (whichever yields a more balanced state).
Dynamic Balancing decides to allocate more PPE threads depending on the Elephant Flow throughput.
When the Elephant Flow terminates, or high overall system utilization occurs, Dynamic Balancing deactivates the PPE threads and returns the removed Firewalls/SNDs to their initial state.
Dynamic Balancing prevents an activation that interferes with preserving the balanced state of the system. It always prioritizes total throughput (Firewalls and SNDs) over a single connection (PPEs)
The Super Instance feature works seamlessly on top of HyperFlow, and provides an additional boost to the Elephant Flow
Dynamic Balancing has various configuration parameters which can be changed during run-time. The ability to change them during run time was introduced in these versions:
R80.40 Jumbo Hotfix Take 180
R81 Jumbo Hotfix Take 77
R81.10 Jumbo Hotfix Take 79
Clish: set dynamic-balancing config <config> <value> Expert: dynamic_balancing –v <config> <value> Note: Use “default” as the <value> to change <config> to default
Name
Purpose
Unit
Default
Min/Max
ALPHA
Balances decision sensitivity, the minimal difference between Firewall CPUs and SNDs CPUs to perform a change.
N/A
10
0/100
EMERGENCY_CPU_HANDLING_THRESHOLD
When the FWs average utilization is above this, SNDs must work harder, by a factor, for a change to take place.
%
50
0/100
EMERGENCY_CPU_HANDLING_FACTOR
The factor by which the SNDs must work harder in the above parameter. For example, 70 translates to a factor of 1.7
N/A
70
0/100
VERBOSE_DEBUG
Enable verbose debugs
BOOLEAN
0
0/1
VERIFY_INTERVAL
State verification interval occurs one time every 300 main loop cycles, which usually translates to every 5 minutes. During this verification, Dynamic Balancing makes sure its internal state is in sync with the system’s state. Use 0 to disable state verification checks
N/A
300
0/1000
RECOVERY_ATTEMPTS
Limit the number of attempts to recover after a state verification failure, disabled by default.
N/A
0
0/100
SUPER_INST_ENABLED
Enable Super Instance feature
BOOLEAN
1
0/1
SUPER_INST_ADD_FW_LOAD
Super instances can be added if FWs average utilization is below this.
%
60
0/100
SUPER_INST_REMOVE_FW_LOAD
Super instances are removed if the FWs average utilization is above this.
%
70
0/100
DMD_WAKEUP_FW_LOAD
HyperFlow PPE threads "wake up" if FWs average utilization is below this, and an Elephant Flow was detected.
%
60
0/100
DMD_SLEEP_FW_LOAD
HyperFlow PPE threads "go to sleep" if FWs average utilization is above this.
Average utilization of SNDs vs FireWall is relatively close (we require 10% difference by default)
FireWalls are utilized more than 50% (EMERGENCY_CPU_HANDLING_THRESHOLD), and SNDs are not working harder, by a factor of 1.7 (EMERGENCY_CPU_HANDLING_FACTOR). This is done to prevent a situation where more than one heavily utilized FireWall instance works on a single CPU.
In Cluster HA mode, changes the active member makes are synced to the passive member. This does not apply to Virtual Routing Redundancy Protocol (VRRP).
No, as at any given time, both members will have same number of Firewall instances. Dynamic Workloads will merely “stop” the instance, meaning new connection are not to be dispatched to it.
Dynamic Balancing in VSX is similar to Security Gateways in that it aims to balance the SNDs and FWs cores. As opposed to Security Gateways, FW instances do not have static core affinity, and their amount does not determine the amount of SNDs. As a result, Dynamic Balancing does not require a certain amount of FW instances to be configured, and only adjusts their core affinity.
When adding an SND, the feature sets the affinity of FWK processes in all VSs to the list of new cores (rather than move a FW instance from one core to a different core, as done in Security Gateways). The maximum quantity of SND cores will be according to the NIC driver, with the highest number of queues in all VSs.
When you add a VS, the feature sets the new VS’s FWK to the current FWKs cores affinity.
Dynamic Balancing requires several configuration changes that can only be set upon boot, to allow best performance in all splits (mostly relevant when more SNDs are needed vs default split).
Dynamic Balancing monitors the system periodically for manual changes that might conflict with its own actions (such as: Firewall instances state or affinity changes, interfaces affinity changes etc.), and will stop itself if such action is detected. Refer to sk163815 for more information.
Dynamic Balancing has mechanisms within its logic aimed to prevent un-wanted changes (uses threshold to not be over sensitive, detects frequent conflicting changes and more). Essentially, it uses existing, established Firewall/Linux commands, that were used in Check Point devices for years, while automating their use to be as effective as possible.
Stopping the Dynamic Balancing does not require a reboot. Disabling Dynamic Balancing requires a reboot, since part of the disable process is to revert the configuration back to default, which includes several settings that can only be set on boot (mainly due to memory allocations performed at system init).
No, but changes made in the CoreXL cpconfig only take effect after reboot. Note that on non-VSX environments, rebooting with a non-default instance number (that is, manual changes done by the user) prevents Dynamic Balancing from starting in order not to overwrite the users' actions.
It stops the Firewall instance(s) of the physical CPU that is going to become an SND. This means that new connections are no longer dispatched to this instance, but it will continue to handle its existing connections.
The "fw ctl multik stat" command shows these Firewall instances as inactive:
It sets the affinity of the stopped Firewall instance(s) to the affinity of the active Firewall instances belonging to the same NUMA node.
The "fw ctl affinity -l -r" command shows these Firewall instances on all (active) Firewall instances' CPUs:
Note that CPU 20 has different inactive Firewall instances on it because it is on a different NUMA node than CPUs 7-17.
Although the Dynamic Dispatcher excludes the stopped instance from its dispatching calculations, in cases where a VPN tunnel was previously created on this instance, some connections, such as NAT-T or ones belong to a non-accelerated tunnel, may be opened on this instance to allow a certain performance optimization.
There are two primary types of CPUs: CoreXL_FW and CoreXL_SND. "BOTH" means the CPU currently operates as a FW and anSND. "OTHER" means the CPU does not currently operate as a FW or SND. In most scenarios, this is an expected, temporary state, caused by CPU role change. If this is not a temporary state, it could imply an affinity misconfiguration.
Notes
Dynamic Balancing manages network card ports that have Multi-Queue enabled. The "mq_mng --show" command shows such ports as "Dynamic". While Dynamic Balancing is active, it assumes control over several resources (listed below). Manual changes may not work, or cause Dynamic Balancing to stop its work (refer to sk163815 for more details):
Changes in affinity of CoreXL Firewall instances, starting or stopping CoreXL Firewall instances, and changing the number of CoreXL Firewall instances.
Changes in Multi-Queue affinity/mode, or changes in the number of RxTx queue weights.
Known Limitations
Issue ID
Description
Comments
PRJ-15874
When you downgrade to Jumbo Hotfix Take where the Dynamic Balancing is not supported, it remains enabled. In this case, the affinity of the Security Gateway will be configured incorrectly.
Disable the Dynamic Balancing before you uninstall the Jumbo Hotfix
Give us Feedback
Thanks for your feedback!
Are you sure you want to rate this stars?
