Support Center > Search Results > SecureKnowledge Details
Default policy is loaded on Azure Gateways after a policy install or after a crash on the Gateway Technical Level
Symptoms
  • Policy installation on a Check Point Gateway in Azure causes the Gateway to crash and load a default policy.

  • Policy installation results in "Error code: 0-2000030" when the policy is pushed to Azure Gateways.

  • Connection destination to the Gateway may trigger a crash.

Cause

The issue is related to a firmware update of the MLX5 driver on the Azure host that enables Mellanox QoS support.

Check Point has identified a Mellanox driver issue that occurs when connections originate on or are destined for the Check Point Gateway in Azure.

If the crash occurs during a policy installation, a race condition may occur, resulting in the Check Point Gateway's applying a default policy upon reboot.


Solution

Table of Contents

  • Affected Environment
  • Unaffected Environment
  • Solution
  • Workaround to ensure stability
  • Removing the Workaround
  • Recovery of a Gateway that rebooted with default policy

Affected Environment

  • The issue impacts all Azure Gateway solutions (VMSS, HA, Single Gateway).

  • Affected Versions:

    Security Gateway Version Affected Image Versions

    R80.20

    All Security Gateway images

    R80.30

    Security Gateway Image version R80.30-273.753 and lower

    R80.40

    Security Gateway Image version R80.40-294.753 and lower

    R81

    Security Gateway Image version R81-392.751 and lower

Note: See sk132192 - CloudGuard for Azure Latest Updates for more information about released image versions.

Unaffected Environment

  • Management Servers and Multi-Domain Management Servers.

  • Security Gateway solutions deployed in other cloud platforms than Azure.

  • Security Gateways using versions R77.30, R80.10.

  • R80.30-based Security Gateways using image versions R80.30-273.801 and higher.

    • Note: Do not install R80.30 Jumbo HF lower than Take 235.
  • R80.40-based Security Gateways using image versions R80.40-294.801 and higher.

    • Note: Do not install R80.40 Jumbo HF lower than Take 114.
  • R81-based Security Gateways using image versions R81-392.807 and higher.

    • Note: Do not install R81 Jumbo HF lower than Take 25.

Solution

Install the applicable Hotfix on the Security Gateway:

Warning: If you configured a workaround cronjob as described below, you must remove it before you install the hotfix.

Important note: In case you installed a hotfix on top of a specific Jumbo Hotfix Take, do not install a new Jumbo hotfix Take lower take than the one mentioned to include the fix.

Security
Gateway
Version

Prerequisite on the Security Gateway

Hotfix Download Link

R81

Jumbo Hotfix Take 25

 Jumbo Hotfix Accumulator for R81 Take_25

Hotfix must be installed on top of Jumbo Hotfix Accumulator for R81 Take_17

R81 Hotfix to update the Mellanox/Nvidia NIC driver in Azure cloud due to issues seen in sk171553.

R80.40

Jumbo Hotfix Take 114

Jumbo Hotfix Accumulator for R80.40 Take_114

Take 94
Hotfix must be installed on top of Jumbo Hotfix Accumulator for R80.40 Take_94

R80.40 Hotfix to update the Mellanox/Nvidia NIC driver in Azure cloud due to issues seen in sk171553.

R80.30

Jumbo Hotfix Take 235

Jumbo Hotfix Accumulator for R80.30 Take_235

Hotfix must be installed on top of Jumbo Hotfix Accumulator for R80.30 Take_228

R80.30 Hotfix to update the Mellanox/Nvidia NIC driver in Azure cloud due to issues seen in sk171553.

R80.20

Jumbo Hotfix Take 39

Jumbo Hotfix Accumulator for R80.20 Take_39

Hotfix installation instructions:

Refer to sk168597 - How to install a Hotfix.

On VMSS environment the recommended method is to scale out to the latest image that includes the above hotfix.

  • R80.30-based Security Gateways using image versions R80.30-273.801 and higher.

  • R80.40-based Security Gateways using image versions R80.40-294.801 and higher.

  • R81-based Security Gateways using image versions R81-392.807 and higher.

    Note: If the workaround was added prior to the hotfix installation, it is recommended to remove the workaround.

    Validate the HF is installed by running this command:
    ethtool -i <Name of Mellanox Interface>
    Interface version should show:
           driver: mlx5_core
           version: 4.6-1.0.1hf1 (14 Jan 21)

    Recovery of a Security Gateway that rebooted with default policy:

    To regain access to the Security Gateway:

    1. Console into the Check Point VM using the Azure portal.

    2. Unload the default policy that is applied to the Security Gateways:

      [Expert@HostName:0]# fw unloadlocal

    3. Install the policy on the Azure Gateways.

    If policy installation continues to fail, follow these instructions:

    1. Clear the state directory on both the Security Gateway and the Management Server. See sk33328 - How to clear $FWDIR/state/ directory to resolve policy corruption issues.

      Note: If the Security Gateway and/or Management Server is running R80.40 version, you must create the following directories:

      • Management Server (for each managed Security Gateway / Cluster object):

        $FWDIR/state/R80.40GW/AMW

      • Security Gateway:

        $FWDIR/state/__tmp/AMW

    2. Clearing the state directory on the Security Gateway removes all defined Dynamic Objects.
      In a VMSS and HA environments, the LocalGatewayExternal and LocalGatewayInternal objects must be added.
      If these Dynamic objects are missing, it may result in a traffic failure in your environment.

      1. Confirm the Dynamic objects have been removed:

        [Expert@HostName:0]# dynamic_objects -l

      2. Add the required Dynamic objects again on the Security Gateway, if they were removed:

        [Expert@HostName:0]# dynamic_objects -n LocalGatewayExternal -r <IP Address of eth0> <IP Address of eth0> -a

        [Expert@HostName:0]# dynamic_objects -n LocalGatewayInternal -r <IP Address of eth1> <IP Address of eth1> -a

        Example:

        [Expert@HostName:0]# dynamic_objects -n LocalGatewayExternal -r 10.0.1.12 10.0.1.12 -a

        [Expert@HostName:0]# dynamic_objects -n LocalGatewayInternal -r 10.0.2.12 10.0.2.12 -a

      3. Verify the Dynamic objects were successfully added.

        [Expert@HostName:0]# dynamic_objects -l

        Example:

        [Expert@HostName:0]# dynamic_objects -l
        object name : LocalGatewayExternal
        range 0 : 10.0.1.11       10.0.1.11
        object name : LocalGatewayInternal
        range 0 : 10.0.2.12       10.0.2.12
        Operation completed successfully

    Note: If policy installation fails with the error about the local.magic file, then follow the applicable scenario from sk33893 - 'Installation failed. Reason: Load on Module failed - failed to load security policy' error during policy installation.





    This solution has been verified for the specific scenario, described by the combination of Product, Version and Symptoms. It may not work in other scenarios.

    Give us Feedback
    Please rate this document
    [1=Worst,5=Best]
    Comment