Support Center > Search Results > SecureKnowledge Details
Management Data Plane Separation Technical Level
Solution

Table of Contents:

  1. Introduction
  2. Minimum Requirements
  3. How it works
    1. Routing Separation
    2. Resource Separation
  4. Configuration
  5. Best Practice
  6. Limitations

(1) Introduction

Management Data Plane Separation allows a Security Gateway to have isolated Management and Data networks.

The network system of each domain (plane) is independent and includes interfaces, routes, sockets, and processes.

The Management Plane is a domain that accesses, provisions, and monitors the Security Gateway. This includes:

  • Access: SSH, FTP, and more
  • Provisioning: Policy installation, Gaia Portal, REST API, and more
  • Monitoring: Logs, SNMP, and more

Any Service, Process, or Port used by the above is considered a part of the Management Plane. Everything else is considered a part of the Data Plane.

(2) Minimum Requirements for Security Gateways

  • R80.40 or higher
  • R80.30SP Quantum Maestro - R80.30SP Jumbo Hotfix Accumulator, Take 73 or higher
  • R80.20SP Quantum Scalable Chassis - R80.20SP Jumbo Hotfix Accumulator, Take 210 or higher
  • A minimum of 4 CPU cores and 3 CoreXL FireWall instances

(3) How it works

The solution is implemented in software, and can be used on a physical or virtual appliance. It includes these capabilities, which can be used independently:

  • Routing Separation
  • Resource Separation

(a) Routing Separation

Routing Separation creates a routing domain (ID 1) that includes an interface that the Security Gateway uses to communicate with Management, and an interface used for the synchronization of cluster members (when using ClusterXL). This domain has its own routing table, in which routing decisions are made. It is not connected with the Data Plane through any virtual adapter. This means that any packet that enters the Security Gateway, on the Management plane or the Data plane, cannot go from one plane to the other. Do not configure non-Management operations on the Management plane network. Examples of non-Management operations: DNS, Proxy, DHCP, and Software Blade portals.

To switch the context to the Management Plane or the Data Plane in Gaia Expert mode (Bash) or see section (4) for clish, use these commands:

  • dplane - switches the network context to the Data plane (ID 0).
  • mplane - switches the network context to the Management plane (ID 1)
For commands in Gaia Clish and a list of Management plane tasks see the Configuration section below.

Tunnel Interface - Planes are isolated and traffic cannot cross between them. The Tunnel interface (mdps_tun) allows only packets that originated from the local gateway to be sent to selected destination(s) through the Management plane, regardless of the plane where the connection was initiated. 

The Tunnel interface is currently available in:

For commands in Gaia Clish and a list of Management plane tasks, see the Configuration section below.

(b) Resource Separation

When Resource Separation is configured, a dedicated CPU core is allocated for joint use by the Management NIC and a single CoreXL FireWall instance. Note - If Hyper-Threading is enabled, Resource Seperation uses two CoreXL FireWall instances.

The CoreXL Firewall instance does not receive any traffic from the CoreXL SND, except for packets that are inbound or outbound to the Management NIC. Because the number of connections is usually small, the Security Gateway usually remains accessible through the Management NIC regardless of how busy the other CoreXL Firewall instances and NICs are.

Consider a Security Gateway with 8 CPU cores and a 2-6 split (2 CPU cores work as CoreXL SND and 6 CPU cores work as CoreXL Firewall instances).

The distribution of CPU cores in a Security Gateway looks like this:

When Resource Separation is enabled, the distribution of CPU cores in a Security Gateway looks like this:

For more information about CoreXL, refer to sk98737 and Performance Tuning Administration Guide for your version.

Notes:

  • To enable the Routing and Resource Separation, at least 4 CPU cores and 3 CoreXL FireWall instances are required.

  • When using Routing and Resource Separation, the affinity for the Management Plane processes is not automatically set to a dedicated CPU core.

  • Because the Routing and Resource Separation uses a dedicated CPU core, fewer CPU core(s) are available for Data Plane inspection.

  • In Quantum Maestro Security Group, Routing and Resource Separation is supported when Security Group Members are connected with interfaces that use a mlx4/mlx5, or i40e drivers.

    One of the NIC queues is dedicated for Management traffic and the rest of the queue handles data traffic. If more than one CPU core is assigned to Management, then one CPU core handles traffic for the management queue and the others are free for user space applications.

(4) Configuration

Important Notes:

  • You must configure all settings in Gaia Clish.

  • On Quantum Scalable Platforms (Maestro and Chassis), you must configure all settings in Global Clish (gclish) of the Security Group.

  • Each plane has its configuration.

    Therefore, you must run these commands in each plane:

    • save configuration <Name of Script>

    • load configuration <Name of Script>

    This applies to:

Syntax in Gaia Clish / Gaia gClish:

{set | add | delete | show} mdps <options>

Configuration:

  • Enabling and disabling the routing separation:

    Syntax:

    clish> set mdps mgmt plane {on | off}

    Notes:

    • You must connect to the Security Gateway / Security Group through the serial console port because connectivity through the Gaia Management interface is lsot for a short time. You must reboot the Security Gateway / Security Group to complete the operation. Therefore, we recommend that you enable/disable the routing separation during a Maintenance window.

      • On R81.10SP, do these steps on the Single Management Object (SMO):
        1. Before you reboot your machines, run this command in the Expert mode:
          g_all touch /tmp/unsync_xfer_files
        2. Reboot your machines. 
        3. After you reboot all machines, run this command in the Expert mode:
          g_all rm /tmp/unsync_xfer_files
    • When you enable the MDPS, all static routes through the current Gaia Management interface become obsolete, because this interface moves to the Management plane (mplane).

      You must add all the required static routes through Gaia Management interface again in the Management plane.

      Workflow:

      1. Get all the current static routes:

        show configuration static-route

      2. Copy the commands you see in the output that use the Gaia Management interface.

      3. Enable the MDPS.

      4. Go to the Management plane (mplane).

      5. Run the commands you copied earlier.

    • On Quantum Scalable Platforms (Maestro and Chassis*), follow these steps after enabling or disabling of MDPS:
      1. Reboot the Standby Site / Standby Chassis
      2. Fail-over to the Standby Site / Standby Chassis
      3. Reboot the Standby Site / Standby Chassis
        * When you enable routing separation on R81.10 Chassis only (not Maestro), first make sure to move the Multi-Queue of the management interface to the management plane, as described in the second note above.
  • Changing the Gaia Clish context:

    clish> set mdps environment {mplane | dplane}

  • Enabling and disabling the resource separation:

    clish> set mdps mgmt resource {on | off}

    Important Notes:

    • You must connect to the Security Gateway / Security Group through the serial console port because connectivity through the Gaia Management interface is lost for a short time. You must reboot the Security Gateway / Security Group to complete the operation. Therefore, we recommend that you enable/disable the resource separation during a Maintenance window.

      • On R81.10SP, do these steps on the Single Management Object (SMO):
        • Before you reboot your machines, run this command in the Expert mode: 
          g_all touch /tmp/unsync_xfer_files
        • Reboot your machines.
        • After you reboot all machines, run this command in the Expert mode: 
          g_all rm /tmp/unsync_xfer_files
    • First, you must configure the applicable network interface, then enable the MDPS.

  • Controlling the number of CPU cores for the resource separation:

    clish> set mdps resource cpus <1-4>

  • Configuring the 'Management' interface:

    When you use Routing or Resource Separation, you must configure the Management interface.

    The interface is used for communication with the Check Point Management Server and to access the Security Gateway.

    Syntax:

    clish> set mdps interface <NAME> management {on | off}

  • Configuring the 'Sync' interface:

    Important - This is not supported on Quantum Scalable Platforms (Maestro and Chassis).

    When you use Routing Separation and ClusterXL, you must configure the Sync interface on the Management Plane.

    The interface is used for ClusterXL synchronization between the cluster members.

    Syntax:

    clish> set mdps interface <NAME> sync {on | off}

  • Adding or deleting routes in the Management plane (in R80.40 and lower versions):

    clish> {add | delete} mdps route <IP ADDRESS> nexthop <IP ADDRESS>

    Important Notes:

    • Starting in R81, this command is deprecated.

      You must switch to the relevant MDPS plane and use the applicable Gaia Clish commands to set, modify, or show the routes.

    • On Quantum Scalable Platforms (Maestro and Chassis) R80.30SP and R80.20SP, you must run the command below in the Expert mode on the Security Group (pay attention to single and double quotes):

      gexec -b all --vs 1 -c "clish -c '<Gaia Clish Routing Command>'"

  • Adding or deleting tasks from the Management plane:

    With this option, it is possible to choose where to place tasks that the Security Gateway runs - in the Management plane or the Data plane.

    When enabling the Routing separation, a set of default tasks bounds to the Management plane. All other tasks remain in the Data plane. A task cannot be bound to both planes.

    Syntax:

    clish> {add | delete} mdps task <options>

    A "Task" can be one of these:

    • Port, Protocol:

      A specific port and protocol from any process is bound to the Management plane. The process itself remains in the Data plane context. This option works only for Check Point known ports.

    • Process:

      A specific process name is bound to the Management plane, including all ports that the process opens.

    • Service:

      OS service may include several processes, all of which are bound to the Management plane.

    • Address:

      Network or hostname to use with the tunnel interface, Any outgoing packets to the selected addresses are sent according to Management plane routing table.

    Notes:

    • Added and deleted tasks are applied during the next restart of the task.

  • List of default tasks when Routing Separation is used:

    For more information about Check Point processes and ports, refer to sk97638 and sk52421.

    Enter the string to filter this table:

    Type Name, URL, Port Number
    Address updates.checkpoint.com
    Address te.checkpoint.com
    Address teadv.checkpoint.com
    Address cws.checkpoint.com
    Address usercenter.checkpoint.com
    Address avupdates.checkpoint.com
    Service cpri_d
    Service sshd
    Service syslog
    Process AutoUpdater
    Process DAService
    Process cloningd
    Process confd
    Process httpd2
    Process ntpd
    Process rest_api_docs
    Process rest_api_run
    Process snmpd
    Process snmpmonitor
    Process start_celery
    Process start_redis
    Process cprid
    Process lldpd
    Port - Protocol 256 - tcp
    Port - Protocol 257 - tcp
    Port - Protocol 263 - tcp
    Port - Protocol 2010 - tcp
    Port - Protocol 5432 - tcp
    Port - Protocol 8989 - tcp
    Port - Protocol 18181 - tcp
    Port - Protocol 18183 - tcp
    Port - Protocol 18184 - tcp
    Port - Protocol 18187 - tcp
    Port - Protocol 18191 - tcp
    Port - Protocol 18192 - tcp
    Port - Protocol 18195 - tcp
    Port - Protocol 18210 - tcp
    Port - Protocol 18211 - tcp
    Port - Protocol 18264 - tcp

(5) Best Practices

When using Routing Separation

  • Do not include Management Plane subnets on any Software blade portal.

  • Connect the Security Gateway to LDAP and other user authentication servers through the Data plane (Applies for versions without Tunnel Interface).

  • Run commands in Expert mode or Gaia Clish in the context of the Data Plane (ID 0) except commands which are network dependent, such as 'ip ...', 'ifconfig', 'netstat', etc.

  • SNMP queries:

    • When using SNMP v2/v2c:

      To query the Security Gateway's Data plane OIDs, add the name suffix "_dplane" to the SNMP community name. For example, if the community name is "test," then the Data plane community name is "test_dplane".

    • When using SNMP v3:

      To query the Security Gateway's Data plane OIDs, you must add the context argument. The context name is "dplane".

  • When using a local license, the license must be issued for the IP address of the Management interface on the Security Gateway.

When using Management Resource

  • Do not configure the CPU affinity for the Management interface.

(6) Known Limitations

Issue ID Description
PMTR-25365 IPv6 is not supported on the Management interface when using Resource Separation.
PMTR-25369 Configuration of Routing or Resource Separation can be done only in Gaia Clish.
PMTR-29698 The use of logical interfaces is not supported on the Management interface (Alias, Bridge, VPN Tunnel, 6in4 Tunnel, PPPoE, Bond, VLAN).

Note - This limitation was resolved in the latest Jumbo Hotfix Accumulator Takes for R80.40 and R81. It is supported in R81.10.
 -

"Authentication failure: check your username and password" message on a Security Gateway when raising the "TACP" privileges of a TACACS user in the following scenario:

  1. Configured the Management Data Plane Separation (MDPS) as described in this article.
  2. Configured Gaia OS roles with different privileges for TACACS users.
  3. Configured a TACACS server.
  4. Logged in with a TACACS user.
  5. Raised the "TACP" privileges in the Gaia Portal (at the top of the "Overview" page, clicked "Enable") or in Gaia Clish (with the command "tacacs_enable <Role>")
  6. Entered the TACACS user password.

To resolve:

  1. Connect to the command line on the Security Gateway.
  2. Log in to Gaia Clish.
  3. Add the Gaia OS "confd" process to the Management Plane. Run: add mdps task process confd
  4. Save changes. Run: save config
 - When using ClusterXL and Routing separation, configuring the same subnet for the Management and Data interface is not supported.
 - In SmartConsole, in the Security Gateway object > Topology, the "Get interfaces with topology" operation fetches only the data plane interfaces.
- Connections from the Security Gateway to Check Point domains "checkpoint.com" might fail when MDPS is enabled.
Follow sk180121.
 - When using Routing Separation, you must collect a Backup/Snapshot on a remote host only through the Management plane.
TM-13547 When adding the loopback interface on SNMP Agent Interfaces in MDPS, this error message appears: "The snmpd not listening - No Response".
To resolve the issue, remove the "lo" interface or add as "Any".
PMTR-66296 The Gaia OS "LLDP" feature is not supported when the MDPS is enabled (applies only to R81.00).
PMTR-81746 When MDPS is enabled, Gaia Portal is not supported on a Security Group.

Limitations on Quantum Scalable Platforms (Maestro and Chassis):

  • Quantum Scalable Platforms R81 (Maestro and Chassis) does not support MDPS (MBS-14161).
    Note: On Quantum Scalable Platforms R81.10 (Maestro and Chassis) MDPS is available as a hotfix for Take 45 of the Jumbo Hotfix Accumulator.

  • Starting from R81.10, you must use this command in the Expert mode of the Security Group to configure settings that are related to the Management interface - including the adding and removing of static routes (pay attention to single and double quotes):

    gexec -b all --vs 1 -c "clish -c '<Gaia Clish Command>'"

  • You must disable CoreXL Dynamic Balancing (PMTR-73771).

  • Sync & Chassis Internal Network (CIN) interfaces are not considered part of the Management Plane.

  • All members on a Security Group must be UP and ACTIVE when enabling or disabling the MDPS.

  • Configuration of Routing Separation can be done only in Gaia Global Clish (gclish).

  • Only Management interfaces and MAGG can be used as a Management interface in the MDPS.

  • When configuring a Syslog server, it must be in the Management interface's subnet.

  • In SmartConsole, in the SMO Security Gateway object, the operation "Get Interfaces with Topology" is not supported. Therefore, any topology modifications must be done manually in the SMO Security Gateway object.

  • Before performing an upgrade or installing/uninstalling a Jumbo Hotfix Accumulator, make sure to disable the MDPS feature and enable it again after the Security Group Members reboot.

  • Gaia Portal (WebUI) is not supported when MDPS is enabled.

Give us Feedback
Please rate this document
[1=Worst,5=Best]
Comment