Support Center > Search Results > SecureKnowledge Details
Log Exporter - Check Point Log Export Technical Level
Solution

Check Point "Log Exporter" is an easy and secure method for exporting Check Point logs over the syslog protocol.

Exporting can be done in few standard protocols and formats.

Log Exporter supports:

  • SIEM applications: Splunk, LogRhythm, Arcsight, RSA, QRadar, McAfee, rsyslog, ng-syslog, and any other SIEM application that can run a Syslog agent.
  • Protocols: Syslog over TCP, Syslog over UDP.
  • Formats: Syslog, Splunk, CEF, LEEF, Generic, JSON, LogRhythm, RSA.
  • Security: Mutual authentication TLS 1.2.
  • Log Types: The ability to export Security logs, Audit logs, or both.
    Note: Audit logs exist on both the Management Server and the Log Server.
  • Filtering: Choose what to export based on field values. Filter out (do not export) Security Gateway connection logs.
  • Links to Logs and Log Attachments: Export links to the relevant log card in SmartView and to the log attachments.

The table below contains the release information for the features:

Feature / Capability Description R80.20 R80.30 R80.40 R81 R81.10
Filtering Choose what to export based on field values Requires
R80.20
Jumbo
Hotfix
Accumulator
Take 103
and higher
Requires
R80.30
Jumbo
Hotfix
Accumulator
Take 107
and higher
Integrated
in the
release
Integrated
in the
release
Integrated
in the
release
Links to Logs and Log Attachments Export links to the relevant log card in SmartView and to the log attachments (Forensics / Threat Emulation reports) Requires
R80.20
Jumbo
Hotfix
Accumulator
Take 127
and higher
Requires
R80.30
Jumbo
Hotfix
Accumulator
Take 107
and higher
Integrated
in the
release
Integrated
in the
release
Integrated
in the
release
API for Attachment IDs Export identifiers of attachments for fetching them via Log API Requires
R80.20
Jumbo
Hotfix
Accumulator
Take 183
and higher
Requires
R80.30
Jumbo
Hotfix
Accumulator
Take 217
and higher
Requires
R80.40
Jumbo
Hotfix
Accumulator
Take 78
and higher
Integrated
in the
release
Integrated
in the
release
DNS Name Usage Configure DNS name (FQDN) as the target-server in addition to IP address Requires
R80.20
Jumbo
Hotfix
Accumulator
Take 190
and higher
Requires
R80.30
Jumbo
Hotfix
Accumulator
Take 228
and higher
Requires
R80.40
Jumbo
Hotfix
Accumulator
Take 92
and higher
Requires
R81
Jumbo
Hotfix
Accumulator
Take 13
and higher
Integrated
in the
release
Reconnection to Load Balancer Initiate reconnection to load balancer every X minutes (configurable) Requires
R80.20
Jumbo
Hotfix
Accumulator
Take 190
and higher
Requires
R80.30
Jumbo
Hotfix
Accumulator
Take 228
and higher
Requires
R80.40
Jumbo
Hotfix
Accumulator
Take 92
and higher
Requires
R81
Jumbo
Hotfix
Accumulator
Take 13
and higher
Integrated
in the
release

Table of Contents

  • How does it Work
  • Installation
  • Uninstall (R77.30 only)
  • Basic Deployment
  • Advanced Deployment - Additional Commands
  • Advanced Configuration After the Deployment
  • Format Configuration
  • Fields Configuration
  • TLS Configuration
  • Filter Configuration
  • Log Fields Mapping for Advanced Fields Configuration
  • SIEM Specific instructions
  • Transition from LEA to Log Exporter
  • Transition from CPLogToSyslog to Log Exporter
  • Troubleshooting
  • Known Limitations
  • Appendix
  • Change Log
  • Related Solutions
  • Revision History

Click Here to Show Entire Article

 

How Does It Work

Show / Hide the section

Log Exporter is a multi-threaded daemon service, running on a Management Server / Log Server. The Log Exporter daemon reads each log, transforms it into the desired format and mapping, and sends it to the configured target. Therefore, we recommend to deploy the Log Exporter on every server that contains logs to be exported.

On a Multi-Domain Security Management Server / Multi-Domain Log Server, if the Log Exporter is deployed on several Domains, each Domain Management Server has its Log Exporter daemon . If you are exporting the logs to several targets, each target have its Log Exporter daemon.

The Log Exporter is implemented as the "ETL" procedure:

  • Extract - Reads incoming logs from the Security Gateway, stored in local files.

  • Transform - Changes the logs according to configuration files (both exported format and field name/values, removing irrelevant fields).

  • Load - Sends the logs to the configured target server over the TCP Syslog / UDP Syslog (takes into consideration the filter configuration, if it exists).

  • Data integrity - Log Exporter stops exporting when disconnected from the 3rd party server and remembers the last position exported. After the connection is established again, the Log Exporter automatically starts exporting logs from the last known position.

The Log Exporter is exporting both online and offline (if any) logs in parallel. In case the 3rd party server is slow, the Log Exporter reduces the offline exporting rate to prioritize the online logs over the offline logs.

 

Installation

Show / Hide the section
Version Description

R80.20 and higher

Log Exporter is Integrated
in the
release

R80.10

Log Exporter is integrated in R80.10 Jumbo Hotfix Accumulator from Take 270 (see PRJ-3792)

R77.30

Install this Log Exporter package on an R77.30 server - Multi-Domain Server, Multi-Domain Log Server, Security Management Server, Log Server, or SmartEvent Server.

  • Date - 14 April 2019
  • CPUSE Online Identifier - Check_Point_R77.30_Log_Exporter_T36_sk122323_FULL.tgz
  • CPUSE Offline Package - (TGZ)

Install the hotfix using CPUSE. See sk92449.

Important Notes:

  • This Log Exporter package can be installed on top of the R77.30 Jumbo Hotfix Accumulator Take 292 and above.
  • To install a higher Take of the R77.30 after you install the Log Exporter package:
    1. Uninstall the Log Exporter package
    2. Install a higher Take of the R77.30 Jumbo Hotfix Accumulator
    3. Install the Log Exporter package again

 

Uninstall (on R77.30 only)

Show / Hide the section
Server Instructions

R77.30 Security Management Server

Uninstall the Log Exporter package using CPUSE. See sk92449.

R77.30 Multi-Domain Server

Uninstalling the Log Exporter package does not remove its configuration files.

To uninstall and completely remove the Log Exporter configuration:

  1. Connect to the command line on the Multi-Domain Security Management Server / Multi-Domain Log Server.

  2. Log in to the Expert mode.

  3. Run:

    cp_log_export delete name all domain-server all --apply-now

  4. On a Multi-Domain Security Management Server:

    1. Go to the MDS context:

      mdsenv

    2. Make sure the environment variable $EXPORTERDIR exists, and its value is $FWDIR/log_exporter:

      echo $EXPORTERDIR

    3. Delete the EXPORTERDIR directory and its contents:

      rm -rf $EXPORTERDIR

  5. Uninstall the Log Exporter package using CPUSE. See sk92449.

  6. Reboot the Multi-Domain Server.

 

Basic Deployment

Common method for creating / modifying log exporters / targets.

Show / Hide the section

These are the ways to configure the Log Exporter:

Configuration Description

In SmartConsole

(R81 and higher)

See the Logging and Monitoring Administration Guide for your version (R81 and higher).

On CLI

(R77.30 and higher)

Syntax:

cp_log_export add name <Name> [domain-server <Name or IP address of Domain Server>] target-server <HostName or IP address of Target Server> target-port <Port on Target Server> protocol {udp | tcp} format {syslog | splunk | cef | leef | generic | json | logrhythm | rsa} [<Optional Arguments>]

Important Notes:

  • The "domain-server" argument is mandatory on a Multi-Domain Security Management Server / Multi-Domain Log Server.

    • Use 'mds' as the value for the "domain-server" argument to export audit logs from the MDS level.
    • Use 'all' as the value for the "domain-server" argument to configure the Log Exporter instance on every Domain.
  • The "target-server" argument can use either the target server IP address or its FQDN.

  • The above command creates a new target directory with the unique name specified in the "name" parameter in the $EXPORTERDIR/targets/ directory, and configures the target parameters with the connection details: IP Address, port, protocol, format, and read-mode.

  • By default, logs are exported in clear text. To export logs using an encryption, see the section "Advanced Deployment - Additional Commands".

  • The Log Exporter daemon does not start automatically.

    To start it, run:

    cp_log_export restart

 

Advanced Deployment - Additional Commands

Advanced parameters for creating / modifying log exporters / targets

Show / Hide the section

Syntax

cp_log_export <Command Name> [<Command Arguments>]

To see the built-in help, run:

cp_log_export <Command Name> help

Commands

Name Description
add Deploys a new Log Exporter instance.
set Updates an existing Log Exporter instance configuration.
delete Removes an existing Log Exporter instance.
show Prints the current configurations of the existing Log Exporter instances.
status Prints the overview statuses of the existing Log Exporter instances.
start Starts the Log Exporter instance.
stop Stops the Log Exporter instance.
restart Restarts the Log Exporter instance.
reexport Resets the current read position and re-exports all logs per the Log Exporter instance configuration.

Parameters

Parameter Name Description add set delete show,
status,
start,
stop,
restart
reexport
name Configure a unique name for the Log Exporter configuration. Mandatory Mandatory Mandatory Optional
Default is "all"
Mandatory
domain-server Configure the name or IP address of the relevant Domain. Mandatory Mandatory Mandatory Optional
Default is "all"
Mandatory
target-server Configure the IP address of the target server (to which the Log Exporter sends the logs). Mandatory Optional N/A N/A N/A
target-port Configure the port on which the target server is listening. Mandatory Optional N/A N/A N/A
protocol Configure the transport protocol to use (TCP or UDP). Mandatory Optional N/A N/A N/A
format Configure the format in which the logs are exported. Optional Optional N/A N/A N/A
read-mode Configure the mode in which the log files are read and exported. Optional Optional N/A N/A N/A
enabled Start the Log Exporter when running the cpstart or mdsstart command. Optional Optional N/A N/A N/A
encrypted Use TLS (SSL) encryption for exporting the logs. Optional Optional N/A N/A N/A
ca-cert Configure the full path to the Certificate Authority PEM certificate file.
Relevant only when the value of 'encrypted' is 'true'.
Optional Optional N/A N/A N/A
client-cert Configure the full path to the client P12 certificate file.
Relevant only when the value of 'encrypted' is 'true'.
Optional Optional N/A N/A N/A
client-secret Configure the challenge phrase that was used to create the client P12 certificate.
Relevant only when the value of 'encrypted' is 'true'.
Optional Optional N/A N/A N/A
filter-action-in

Export all logs with a specific action.
The value must be surrounded by double quotes ("").
Multiple values are supported and must be separated by a comma.

Optional Optional N/A N/A N/A
filter-origin-in

Export all logs from a specific origin.
The value must be surrounded by double quotes ("").
Multiple values are supported and must be separated by a comma.

Optional Optional N/A N/A N/A
filter-blade-in Export all logs that belong to a specific blade.
The value must be surrounded by double quotes ("").
Multiple values are supported and must be separated by a comma.
Predefined blade families can be selected (TP, Access, Endpoint, Mobile).
Optional Optional N/A N/A N/A
--apply-now Apply any change that was done in the "add" immediately. Optional Optional Mandatory N/A Mandatory
export-link Add a field to the exported log that represents a link to SmartView that shows the log card. Optional Optional N/A N/A N/A
export-attachment-link Add a field to the exported log that represents a link to SmartView that shows the log card and automatically opens the attachment. Optional Optional N/A N/A N/A
export-link-ip Make the 'export-link' and the 'export-attachment-link' use a customized IP address (for example, for a Log Server behind NAT). Optional Optional N/A N/A N/A
export-attachment-ids Add a field to the exported log that represents the ID of log's attachment (if exists). Optional Optional N/A N/A N/A
reconnect-interval Schedule a reconnection to the target server. The log exporter will reconnect to the remote server according to the specified time intervals. Optional Optional N/A N/A N/A
export-log-position Send the position of the exported log. Optional Optional N/A N/A N/A

Important Note - Using the 'filter-action-in' / 'filter-origin-in' / 'filter-blade-in' replaces any other filter configuration that was declared earlier on these fields directly in the filtering XML. Other field filters are not overridden.

 

Advanced Configuration After the Deployment

Modifying Log Exporter instance configuration without using the deployment script

Show / Hide the section

After deploying a new instance of Log Exporter, all related files to that deployment can be found in:
$EXPORTERDIR/targets/<Name of Log Exporter Configuration>

On a Multi-Domain Security Management Server / Multi-Domain Log Server, the environment variable EXPORTERDIR exists in each Domain, and its value is changed automatically when you switch between context of Domains with the "mdsenv" command.

Important Note - You must restart the Log Exporter instance for the new settings to take effect. Run the "cp_log_export restart" command.

Target Configuration XML

The Log Exporter configuration for the target server is saved in:
$EXPORTERDIR/targets/<Name of Log Exporter Configuration>/targetConfiguration.xml

These are some configuration options:

Parameter Description Valid / Default Values
<version></version> The current Log Exporter version - used for upgrades.  
<is_enabled></is_enabled> Determines whether the process is monitored by the watch dog.
  • true
  • false

Destination Parameters

Parameter Description Valid / Default Values
type Reserved for future use  
<ip></ip> The IP address or FQDN of the target server Any IPv4 address or FQDN
<port></port> The port on which the target server is listening Any valid port number
<protocol></protocol> The Layer 4 protocol to use TCP or UDP
<reconnect_interval></reconnect_interval> Determines how frequently to start the connection to the target server after it is lost Number of minutes

Security Parameters

Discussed in more detail in the "TLS Configuration" section.

Parameter Description Valid / Default Values
<security></security> Determines whether the connection data is sent in clear text or encrypted.
  • clear (clear text - this is the default)
  • tls (encrypted)
<pem_ca_file></pem_ca_file> The location of the root Certificate Authority PEM file.  
<p12_certificate_file></p12_certificate_file> The location of the client key pair in the P12 format.  
<client_certificate_challenge_phrase></client_certificate_challenge_phrase> The challenge phrase that was used to create the P12 certificate.
The value is hashed after restarting the process.
 

Source Parameters

Parameter Description Valid / Default Values
<folder></folder> The path where the log files are located The default location is $FWDIR/log/
<log_files></log_files> Determines which log records to export or how far back to read the log records from the $FWDIR/log/fw.log file
  • <Number> - reads logs from the specific number (default=1) of days back (recommended)
  • <Specific File Name> - reads logs from the specified file
  • on-line
  • If no value is specified, uses 'on-line'
<log_types></log_types> Determines which logs to export based on their type
  • all (default)
  • log
  • audit
<read_mode></read_mode> Determines whether to export complete logs or only their delta.
  • semi-unified (default)
  • raw

Resolver Parameters

Parameter Description Valid / Default Values
<mappingConfiguration></mappingConfiguration> Configures the XML file that contains the log field mapping scheme.
If left empty, uses the default settings.
Default values are based on the 'format'.
<exportAllFields>true</exportAllFields>

When this field is set to 'true', all log fields are sent regardless of whether they appear in the mapping scheme, except for specifically black-listed fields in the relevant log format mapping file (<exported>false</exported>).

When this field is set to 'false', only those fields which appear in the relevant log format mapping file are sent (with exported flag set to 'true': <exported>true</exported>)

  • true
  • false

Format Parameters

Parameter Description Valid / Default Values
<formatHeaderFile></formatHeaderFile> Configures the XML file that contains the log header format scheme.
If left empty, uses the default settings.
Default values are based on the 'format'.

General Filter Configuration Path

Parameter Description Valid / Default Values
<dynamicFilter></dynamicFilter> Configures the XML file that contains the filtering configuration.
If left empty, uses the default settings.
The default path is:
conf/FilterConfiguration.xml

SmartView links parameters

Parameter Description Valid / Default Values
export_log_link Adds a field to the exported log that represents a link to SmartView that shows the log card.
  • true
  • false (default)
export_attachment_link Adds a field to the exported log that represents a link to SmartView that shows the log card and automatically opens the attachment.
  • true
  • false (default)
export_link_ip Makes the 'export_log_link' and the 'export_attachment_link' use a customized IP address (for example, for a Log Server behind NAT).
  • empty (default)
  • IPv4 Address

Parameters to filter out the Security Gateway connections

This configuration allows Log Exporter instance to filter out the Security Gateway traffic logs for several Software Blades ('Firewall-1 & VPN-1', 'HTTPS Inspection' and 'Security Gateway/Management').

Note - Security Gateway session logs are still exported (generated by tracking a Security Gateway rule per session).

Parameter Description Valid / Default Values
<filter filter_out_by_connection="false">

Determines whether to filter out the access logs.

Note: No other Software Blade filters are currently supported. This is planned in future releases.

  • true (filters out the connection logs)
  • false

Limitation: HTTPS Inspection logs, Security Gateway logs generated not from rules, and a few NAT update logs are still exported.

 

Format Configuration

Show / Hide the section

Every format has its own predefined format configuration file that configures the format of the exported logs, the delimiters, fields that are part of the header, and so on.

These files are located in:
$EXPORTERDIR/targets/<Name of Log Exporter Configuration>/conf/*FormatDefinition.xml

Note: Do not edit the original *FormatDefinition.xml files. Doing so causes a data loss after an upgrade. Instead, create a copy of the file and modify the copied file, while leaving the original intact. After modifying the copied file, refer to it (using a full path) in the <formatHeaderFile> element in the relevant targetConfiguration.xml file.

Body

Parameter Description Syslog Splunk RSA CEF LEEF LogRhythm Generic
<start_message_body></start_message_body> The character preceding the log data payload. [            
<end_message_body></end_message_body> The character following the log data payload. ]            
<message_separator></message_separator> The delimiter that separates logs. &#10; (&#10;=='\n') &#10; ('\n') &#10; ('\n') &#10; ('\n') &#10; ('\n') &#10; ('\n') ('\n')
<fields_separatator></fields_separatator> The delimiter that separates log fields. '; '
(semicolon + space)
|
(pipe)
' '
(space)
' '
(space)
&#09
(<TAB>)
|
(pipe)
' '
(space)
<field_value_separatator></field_value_separatator> The assignment operator. : = = = = = =
<value_encapsulation_start>&quot;</value_encapsulation_start> The value encapsulation operator (start) "       "    "
<value_encapsulation_end>&quot;</value_encapsulation_end> The value encapsulation operator (end) "       "   "
<escape_chars>
  <char>
    <orig></orig>
    <escaped></escaped>
  </char>
</escape_chars>

Escaping unwanted characters.

The escape functionality replaces the string that's encapsulated by the 'orig' tags with the one encapsulated by the 'escaped' tags

;\ --> \\
" --> \"
&#10; --> ' '
] --> \]
| --> ;
= --> \=
&#10; --> ' '
= --> \=

&#10; --> ' '
;\ --> \\
= --> \=
&#10; --> ' '
| --> \|
= --> \=
&#10; --> ' '
| --> ;
= --> \=
&#10; --> ' '
\ --> \\
" --> '
&#10; --> ' '

Header

Parameter Description Default values<br/ >for Syslog Default values<br/ >for Splunk Default values<br/ >for RSA Default values<br/ >for CEF Default values<br/ >for LogRhythm
<header_format></header_format> The delimiter between the header values and the number of values. Every {} is replaced with one value. ' ' (space) time={}|hostname={}| <134> | ; LOGV2 {}|

Notes:

  • To add a constant string to the header, add the string to the <header_format> tag value.
  • To add a new field to the header, add a new header format replacement string (for example: {}) to the <header_format> and add the relevant information in the <headers> tag.

 

Fields Configuration

Show / Hide the section

Every format has its own predefined fields configuration file that allow to change the name / value of the exported field, filter out irrelevant fields, and so on.

These files are located under each deployment directory:
$EXPORTERDIR/targets/<Name of Log Exporter Configuration>/conf/*FieldsMapping.xml

Note: Do not edit the original *FieldsMapping.xml files. Doing so causes a data loss after an upgrade. Instead, create a copy of the file and modify the copied file, while leaving the original intact. After modifying the copied file, refer to it (using a full path) in the <mappingConfiguration> element in the relevant targetConfiguration.xml file.

Parameter Description Values
<table> Some fields appear in tables depending on the log format.
This information can be found in the ELG log - one entry for every new field.
A field can appear in multiple tables, each distinct instance is considered as a new field.
 
<exported></exported>

Optional.
You can filter out specific fields by using the 'exported' tag (value: true, or false) in the mapping configuration file.
Alternatively, if the 'exportAllFields' tag in the 'targetConfiguration.xml' file is set to 'false', only those fields which are listed in the mapping file are exported.

Note - This parameter can also be configured on a table field level to allow / prevent its export when the field is part of a table.

  • true
  • false
<origName></origName>

The name of the field to be mapped to <dstName>.

Refer to Log Fields Mapping for Advanced Fields Configuration.

 
<dstName></dstName> The new mapping scheme name for the desired field.  
<required></required> Optional.
When set to 'true', only logs which contain this field are exported.
  • true
  • false

 

TLS Configuration

Show / Hide the section

How to set a secured connection between the Log Exporter and the Syslog server

Log Exporter can export logs over an encrypted connection using the TLS protocol.

When using TLS, it is important to know that only mutual authentication is allowed.

For mutual authentication, the Log Exporter needs these certificates:

  • CA certificate (in the PEM format) that signed both the client (Log Exporter side) and server (syslog server side) certificates
  • Client certificate (in the P12 format) on the Management Server / Log Server with Log Exporter

Notes:

  • The Management Server / Log Server with Log Exporter must be able to connect to CA.
  • In addition to these two certificates, a third certificate should be installed on the Syslog server (based on the server requirement). It is also possible to use self-signed certificates.

The procedure below uses the openssl commands on a non-Check Point server.

Creating a CA Certificate:

  1. Create a CA key:

    openssl genrsa -out ca.key 2048

  2. Create a CA Certificate:

    openssl req -x509 -new -nodes -key ca.key -days 2048 -out ca.pem

    You are prompted to provide information regarding the certificate. Apart from the Common Name (it is recommended to use the device IP address as the Common Name), all other fields are optional and can be skipped. If you are purchasing an SSL certificate from a certificate authority, it often requires these additional fields.

Creating a Log Exporter Certificate:

  1. Create a Log Exporter key:

    openssl genrsa -out cp_client.key 2048

  2. Create a Log Exporter CSR file:

    openssl req -new -key cp_client.key -out cp_client.csr

  3. Create a Log Exporter CRT file:

    openssl x509 -req -in cp_client.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out cp_client.crt -days 2048 -sha256

  4. Create Log Exporter P12 file:

    openssl pkcs12 -inkey cp_client.key -in cp_client.crt -export -out cp_client.p12

Creating a Syslog Server Certificate:

  1. Create a Target Server key:

    openssl genrsa -out server.key 2048

  2. Create a Target Server CSR file:

    openssl req -new -key server.key -out server.csr

  3. Create a Target Server CRT file:

    openssl x509 -req -in server.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out server.crt -days 2048 -sha256

Note: Several SIEM applications require the Server certification to be in a specific format. For more information, refer to the section "SIEM Specific instructions".

 

Filter Configuration

Choose what to export based on field values.

Show / Hide the section

The Log Exporter can filter logs based on the field values.

Because field mapping operation is done before the actual filtering, make sure to use the dst name / value of the fields, if configured specifically in the *FieldsMapping.xml file (based on the format). Otherwise, use the name / value as written in the raw log file.

It is possible to configure what to export or what not to export.

The filter configuration file is located in:
$EXPORTERDIR/targets/<Name of Log Exporter Configuration>/conf/FilterConfiguration.xml

Parameter Description Valid / Default Values
<filterGroup operator=""></filterGroup>

A group of fields that determine what to export.

The relation between the fields is determined by the operator value.

Refer to Log Fields Mapping for Advanced Fields Configuration.

operator[and | or]

<field name="" operator="">
  <value operation=""></value>
</field>

Declare a single field filter that participates in the filter group.

  • name - The name of the field to filter on.
  • operator - Declares the operator (and / or) between the various declared operations.
  • operation - Declares the matching logic regarding the declared value.
  • value - The specific value to filter on. Multiple values for a single operation is supported and should be added as a separate row.

operator[and | or]

operation[eq - equal | neq - not equal | gt - greater than | lt - less than ]

These are the ways to configure the filtering feature:

Configuration Method Description

Using the cp_log_export command

This command configure filtering for Action / Blade / Origin fields only.

The syntax is:

  • cp_log_export set name <name> filter-action-in "value1,value2"

  • cp_log_export set name <name> filter-origin-in "value1,value2"

  • cp_log_export set name <name> filter-blade-in "value2"

In addition, it is possible to use predefined families for "filter-blade-in" value:

  • Access - For exporting Access logs only (Security Gateway/Management, VPN-1 & FireWall-1, Firewall, Application Control, URL Filtering, Content Awareness, Connectra, Mobile Access, Compliance blade, Core, DDoS Protector, Identity Awareness, Identity Logging, UA WebAccess).
  • TP - For exporting Threat Prevention logs only (Anti-Bot, Anti-Malware, Threat Emulation, IPS, IPS-1, SmartDefense, Anti-Virus, New Anti Virus, Anti-Spam and Email Security, Threat Extraction, MTA).
  • EndPoint - For exporting Endpoint logs only (Anti-Bot, Anti Malware, Threat Emulation, IPS, IPS-1, SmartDefense, Anti-Virus, New Anti Virus, Anti-Spam and Email Security, Threat Extraction, MTA ).
  • Mobile - For exporting Mobile logs only (WIFI Network, Mobile App, OS Exploits, Device, Network Security, Cellular Network, Network Access, iOS Profiles, Text Message, On-device Network Protection).

Modifying the FilterConfiguration.xml file manually

You can new fields to this file.

For example:

<filters>
  <filterGroup operator="and">
    <field name="action" operator="and">
    </field>
    <field name="origin" operator="and">
    </field>
    <field name="product" operator="or">
    </field>
    <field name="severity" operator="or">
      <value operation="eq">3</value>
        <value operation="eq">4</value>
    </field>
  </filterGroup>
</filters>

Limitations:

  • The relation between the values of the same operation is only logical "OR".

    Example:

    cp_log_export set name <target-name> filter-action-in "accept,drop"

    Only logs with action = "accept" OR action = "drop" are exported.

  • Filtering is not supported for any of these fields:

    • app_category
    • app_desc
    • app_properties
    • app_risk
    • app_rule_name
    • appi_name
    • category
    • cvpn_category
    • cvpn_resource
    • desc
    • HTTPS_inspection_rule_name
    • matched_category
    • name
    • properties
    • time
    • UUID
  • Filtering for a certain field with the a double "NOT" condition "not equal(value1) OR not equal(value2)" is not supported.
    When editing the filtering XML, make sure to have a maximum of one line of "neq" operation in each field.

 

Log Fields Mapping for Advanced Fields Configuration

Show / Hide the section

Check Point Infinity solution includes multiple log fields, representing the diversity of Check Point's products.
The log fields mapping helps you understand security threats, logs language, complex queries and SIEM.

For information on Check Point's Log Fields Mapping, refer to sk144192.

 

SIEM Specific instructions

How to configure SIEM applications to receive logs optimally.

Show / Hide the section

Rsyslog

Show / Hide the section

By default, Rsyslog is not configured to use the RFC 5424 timestamp format. Therefore, you should manually change the Rsyslog setting for it to be compliant with the Log Exporter output format.

On the Rsyslog server:

  1. Edit the /etc/rsyslog.conf file.

  2. Comment out this line (add the # character in the beginning), if it is not commented out already:

    #"$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat"

  3. Add this line in the file:

    $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format

  4. Save the change in the file and close it.

  5. Restart the Rsyslog service:

    service rsyslog restart

ArcSight

Show / Hide the section

ArcSight recommends to name the server certificate file as 'syslog-ng'.

  1. Convert the key to the P12 format:

    openssl pkcs12 -inkey syslogServer.key -in syslogServer.crt -export -out syslog-ng.p12 -name "syslogng-alias" -password pass:changeit

  2. Make sure the value of the environment variable ARCSIGHT_HOME is the path to the connector installation directory.

  3. Run the certificate manager on the Linux KDE console:

    $ARCSIGHT_HOME/current/bin/arcsight agent keytoolgui

  4. From the File menu, open the keystore:

    $ARCSIGHT_HOME/current/jre/lib/security/cacerts

    The password is: changeit
  5. From the menu, select Import Trusted Certificate

  6. From the file dialog, select ca.pem and save it.

  7. Save and close the certificate manager.

  8. Edit the agent.properties file to enable the mutual authentication:

    vi $ARCSIGHT_HOME//current/user/agent/agent.properties

  9. For the 'syslogng.mutual.auth.enabled' parameter, configure the value 'true':

    syslogng.mutual.auth.enabled=true

  10. Add these lines at the bottom of the agent.properties file:

    syslogng.tls.keystore.file=user/agent/syslog-ng.p12

    syslogng.tls.keystore.alias=syslogng-alias

  11. Save the change in the file and close it.

  12. Restart the service:

    /etc/init.d/arc_connector_name restart

Splunk

Show / Hide the section

We recommend to use Check Point App for Splunk when exporting logs to a Splunk server.

For more information about installation and deployment, see the Check Point App for Splunk User Guide.

In addition, to configure an encrypted connection:

  1. Generate the server PEM file according to the Splunk TLS Documentation:

    cat syslogServer.crt syslogServer.key RootCA.pem > splunk.pem

  2. Edit the 'inputs.conf' file on the Splunk server:

    vi /opt/splunk/etc/apps/<Name of the app, where the configuration is saved>/local/inputs.conf

  3. Update the file to use TLS:

    [SSL]
    serverCert = <full path to server PEM file>
    sslPassword = <challenge password>
    requireClientCert = true
    [tcp-ssl://<port>]
    index = <index>
    
  4. Save the change in the file and close it.

  5. Edit the 'server.conf' file on the Splunk server:

    vi /opt/splunk/etc/system/local/server.conf

  6. Update the file to use the relevant Certificate Authority PEM file:

    [sslConfig]
    sslRootCAPath = <full path to CA PEM file>
    
    [SSL]
    cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH
    
  7. Save the change in the file and close it.

  8. Restart the Splunk daemon:

    /opt/splunk/bin/splunk restart

QRadar

Show / Hide the section
  1. In the Authentication Mode field, select TLS And Client Authentication.

  2. Upload the Check Point certificate and private key to QRadar to the same directory.

  3. Provide the absolute path the uploaded files in the Provide Certificate option.

Notes:

  • When using Client Authentication, you must provide the absolute path to the client certificate.
  • Make sure the "Common Name" is unique in every certificate.

 

Transition from LEA to Log Exporter

Recommended method to move from the existing LEA connector to the new Log Exporter.

Show / Hide the section
  1. Delete the OPSEC application object from SmartConsole R80.10 and higher / SmartDashboard R77.30, if you only use it for the OPSEC application.
    Alternatively, remove the LEA client entity from it, if you do not use it:

  2. If this is the only OPSEC LEA client, configure the $FWDIR/conf/fwopsec.conf file (on the Check Point Management Server / Log Server) to not allow LEA (if this is not the only OPSEC LEA client, skip this step):

    Change these lines:

    From:

    To:

    #
    lea_server auth_port 18184
    lea_server port 0
    #
    
    #
    #lea_server auth_port 18184
    #lea_server port 0
    #
    
  3. Install the Log Exporter. See the Installation section.

 

Transition from CPLogToSyslog to Log Exporter

Recommended method to move from the existing CPLogToSyslog to the new log exporter.

Show / Hide the section
  1. Uninstall the CPLogToSyslog package using CPUSE. See the instructions in sk92449 - section 4-C.

  2. Install the Log Exporter. See the Installation section.

 

Troubleshooting

Show / Hide the section
Symptoms Cause Suggested solution
Logs are not exported after adding a filter to the FilterConfiguration.xml file, or by using the cp_log_export command. cp_log_export adds the default values to the FilterConfiguration.xml file, while the field names should be the same as the exported name. It causes the filter mechanism to not match any log.
  1. In the relevant XXXFieldsMapping.xml file, look for the relevant mapped field.

  2. Find the element named <dstName> and copy it.

  3. Edit the <exporter-dir>/conf/FilterConfiguration.xml file.

  4. Replace the field name to the previously copied one.

Assigning a field in the FieldsMapping file as export false, but keep exporting the field.

For example: When the user assign for the field "layer_uuid export=false", but he keeps seeing this field as part of the log in the Log Server.

The field is part of a table in the log, and the standard configuration to filter out a field is not effective on a table field.

To prevent these fields from been exported, you need to:

  1. Go to the $EXPORTERDIR/targets/<exporter_name>/conf/ directory.

  2. Edit the "Fields Mapping" file you use (that corresponds to the format you export).

  3. Look for 'match_table' tag:

    <tableName>match_table</tableName>
    
  4. Add the required lines:

    • If the 'match_table' tag does not exist, add these lines inside the 'fields' tag:

      
      <table>
        <tableName>match_table</tableName>
         <fields>
          <field><origName>field_name</origName><exported>false</exported></field>
         </fields>
      </table>
      
    • If the 'match_table' tag exists, add this line inside the 'fields' tag:

      <table>
        <tableName>match_table</tableName>
         <fields>
          <field><origName>field_name</origName><exported>false</exported></field>
         </fields>
      </table>
      

    The file should look like this:

    <fields>
    <!-- Filter out fields -->
      <field><origName>field_name1</origName><exported>false</exported></field>
      <field><origName>field_name2</origName><exported>false</exported></field>
      <field><origName>field_name3</origName><exported>false</exported></field>
        ... ...
        <table><tableName>match_table</tableName>
          <fields>
            <field> ... </field>
            ... ...
            <field><origName>field_name</origName><exported>false</exported></field>
          </fields>
        </table>
    
    <!-- End of filter out -->
    </fields>
    
  5. Save the change in the file and close it.

  6. Restart the Log Exporter to load the new settings:

    cp_log_export restart name <exporter_name>

 

Known Limitations

Show / Hide the section
  • If you want to change this environment, you must first consult your Check Point partner or vendor. This release is built for the specific environment. Upgrades or other changes can overwrite Hotfix functionality and environment customizations.

  • On a Multi-Domain Security Management Server / Multi-Domain Log Server, when exporting logs from a certain Domain using the UDP protocol, the exported log's header contains the IP address of the Multi-Domain Server and not the IP address of the Domain Management Server / Domain Log Server.

 

Appendix

Show / Hide the section

Special log fields

Field Description
loguid

Some Check Point logs are updated over time. Update logs have the same loguid value. Check Point SmartLog client correlates those updates into a single unified log.

When sending update logs to 3rd party (SIEM) servers, in the default raw read-mode, they arrive as distinct logs. Best use the semi-unified read-mode, which sends few instances of the log, but each instance contains the entire event chain until this update. Administrators can alternatively use the loguid field to correlate update logs and get the full event chain themselves.

Note: All related log & log-updates share the same initial time as the 1st log (in semi-unified mode).

Example of update logs includes the total number of bytes sent and received over time or the severity field which is updated over time as more information becomes available.

hll_key

Stands for High-Level Log key. This concept was introduced in R80.10, where Multiple connection logs can comprise one session with one shared hll_key.

For example, when browsing a webpage, you might have multiple connection logs which are related to the same session. Connection logs which are part of the same session share the same hll_key value.

Syslog-NG Listener configuration

When configuring a source on a Syslog NG server it is recommended to use the syslog-protocol flag.

For example:

source s_network { network(transport("tcp") port(514) flags(syslog-protocol) ); };

Splunk Listener configuration

It is recommended to add these time settings to your source type:

TIME_FORMAT = %s
TIME_PREFIX = time=
MAX_TIMESTAMP_LOOKAHEAD = 15

ArcSight Listener configuration

The Log Exporter solution does not work with the OPSEC LEA connector. Instead, you must install the ArcSight Syslog-NG connector.

ArcSight Common Event Format (CEF) Mapping

CEF is an extensible, text-based format designed to support multiple device types by offering the most relevant information. Message syntaxes are reduced to work with ESM normalization. Specifically, CEF defines a syntax for log records comprised of a standard header and a variable extension, formatted as key-value pairs. The CEF format can be used with on-premise devices by implementing the ArcSight Syslog SmartConnector. CEF can also be used by cloud-based service providers by implementing the SmartConnector for ArcSight Common Event Format REST.

CEF Header format

  Version Device Vendor Device Product Device Version Device Event Class ID Name Severity
Default CEF:0 Check Point Log Update Check Point Log Log 0
Values - - Product Name (Blade) -

Attack Name

Protection Type

Verdict

Matched Category

DLP Data Type

Application Category

Application Properties

Protection Name

Application Name

Message Info

Service ID

Service

Application Risk

Risk

Severity

QRadar Log Event Extended Format (LEEF) Mapping

The Log Event Extended Format (LEEF) is a customized event format for IBM Security QRadar.

LEEF Header format

  LEEF:Version Vendor Product Version EventID
Default LEEF:2.0 Check Point Log Update 1.0 Check Point Log
Values - - Product Name (Blade) -

Protection Name

Application Name

Action

Note: The time format is not compliant with the official LEEF format.

Until the time IBM adds support for Epoch time format, Log Exporter with LEEF format is only partially supported.

 

Change Log

Show / Hide the section

Note: Takes listed in the table below are related to the Log Exporter, and not to Jumbo Hotfix Accumulators.

Issue ID Description
Log Exporter Take 51 (14 April 2019)
SL-1822

Installation of R80_10_JHF_LOGOUT fails because of the script updateExistingExporters.sh when there are no exporters in the server.

Log Exporter Take 50 (05 March 2019)
SL-2003

Added filtering support. Now you can decide which logs to export.

Log Exporter Take 43 (20 January 2019)
PMTR-13842

Added support for exporting logs to the new Check Point's Splunk application.

SL-1817

Log Exporter was getting stuck after 7 hours of uptime

SL-1932

Log Exporter could not be installed on top of R80.10 Jumbo Hotfix Accumulator Take_169 and above.

 

 

Revision History

Show / Hide the section
Date Description
26 Sep 2021 Improved the article formatting
15 Aug 2021 Updated that configuration in SmartConsole is available starting from R81
28 July 2021 Added the RSA and JSON formats
13 July 2021 Improved the article formatting
17 Mar 2021 Updated the description of existing functionalities
13 Oct 2020 Updated "Known Limitations" section
19 Aug 2020 Updated the "Installation" section
06 July 2020 Edited R80.30 installation section
09 June 2020 Changed the note to "Audit logs exist on every Log Server"
13 Apr 2020 Updated the "Limitations" in the "Filtering Configuration" section
19 Feb 2020 Updated to Check_Point_R80.20_JHF_T118_Log_Exporter_Enhancements_T5_sk122323_FULL.tgz which fixed an issue with ds.conf when installing on top of R80.20 Jumbo Hotfix Accumulator Take_118.
19 Jan 2020 Added a new scenario in the "Troubleshooting" section
01 Jan 2020 Added the LogRhythm format on top of R80.20 JHF Take 118 and on top of R80.30 JHF Take 111
12 Dec 2019 Added the "Troubleshooting" section
02 Oct 2019 Added SmartView links on top of R80.20 JHF Take 103
08 July 2019

Added the "Log Fields Mapping for Advanced Fields Configuration" section

Give us Feedback
Please rate this document
[1=Worst,5=Best]
Comment