Support Center > Search Results > SecureKnowledge Details
Log Exporter - Check Point Log Export Technical Level
Solution

Check Point "Log Exporter" is an easy and secure method for exporting Check Point logs over syslog.
Exporting can be done in few standard protocols and formats.

Log Exporter supports:

  • SIEM applications: Splunk, LogRhythm, Arcsight, RSA, QRadar, McAfee, rsyslog, ng-syslog and any other SIEM application that can run a syslog agent.
  • Protocols: syslog over TCP or UDP.
  • Formats: Syslog, Splunk, CEF, LEEF, LogRhythm, Generic.
  • Security: Mutual authentication TLS 1.2.
  • Log Types: The ability to export security logs / audit logs or both.
    (Note: Audit logs exist on both MGMT and Log Server).
  • Filter out firewall connection logs.
  • Filtering: choose what to export based on field values.
  • Links to Log Attachments: export links to the relevant log card in SmartView and the log attachments.
The following table contains release information of the features:

Feature / Capability Description R80.20 R80.30 R80.40 R81
Filtering Choose what to export based on field values Jumbo HF Take 103 and higher Jumbo HF Take 107 and higher GA GA
Links to Log Attachments Export links to the relevant log card in SmartView and the log attachments (Forensics\Threat Emulation reports) Jumbo HF Take 127 and higher Jumbo HF Take 107 and higher GA GA
Attachments IDs API Export identifiers of attachments for fetching them via Log API Jumbo HF Take 183 and higher Jumbo HF Take 217 and higher Jumbo HF Take 78 and higher GA
DNS Name Usage Configure DNS name (FQDN) as the target-server in addition to IP address Jumbo HF Take 190 and higher Jumbo HF Take 228 and higher Jumbo HF Take 92 and higher Jumbo HF Take 13 and higher
Reconnection to Load Balancer Support Initiate reconnection to load balancer every X minutes (configurable) Jumbo HF Take 190 and higher Jumbo HF Take 228 and higher Jumbo HF Take 92 and higher Jumbo HF Take 13 and higher

Table of Contents

  • How does it Work
  • Installation
  • Uninstall
  • Basic Deployment
  • Advanced Deployment - Additional Commands
  • Advanced Configuration Post Deployment
  • Format Configuration
  • Fields Configuration
  • TLS Configuration
  • Filter Configuration
  • Log Fields Mapping for Advanced Fields Configuration
  • SIEM Specific instruction
  • Transition from LEA to Log exporter
  • Transition from CPLogToSyslog to Log exporter
  • Troubleshooting
  • Known Limitations
  • Appendix
  • Change Log

 

Click Here to Show Entire Article

 

How Does It Work

Show / Hide the section

Log Exporter is a multi-threaded daemon service, running on a log server. Each log that is written on the log server is read by the log exporter daemon, transformed into the desired format and mapping and then sent to the end target.

On MDS/MLM, if log exporter is deployed on several domains, each domain server will have its own log exporter daemon service. If exporting the logs to several targets, each target will have its own log exporter daemon.

Log Exporter is implemented as ETL procedure:

  • Extract - Reads incoming logs from the Security Gateway, stored in local files.

  • Transform - Changes the logs according to configuration files (both exported format and field name/values, irrelevant fields removal).

  • Load - Sends the logs to the configured target server in TCP \ UDP (takes into consideration filter configuration if exists).

  • Data integrity - Log Exporter stops exporting when disconnected from the 3rd party and remember the last position exported.

    Once the connection is re-established, log exporter will automatically start exporting logs from the last known position.

Log exporter is exporting both online and offline (if any) logs in parallel. In case the 3rd party server is slow, log exporter will reduce the offline exporting rate, to prioritize the online logs over them.

 

Installation

Show / Hide the section

Log Exporter is integrated in R80.10 with Jumbo Hotfix Accumulator Take_270 and higher versions.

  • R77.30

    Install this release on a R77.30 Multi-Domain Server, Multi-Domain Log Server, Security Management Server, Log Server or SmartEvent Server.

    Note: Log Exporter can be installed on top of R77.30 Jumbo Hotfix Take 292 and above. 

    **This hotfix must be installed after the Jumbo, and will need to be uninstalled to upgrade to a higher Jumbo take, and then reinstalled after the newer Jumbo is in place. 

     

    Version Date CPUSE Online Identifier CPUSE offline
    package
    R77.30 14 April 2019 Check_Point_R77.30_Log_Exporter_T36_sk122323_FULL.tgz (TGZ)

     

    Install the hotfix using CPUSE, see sk92449.

 

Uninstall

Show / Hide the section


  • Management server - uninstall the feature via CPUSE, see sk92449.

  • Multi-Domain Management server - Uninstalling the package does not remove the configuration files, in order to uninstall and completely remove all configurations, do the following:

    1. Run:

      cp_log_export delete name all [domain-server all] --apply-now

      Note: Do not forget to add: <domain-server all> on MDS/MLM machines.

    2. If running on Multi-Domain Management Server: switch to MDS's environment by running: # mdsenv

    3. Make sure $EXPORTERDIR exists and is pointing to $RTDIR/log_exporter ($FWDIR/log_exporter on R77.30). Run:

      # rm -rf $EXPORTERDIR

    4. Uninstall the hotfix using CPUSE, see sk92449.

Note: After uninstalling the hotfix with CPUSE, reboot the machine.

 

Basic Deployment

Common method for creating / modifying log exporters / targets.

Show / Hide the section

There are 2 ways to configure Log Exporter: SmartConsole and CLI.

In order to configure Log Exporter using CLI commands, do the following on the log server:

cp_log_export add name <name> [domain-server <domain-server>] target-server <target-server IP/host name> target-port <target-port> protocol <(udp|tcp)> format <(syslog)|(cef)|(splunk)|(logrhythm)|(generic)> [optional arguments]

 

  • On MDS/MLM: domain-server argument is mandatory.
    • Use 'mds' as the value for domain-server in order to export mds level audit logs.
    • Use 'all' as the value for domain-server in order to configure instance on every domain.
    • Use the domain-server IP address / Name to configure the instance on a specific domain.
  • Target-server can use either the target server IP address or it's DNS name.
This will create a new target directory with the unique name specified in the name parameter under $EXPORTERDIR/targets/<deployment_name>, and set the target configuration parameters with the connection details: IP Address, port, protocol, format and read-mode.
The above deployment will export logs in clear text. In order to export logs using encryption, please see "Advanced Deployment - Additional Commands" section.

The new log exporter does not start automatically.
To start it run: cp_log_export restart
 

In order to configure Log Exporter using SmartConsole, please see Logging and Monitoring Administration Guide

 

Advanced Deployment - Additional Commands

Advanced parameters for creating / modifying log exporters / targets

Show / Hide the section


Usage

cp_log_export <command-name> [command-arguments]

In order to understand a specific command usage run:
cp_log_export <command-name> help

 

Commands


Command Name Command Description
add Deploys a new Log Exporter instance.
set Updates an existing Log Exporter instance configuration.
delete Removes an existing Log Exporter instance.
show Prints the current configurations of the existing Log Exporter instances.
status Prints the overview statuses of the existing Log Exporter instances.
start Starts Log Exporter instance. 
stop Stops Log Exporter instance.
restart Restarts Log Exporter instance.
reexport Resets the current read position and re-exports all logs per the Log Exporter instance configuration.

Parameters


Parameter Name Description add set delete show/status/start/stop/restart reexport
 name Unique name of the exporter configuration. Mandatory Mandatory Mandatory Optional - Default all  Mandatory
 domain-server The relevant domain-server name or IP Mandatory Mandatory Mandatory Optional - Default all Mandatory
 target-server Exporting the logs to this ip address  Mandatory Optional  N/A  N/A N/A
 target-port The port on which the target is listening to Mandatory Optional N/A N/A N/A
 protocol Transport protocol to use Mandatory Optional  N/A N/A N/A
 format The format in which the logs will be exported  Optional  Optional  N/A  N/A  N/A 
 read-mode The mode in which the log files will be read and exported Optional  Optional N/A N/A N/A
 enabled Allow log_exporter to start on cpstart/mdsstart Optional  Optional N/A  N/A N/A
 encrypted Using TLS (SSL) encryption for exporting the logs Optional Optional N/A  N/A  N/A 
 ca-cert Full path to the CA pem certificate file
Relevant only when encrypted is true 
Optional Optional  N/A N/A N/A
 client-cert  Full path to the client p12 certificate file Relevant only when encrypted is true Optional Optional N/A N/A N/A
 client-secret The challenge phrase that was used in order to create the client p12 certificate
Relevant only when encrypted is true
Optional Optional N/A  N/A N/A
 filter-action-in

Exporting all logs with a specific action. value should be surrounded by "" and multiple values are supported separated by a comma.

Optional Optional N/A N/A N/A
 filter-origin-in

Exporting all logs from a specific origin. value should be surrounded by "" and multiple values are supported separated by a comma.

Optional Optional  N/A  N/A  N/A 
 filter-blade-in

Exporting all logs that belong to a specific blade. value should be surrounded by "" and multiple values are supported separated by a comma.

Predefined blade families can be selected (TP, Access, Endpoint, Mobile).

Optional Optional N/A N/A N/A
 --apply-now Applying any change that was done in the add immediately Optional Optional Mandatory N/A Mandatory
 export-link Add a field to the exported log that represents a link to SmartView that shows the log card Optional Optional N/A N/A N/A
 export-attachment-link Add a field to the exported log that represents a link to SmartView that shows the log card and automatically opens the attachment Optional Optional N/A N/A N/A
 export-link-ip Make the above 2 links use a customized IP (e.g. for NATed log server) Optional Optional N/A N/A N/A
export-attachment-ids Add a field to the exported log that represents the id of log's attachment (if exists)  Optional Optional N/A N/A N/A
reconnect_interval Schedule reconnection to the target server Optional Optional N/A N/A N/A

Note: Using filter-action-in \ filter-origin-in \ filter-blade-in will replace any other filter configuration that was pre declared on these fields directly in the filtering XML. Other fields filters will not be overridden.

 

Advanced Configuration Post Deployment

Modifying Log Exporter instance configuration without using the deployment script

Show / Hide the section

After deploying a new instance of Log Exporter, all related files to that deployment can be found under $EXPORTERDIR/targets/<deployment name>

On an MDS/MLM server, EXPORTERDIR environment variable is per domain, and its value is changed automatically when switching between domain server contexts with the mdsenv command.

 Note: You must restart Log Exporter instance for the new settings to take effect.

Target Configuration XML

Log Exporter target configuration is saved in a file located under each deployment folder: $EXPORTERDIR/targets/<deployment_name>/targetConfiguration.xml

Listed below are some of the configuration options:

Parameter Description Possible/Default Values
<version></version> Current Log Exporter version - used for upgrades.  
<is_enabled></is_enabled> Determines whether or not the process will be monitored by the watch dog.  true/false

Destination Parameters

Parameter Description Possible/Default Values
type Reserved for future use.  
<ip></ip> The IP address or DNS name of the target server that will receive the logs Any IPv4 address or DNS name.
<port></port> The port on which the target is listening to. Any valid port number.
<protocol></protocol>  The protocol that will be used in the connection.  UDP / TCP
<reconnect_interval></reconnect_interval> Determines whenever to reinitiate the connection to the target server Number of minutes

Security Parameters

Discussed in more detail in the "TLS Configuration" section.

Parameter Description Possible/Default Values
<security></security> Determines weather or not the connection data will be sent in clear text or encrypted.  clear [default] / tls
<pem_ca_file></pem_ca_file> The location of the root CA pem file.  
<p12_certificate_file></p12_certificate_file> The location of the client key pair in p12 format.  
<client_certificate_challenge_phrase></client_certificate_challenge_phrase> The challenge phrase that was used in order to create the p12 certificate.
Value will be hashed after after restarting the process .
 

Source Parameters

Parameter  Description Possible/Default Values 
<folder></folder> The path where the log files are located. Default location is $FWDIR/log/
<log_files></log_files> Determines which log files will be exported or how far back to read logs from fw.log. read logs from [number] - (default=1) days back (recommended) | <specific file name> | on-line (no value=on-line)
<log_types></log_types> Determines which log will be exported based on their type all [default] / log / audit
<read_mode></read_mode> Determines whether to export complete logs or delta only. semi-unified [default] | raw

Resolver Parameters

Parameter Description Possible/Default Values
<mappingConfiguration></mappingConfiguration> The XML file containing the log field mapping scheme. If left empty will use the default settings. Default values are based on the 'format'.
<exportAllFields>true</exportAllFields>

When this field is set to 'true' all log fields will be sent regardless of whether or not they appear in the mapping scheme, , except for specifically black-listed fields in the relevant log format mapping file (<exported>false</exported>).

When set to 'false' only those fields which appear in the relevant log format mapping file will be sent (with exported flag true: <exported>true</exported>)

 true / false

Format Parameters

Parameter Description Possible/Default Values
<formatHeaderFile></formatHeaderFile> The XML file contains the log header format scheme. If left empty will use the default settings. Default values are based on the 'format'.

General Filter Configuration Path

Parameter Description Possible/Default Values
<dynamicFilter></dynamicFilter> The XML file containing the filtering configuration. If left empty, default configuration will be used. Default path is conf/FilterConfiguration.xml

SmartView links parameters

Parameter Description Possible/Default Values
export_log_link Add a field to the exported log that represents a link to SmartView that shows the log card. True/False [default]
export_attachment_link Add a field to the exported log that represents a link to SmartView that shows the log card and automatically opens the attachment. True/False [default]
export_link_ip Make the above 2 links use a customized IP (e.g. for NATed log server). IPv4 / empty [default]

Filter out firewall connections Parameters

This configuration allows Log Exporter instance to filter out firewall connection logs for several blades ('Firewall-1 & VPN-1', 'HTTPS Inspection' and 'Security Gateway/Management').

Note: Firewall session logs will still be exported (Generated by tracking a firewall rule by per session).  

Parameter Description Possible/Default Values
<filter filter_out_by_connection="false">

 Determines if the access logs should be filtered out.

When set to 'true' VPN-1 & Firewall-1 connection logs will be filtered out

Note: No other blade filters are currently supported. This will be expanded upon in future releases.

 true / false

Limitation: HTTPS inspection logs, Non-rulebase generated Firewall logs & a few Firewall NAT update logs will still be exported.

 

Format Configuration

Show / Hide the section

Every format has its own predefined format configuration file that defines the format of the exported logs, what will be the delimiters, what fields will be part of the header and etc.

These files are located under each deployment folder: $EXPORTERDIR/targets/<deployment_name>/conf/*FormatDefinition.xml

Note: Do not edit the original *FormatDefinition.xml files. Instead, copy the file to a different file name and modify the copied file while leaving the original intact. After done modifying the file, refer to it (full path) in <formatHeaderFile> element in the relevant targetConfiguration.xml.

Body

Parameter Description Syslog Splunk RSA CEF LEEF LogRhythm Generic
<start_message_body></start_message_body> The character preceding the log data payload. [          
<end_message_body></end_message_body> The character following the log data payload. ]          
<message_separator></message_separator> The delimiter that separates logs. &#10; (&#10;=='\n') &#10; ('\n') &#10; ('\n') &#10; ('\n') &#10; ('\n') &#10; ('\n') ('\n')
<fields_separatator></fields_separatator> The delimiter that separates log fields. '; ' (semi colon, space)  | (pipe)  ' ' (space)  ' ' (space)  &#09; (<TAB>)  | (pipe)  ' ' (space)
<field_value_separatator></field_value_separatator> The assignment operator.  :  = =  =  =  =  =
<value_encapsulation_start>&quot;</value_encapsulation_start> The value encapsulation operator (start)  "      "    "
<value_encapsulation_end>&quot;</value_encapsulation_end> The value encapsulation operator (end)  "      "    "
<escape_chars>
<char>
<orig></orig>
<escaped></escaped>
</char>
</escape_chars>

Escaping unwanted characters.

The escape functionality will replace the string that's encapsulated by the 'orig' tags with the one encapsulated by the 'escaped' tags 

 \ --> \\

" --> \"

&#10; --> ' '

] --> \]

 | --> ;

= --> \=

&#10; --> ' '

= --> \=

&#10; --> ' '
 \ --> \\

= --> \=

&#10; --> ' '

| --> \|

= --> \=

&#10; --> ' '

| --> ;

= --> \=

&#10; --> ' '

 \ --> \\

" --> '

&#10; --> ' '

 

Header

Parameter Description Default values for syslog Default values for Splunk Default values for RSA Default values for CEF Default values for LogRhythm
<header_format></header_format>  The delimiter between the header values and the number of values. Every {} will be replaced with one value.  ' ' (space) time={}|hostname={}| <134> LOGV2 {}|

Note: If you want to add constant string to the header, you can do this by adding the string to <header_format> tag value.
Note: If you want to add a new field to the header, you need to add a new header format replacement string (for example: {}) to the <header_format> and also add the relevant information under <headers> tag.

 

Fields Configuration

Show / Hide the section

Every format has its own predefined fields configuration file that allow to change the name / value of the exported field,  filter out irrelevant fields and etc.
These files are located under each deployment folder: $EXPORTERDIR/targets/<deployment_name>/conf/*FieldsMapping.xml

Note:
Please do not edit the original *FieldsMapping.xml files. Copy the file to a different file name and modify the copied file while leaving the original intact. After done modifying the file, refer to it (full path) in <mappingConfiguration> element in the relevant targetConfiguration.xml.

Parameter Description Values
<table> Some fields will appear in tables depending on the log format. This information can be found in the elg log - one entry for every new field. A field can appear in multiple tables, each distinct instance is considered as a new field.   
<exported></exported>  [optional] You can filter out specific fields by using the 'exported' true/false tag in the mapping configuration file. Alternatively, if the 'exportAllFields' tag in the 'targetConfiguration.xml' file is set to false, only those fields which are listed in the mapping file will be exported.
Note: this parameter can also be configured on a table field level to allow / prevent its export when it the field is part of a table.
 true \ false
<origName></origName>  The name of the field that will be mapped to <dstName>. refer to
Log Fields Mapping for Advanced Fields Configuration
 
 <dstName></dstName>  The new mapping scheme name for the desired field.  
<required></required> [optional] When set to 'true' only logs which contain this field will be exported.  true \ false

 

TLS Configuration

Show / Hide the section

How to set a secured connection between the log exporter and the syslog server

Log Exporter has the ability to export logs over an encrypted connection using TLS protocol.
When using TLS, it is important to know that only mutual authentication is allowed.
For mutual authentication Log Exporter needs the following certificates:

  • CA certificate that signed both client (Log Exporter side) and server (syslog server side) certificates. Required format: PEM.
  • Client (Log Exporter side) certificate. Required format: P12.
Note: CA server needs to be routable from Log Server in order to establish the connection.
Note: In addition to these 2 certificates, a third certificate should be installed on the syslog server side based on the server requirement.

It is also possible to use self-signed certificates.
The following procedure uses openssl commands that have to be run on non-Check Point server.

CA Cert Creation
  1. Create CA key:

    openssl genrsa -out ca.key 2048

  2. Create CA Cert:

    openssl req -x509 -new -nodes -key ca.key -days 2048 -out ca.pem

    You will be prompted to provide information regarding the certificate. Apart from the Common Name (it is recommended to use the device IP address as the Common Name), all other fields are optional and can be skipped. If you are purchasing an SSL certificate from a certificate authority, it is often required that these additional fields.

Log Exporter Cert Creation
  1. Create Log Exporter key:

    openssl genrsa -out cp_client.key 2048

  2. Create Log Exporter CSR file:

    openssl req -new -key cp_client.key -out cp_client.csr

  3. Create Log Exporter CRT file:

    openssl x509 -req -in cp_client.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out cp_client.crt -days 2048 -sha256

  4. Create Log Exporter P12 file:

    openssl pkcs12 -inkey cp_client.key -in cp_client.crt -export -out cp_client.p12


Syslog Server Cert Creation
  1. Create Taregt Server key:

    openssl genrsa -out server.key 2048

  2. Create Taregt Server CSR file:

    openssl req -new -key server.key -out server.csr

  3. Create Taregt Server CRT file:

    openssl x509 -req -in server.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out server.crt -days 2048 -sha256

Note: several SIEM applications require the Server certification to be in a specific format. For more information, refer to SIEM Specific Instructions section.

 

Filter Configuration

Choose what to export based on field values.

Show / Hide the section

Log Exporter has the ability to filter logs based on field values.
Since fields mapping operation is done before filtering, make sure to use the dst name / value of the fields if configured specifically in *FieldsMapping.xml file (based on the format). Otherwise, use the name / value as written in the raw log file.
It is possible to configure what to export or what not to export.
The filter configuration file is located under each deployment folder: $EXPORTERDIR/targets/<deployment-name>/conf/FilterConfiguration.xml

Parameter Description Possible/Default Values
<filterGroup operator=""></filterGroup>

 A group of fields that will determine what to export.

The relation between the fields is determined by the operator value.

refer to:

Log Fields Mapping for Advanced Fields Configuration

 

operator[and / or]

<field name="" operator=""><value operation=""></value>

</field>

 Declare a single field filter that will participate in the filter group.

name: The name of the field to filter on.

operator: Declares the operator (and \ or) between the various declared operations.

operation: Declares the matching logic regarding the declared value.

value: the specific value to filter on. Multiple values for a single operation is supported and should be added as a separate row.

 

operator[and / or]

operation[eq - equal / neq  - not equal /gt - greater than  / lt - less than ] 


There are 2 ways to configure filtering feature:

  1. Using cp_og_export command

    This command allows you to configure filtering for action / blade / origin fields only.
    The syntax for cp_log_export usages is:
       cp_log_export set name <name> filter-action-in "value1,value2"
       cp_log_export set name <name> filter-origin-in "value1,value2"
       cp_log_export set name <name> filter-blade-in "value2"

    In addition, it is possible to use predefined families for "filter-blade-in" value:

    • TP for exporting Threat Prevention logs only (Anti-Bot,Anti Malware,Threat Emulation,IPS,IPS-1,SmartDefense,Anti-Virus,New Anti Virus,Anti-Spam and Email Security,Threat Extraction,MTA).
    • Access for exporting Access logs only (Security Gateway/Management,VPN-1 & FireWall-1,Firewall,Application Control,URL Filtering,Content Awareness,Connectra,Mobile Access,Compliance blade,Core,DDoS Protector,Identity Awareness,Identity Logging,UA WebAccess).
    • Mobile for exporting Mobile logs only (WIFI Network,Mobile App,OS Exploits,Device,Network Security,Cellular Network,Network Access,iOS Profiles,Text Message,On-device Network Protection).
    • EndPoint for exporting Endpoint logs only (Anti-Bot,Anti Malware,Threat Emulation,IPS,IPS-1,SmartDefense,Anti-Virus,New Anti Virus,Anti-Spam and Email Security,Threat Extraction,MTA ).
  2. Modifying FilterConfiguration.xml file manually

    It is allowed to add new fields to this file - for example:

    <filters>
            <filterGroup operator="and">
                    <field name="action" operator="and">
                    </field>
                    <field name="origin" operator="and">
                    </field>
                    <field name="product" operator="or">
                    </field>
                    <field name="severity" operator="or">
                            <value operation="eq">3</value>
                            <value operation="eq">4</value>
                    </field>
            </filterGroup>
    </filters>

Limitations:

  • The relation between the values of the same operation is only OR.

    Example:

    cp_log_export set name <target-name> filter-action-in "accept,drop"

    Only logs with action = "accept" OR action= "drop" will be exported.

  • Filtering is not supported for any of the following fields : time, category, UUID,appi_name ,app_desc, app_category, matched_category, app_properties, app_rule_name, app_risk, HTTPS_inspection_rule_name, cvpn_resource, cvpn_category, name, desc, properties.

  • Filtering on a certain field with the condition: "not equal(value1) OR not equal(value2)" is not supported. When editing the filtering XML make sure to have a maximum of one line of "neq" operation in each field.

 

Log Fields Mapping for Advanced Fields Configuration

Show / Hide the section

Check Point Infinity solution includes multiple log fields, representing the diversity of Check Point's products.
The log fields mapping will help you understand security threats, logs language, complex queries and SIEM.

For information on Check Point's Log Fields Mapping, refer to: sk144192.

 

SIEM Specific instruction

How to configure SIEM applications to optimally receive logs.

Show / Hide the section

Rsyslog

Rsyslog is not configured to use RFC5424 timestamp format by default, therefore you should manually change Rsyslog setting for it to be compliant with Log Exporter output format.

On the syslog server:

1. Open /etc/rsyslog.conf

2. If there’s an uncommented line: “$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat”, comment it.

3. Add the following line: $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format

4. Save and close the file.

5.Restart Rsyslog:
  service rsyslog restart

 

ArcSight

It is recommended by ArcSight to name the server certificate as 'syslog-ng'

1. Convert the key to p12 format:
    openssl pkcs12 -inkey syslogServer.key -in syslogServer.crt -export -out syslog-ng.p12 -name "syslogng-alias" -password pass:changeit

2. make sure the environment variable ARCSIGHT_HOME to be the connector install directory:

 

  • Run the certificates manager on the Linux KDE console: ARCSIGHT_HOME/current/bin/arcsight agent keytoolgui
  • From the File menu open the keystore: $ARCSIGHT_HOME/current/jre/lib/security/cacerts (password "changeit").
  • From the menu select "Import Trusted Certificate"
  • From the file dialog, select ca.pem and save it.
  • Save and close the certificate manager.
  • Edit the agent.properties file to enable mutual authentication by:

 

         vi $ARCSIGHT_HOME//current/user/agent/agent.properties

         1. Change this value to True:

             syslogng.mutual.auth.enabled=false -> true

         2. Add these line to the bottom:

             syslogng.tls.keystore.file=user/agent/syslog-ng.p12

             syslogng.tls.keystore.alias=syslogng-alias

  • run /etc/init.d/arc_connector_name restart

 

Splunk

It is recommended to use Check Point App for Splunk when exporting logs to Splunk server.

For more information about installation and deployment, please see the Check Point App for Splunk User Guide.

In addition, in order to configure an encrypted connection, do the following:

1. Generate server PEM file according to Splunk TLS Documentation:
    cat syslogServer.crt syslogServer.key RootCA.pem > splunk.pem

2. Update the inputs.conf file on the Splunk server to use TLS:
    vi /opt/splunk/etc/apps/<the app where the configuration is saved>/local/inputs.conf

    [SSL]
    serverCert = <full path to server PEM file>
    sslPassword = <challenge password>
    requireClientCert = true

    [tcp-ssl://<port>]
    index = <index>

3. Update the server.conf file on the Splunk server to use the relevant CA PEM file:
    vi /opt/splunk/etc/system/local/server.conf

    [sslConfig]
    sslRootCAPath = <full path to CA PEM file>

    [SSL]
    cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH

4. Restart Splunk daemon:
    /opt/splunk/bin/splunk restart

QRadar

1. On the Authentication Mode field choose "TLS And Client Authentication"

    When using Client Authentication you need to provide the absolute path to the client certificate.

2. Upload the Check Point certificate and private key to QRadar and provide the absolute path to those under the "Provide Certificate" option.

 Note: Make sure the Common Name is different in every certificate.

 

Transition from LEA to Log exporter

Recommended method to move from the existing LEA connector to the new log exporter.

Show / Hide the section

1. Delete OPSEC application object from the GUI, if it is the only use for the OPSEC application, or alternatively remove the LEA client entity from it if it’s not :

2. If this is the only OPSEC LEA client ( otherwise skip this step) - Configure $FWDIR/conf/fwopsec.conf to not allow LEA:
    Comment out those lines:

From:

#

lea_server auth_port 18184

lea_server port 0

#

         ---->          

To:

#

# lea_server auth_port 18184

# lea_server port 0

#

3. Install the log exporter according to the installation guide above.

 

Transition from CPLogToSyslog to Log exporter

Recommended method to move from the existing CPLogToSyslog to the new log exporter.

Show / Hide the section

  1. Uninstall the CPLogToSyslog package using CPUSE. see instructions in sk92449 section 4-C
  2. Install the log exporter according to the installation guide above.

 

Troubleshooting

Show / Hide the section

Symptoms Cause Suggested solution
Logs are not exported after adding a filter to FilterConfiguration.xml or by using the cp_log_export command. cp_log_export adds the default values to FilterConfiguration.xml, while the field names should be same as the exported name. It causes the filter mechanism to not match any log.
  1. Go to the relevant XXXFieldsMapping.xml and look for the relevant mapped field.
  2. Find the element named <dstName> and copy it.
  3. Open <exporter-dir>/conf/FilterConfiguration.xml and replace the field name to the previously copied one.
Assigning a field in FieldsMapping file as export false, but keep exporting the field.
For example : When the user assign for the field layer_uuid export=false, but he keeps seeing this field as part of the log in the log Server.
The field is part of a table in the log, and the standard configuration to filter out field is not effective on a table field. In order to prevent these fields from been exported, you need to:
    1. Go to $EXPORTERDIR/targets/<exporter_name>/conf/
    2. Modify the 'Fields Mapping' file you use (in accordance with the format you export), look for the following tag:
      <tableName>match_table</tableName>
        1. If not exist, add inside 'fields' tag the following lines:

      <table>
        <tableName>match_table</tableName>
         <fields>
          <field><origName>field_name</origName><exported>false</exported></field>
         </fields>
      </table>


        1. If exist, add inside the match_table 'fields' tag the following tag:

            
      <field><origName>field_name</origName><exported>false</exported></field>

Output should be like this:
<fields>
<!-- Filter out fields -->
  <field><origName>field_name1</origName><exported>false</exported></field>
  <field><origName>field_name2</origName><exported>false</exported></field>
  <field><origName>field_name3</origName><exported>false</exported></field>
    .
    .
    .
    <table><tableName>match_table</tableName>
      <fields>
       <field>... </field>
       .
       .
       .
        <field><origName>field_name</origName><exported>false</exported></field>
      </fields>
    </table>

<!-- End of filter out -->
</fields>


Save the changes, and restart the exporter by running cp_log_export restart name <exporter_name> in order to load the new settings.

 

Known Limitations

Show / Hide the section

  • If you want to change this environment, you must first consult with your Check Point partner or vendor. This release is built for the specific environment. Upgrades or other changes can overwrite Hotfix functionality and environment customizations.
  • When exporting logs from a certain domain using UDP protocol, the exported log's header will contain the MDS IP and not the CMA/CLM's IP.

 

Appendix

Show / Hide the section

Special log fields

loguid - Some checkpoint logs are updated over time. Update logs will have the same loguid value. Checkpoint SmartLog client will correlate those updates into a single unified log.

When sending update logs to 3rd party (SIEM) servers, in the default raw read-mode, they will arrive as distinct logs. Best use the semi-unified read-mode, which will send few instances of the log, but each instance will contain the entire event chain until this update. Admins can alternatively use the loguid field to correlate update logs and get the full event chain themselves.

Note: All related log & log-updates will share the same initial time as the 1st log (in semi-unified mode).


Example of update logs includes the total amount of bytes sent and received over time or the severity field which will be updated over time as more information becomes available.

hll_key - hll_key stands for High-Level Log key. This concept was introduced in R80.10, where Multiple connection logs can comprise one session with one shared hll_key. For example, when browsing a webpage you might have multiple connection logs which are related to the same session. Connection logs which are part of the same session will share the same hll_key value.

Syslog-NG Listener configuration

When configuring a source on a Syslog NG server it is recommended to use the syslog-protocol flag.

For example: source s_network { network(transport(“tcp”) port(514) flags(syslog-protocol) ); };

 

Splunk Listener configuration

It is recommended to add these time settings to your sourcetype:

TIME_FORMAT = %s
TIME_PREFIX = time=
MAX_TIMESTAMP_LOOKAHEAD = 15

 

ArcSight Listener configuration

The Log Exporter solution doesn't work with the OPSEC LEA connector. Instead, you must install ArcSight Syslog-NG connector.

 

ArcSight Common Event Format (CEF) Mapping

CEF is an extensible, text-based format designed to support multiple device types by offering the most relevant information. Message syntaxes are reduced to work with ESM normalization. Specifically, CEF defines a syntax for log records comprised of a standard header and a variable extension, formatted as key-value pairs. The CEF format can be used with on-premise devices by implementing the ArcSight Syslog SmartConnector. CEF can also be used by cloud-based service providers by implementing the SmartConnector for ArcSight Common Event Format REST.

 

CEF Header format

  Version Device Vendor Device Product Device Version Device Event Class ID Name Severity
Default CEF:0 Check Point Log Update Check Point  Log  Log 
Values - Product Name (Blade)

Attack Name

Protection Type

Verdict

Matched Category

DLP Data Type

Application Category

Application Properties

 Protection Name

Application Name

Message Info

Service ID

Service

Application Risk

Risk

Severity

 

QRadar Log Event Extended Format (LEEF) Mapping

The Log Event Extended Format (LEEF) is a customized event format for IBM Security QRadar.

LEEF Header format

  LEEF:Version Vendor Product Version EventID
Default LEEF:2.0 Check Point Log Update 1.0 Check Point Log
Values - - Product Name (Blade) -

Protection Name

Application Name

Action

Note: The time format is not compliant with the official LEEF format.

Until such a time as IBM will add support for epoch time format Log Exporter with LEEF format is only partially supported.

 

Change Log

Show / Hide the section

Change Log

Note: Takes listed in the table below are related to the Log Exporter take and not the Jumbo take.
 
Mar 17, 2021 SK was updated with all existing functionalities and better descriptions
Oct 13, 2020 Updated Known Limitation section
Aug 19, 2020 Updated the Installation section
Jul 06, 2020 Edited R80.30 installation section
Jun 09, 2020 Changed note to: Audit logs exist on every Log Server
Apr 13, 2020 Updated Limitation for Filtering Configuration section
Feb 19, 2020 Updated to Check_Point_R80.20_JHF_T118_Log_Exporter_Enhancements_T5_sk122323_FULL.tgz which fixed an issue with ds.conf when installing on top of R80.20 Jumbo HF Take_118.
Jan 19, 2020 Added scenario to Troubleshooting section
Jan 1, 2020 Added LogRhythm format for R80.20 on top of JHF T118 and R80.30 on top of JHF T111.
Dec 12, 2019 Added Troubleshooting section.
Oct 2, 2019 Added SmartView links For R80.20 on top of Jumbo Hotfix Accumulator Take_103
July 8, 2019

Added "Log Fields Mapping for Advanced Fields Configuration" section

Issue ID Description
take 51 (April 14, 2019)
SL-1822

Installation of R80_10_JHF_LOGOUT fails because of the script updateExistingExporters.sh when there are no exporters in the server.

take 50 (March 5, 2019)
SL-2003

Added filtering support. Now you can decide which logs to export.

Take 43 (January 20, 2019)
PMTR-13842

Added support for exporting logs to the new Check Point's Splunk application.

SL-1817

Log Exporter was getting stuck after 7 hours of uptime

SL-1932

Log Exporter could not be installed on top of R80.10 Jumbo HF Take_169 and above.

 

Related solutions:

 

Give us Feedback
Please rate this document
[1=Worst,5=Best]
Comment