• R80.20

Log Exporter is already integrated in R80.20. There is no need to install in it a dedicated package.

Note: In order to preserve the Log Exporter configuration before upgrading to R80.20, please follow sk127653 - How to backup and restore Log Exporter configuration on upgrade to R80.20

Note: In order to support exporting logs in Splunk format, please install R80.20 Jumbo Hotfix Take 5 and above. 

Note: Filtering ability has been integrated into Jumbo Hotfix Accumulator for R80.20 Take 103.

• R80.10

Install this release on a R80.10 Multi-Domain Server, Multi-Domain Log Server, Security Management Server, Log Server or SmartEvent Server.
Note: Log Exporter can be installed on top of R80.10 Jumbo Hotfix Take 56 and above.

Note: This hotfix must be installed after the Jumbo, and will need to be uninstalled to upgrade to a higher Jumbo take, and then reinstalled after the newer Jumbo is in place.

Note: Take care to install the latest Log Exporter take available for download below, in order to avoid a conflict with the Jumbo HF.

• R77.30

Install this release on a R77.30 Multi-Domain Server, Multi-Domain Log Server, Security Management Server, Log Server or SmartEvent Server.
Note: Log Exporter can be installed on top of R77.30 Jumbo Hotfix Take 292 and above. 

**This hotfix must be installed after the Jumbo, and will need to be uninstalled to upgrade to a higher Jumbo take, and then reinstalled after the newer Jumbo is in place. 

 

Version	Date	CPUSE Online Identifier	CPUSE offline package
R80.10	14 April 2019	Check_Point_R80.10_Log_Exporter_T51_sk122323_FULL.tgz	(TGZ)
R77.30	14 April 2019	Check_Point_R77.30_Log_Exporter_T36_sk122323_FULL.tgz	(TGZ)

Install the hotfix using CPUSE, see sk92449.

Management server - uninstall the feature via CPUSE, see sk92449.

Multi-Domain Security Management - Uninstalling the package does not remove the configuration files, in order to uninstall and completely remove all configurations, do the following:

• Run:

cp_log_export delete name all [domain-server all] --apply-now

apply-now false
Do not forget to add <domain-server all> on MDS/MLM machines


• If running on Multi-Domain Management Server: switch to MDS's environment by running: # mdsenv


• Make sure $EXPORTERDIR exists and is pointing to $RTDIR/log_exporter ($FWDIR/log_exporter on R77.30). Run:

# rm -rf $EXPORTERDIR


• Uninstall the hotfix using CPUSE, see sk92449.

Note: After uninstalling the hotfix with CPUSE, reboot the machine.

In order to configure a new target for the logs do the following on the log server:

cp_log_export add name <name> [domain-server <domain-server>] target-server <target-server IP address> target-port <target-port> protocol <(udp|tcp)> format <(syslog)|(cef)|(splunk)(generic)> [optional arguments]

 

• On MDS/MLM: domain-server argument is mandatory, you can use 'mds' as the value for domain-server in order to export mds level audit logs
• This will create a new target directory with the unique name specified in the name parameter under $EXPORTERDIR/targets/<deployment_name>, and set the target configuration parameters with the connection details: IP Address, port, protocol, format and read-mode.
• The recommended read-mode for splunk format is semi-unified which ensures you will get a complete data.
• Note the above deployment will export the logs in clear text.
• The new log exporter does not start automatically. To start it run: cp_log_export restart

If you wish to send the logs over an encrypted connection, please refer to the "TLS Configuration" section.

Usage

cp_log_export <command-name> [command-arguments]

In order to understand a specific command usage run:
cp_log_export <command-name> help

 

Commands

Command Name	Command Description
add	Deploy a new Check Point logs exporter.
set	Updates an existing exporter's configuration
delete	Removes an exporter.
show	Prints an exporter's current configuration.
status	Shows an exporter's overview status.
start	Starts an exporter process.
stop	Stops an exporter process.
restart	Restarts an exporter process.
reexport	Resets the current position, and re-exports all logs per the configuration.

Parameters

Parameter Name	Description	add	set	delete	show/status/start/stop/restart	reexport
name	Unique name of the exporter configuration.	Mandatory	Mandatory	Mandatory	Optional - Default all	Mandatory
domain-server	The relevant domain-server name or IP	Mandatory	Mandatory	Mandatory	Optional - Default all	Mandatory
target-server	Exporting the logs to this ip address	Mandatory	Optional	N/A	N/A	N/A
target-port	The port on which the target is listening to	Mandatory	Optional	N/A	N/A	N/A
protocol	Transport protocol to use	Mandatory	Optional	N/A	N/A	N/A
format	The format in which the logs will be exported	Optional	Optional	N/A	N/A	N/A
read-mode	The mode in which the log files will be read and exported	Optional	Optional	N/A	N/A	N/A
enabled	Allow log_exporter to start on cpstart/mdsstart	Optional	Optional	N/A	N/A	N/A
encrypted	Using TLS (SSL) encryption for exporting the logs	Optional	Optional	N/A	N/A	N/A
ca-cert	Full path to the CA pem certificate file Relevant only when encrypted is true	Optional	Optional	N/A	N/A	N/A
client-cert	Full path to the client p12 certificate file Relevant only when encrypted is true	Optional	Optional	N/A	N/A	N/A
client-secret	The challenge phrase that was used in order to create the client p12 certificate Relevant only when encrypted is true	Optional	Optional	N/A	N/A	N/A
filter-action-in	Exporting all logs with a specific action. value should be surrounded by "" and multiple values are supported separated by a comma.	Optional	Optional	N/A	N/A	N/A
filter-origin-in	Exporting all logs from a specific origin. value should be surrounded by "" and multiple values are supported separated by a comma.	Optional	Optional	N/A	N/A	N/A
filter-blade-in	Exporting all logs that belong to a specific blade. value should be surrounded by "" and multiple values are supported separated by a comma. Predefined blade families can be selected (TP, Access, Endpoint, Mobile).	Optional	Optional	N/A	N/A	N/A
--apply-now	Applying any change that was done in the add immediately	Optional	Optional	Mandatory	N/A	Mandatory

Note: Using filter-action-in\filter-origin-in\filter-blade-in will replace any other filter that was pre declared on these fields directly in the filtering XML. Other fields filters will not be overridden.

The only allowed authentication method via TLS is mutual authentication. For mutual authentication log exporter will need the following certificates:

• A pem Certificate Authority (CA) certificate (should contain only the certificate of CA that signed the client/server certificates, not the parent CA).
• A .p12 format client certificate.

If you do not already have the required certificates, you can follow the procedure below.

The following procedure is an example of creating the required certificates. You have alternative procedures for achieving this.

Important Note: The below openssl commands need to be run on a 3rd party CA server. (Not on the Log Exporter machine)
The commands are not supported on a Check Point Security Management server/MDM server.

Note: CA server needs to be routable from log exporter device.

Create Self Signed CA

Run this in case you do not already have a trusted CA pem:

 

1. Generate the root CA key - Do not pass to anyone:

openssl genrsa -out RootCA.key 2048


2. Generate the root CA pem:

openssl req -x509 -new -nodes -key RootCA.key -days 2048 -out RootCA.pem

You will be prompted to provide information regarding the certificate. This information is known as a Distinguished Name (DN). An important field in the DN is the Common Name(CN), which should be the exact Fully Qualified Domain Name (FQDN) of the host that you intend to use the certificate with. Apart from the Common Name all other fields are optional and can be skipped. If you are purchasing an SSL certificate from a certificate authority, it is often required that these additional fields, such as "Organization", accurately reflect your organization's details.

Here is an example of what the prompt will look like:

---

Country Name (2 letter code) [AU]:US

State or Province Name (full name) [Some-State]:New York

Locality Name (eg, city) []:MyCity

Organization Name (eg, company) [Internet Widgits Pty Ltd]:MyCompany

Organizational Unit Name (eg, section) []:MyDepartment

Common Name (e.g. server FQDN or YOUR name) []:www.company.com

Email Address []:

Note: It is recommended to use the device IP address as the Common Name.

Create Client (log_exporter) .p12 Certificate File

 

1. Generate client key - Do not pass to anyone:

openssl genrsa -out log_exporter.key 2048


2. Generate client certificate sign request:

openssl req -new -key log_exporter.key -out log_exporter.csr


3. Sign the certificate using the CA files:

openssl x509 -req -in log_exporter.csr -CA RootCA.pem -CAkey RootCA.key -CAcreateserial -out log_exporter.crt -days 2048 -sha256


4. Convert to p12 format:

openssl pkcs12 -inkey log_exporter.key -in log_exporter.crt -export -out log_exporter.p12

Note: The challenge phrase used in this conversion is required in the log_exporter TLS configuration.

 

Update the security parameters on the Check Point exporting server:

 

1. If running on MDS/MLM switch to the required domain by running:

mdsenv <domain server name or ip>


2. Go to the deployment directory:

cd $EXPORTERDIR/targets/<deployment name>


3. Create a directory for the certificates:

mkdir certs


4. Copy RootCA.pem, and log_exporter.p12 to certs directory.

   Important: RootCA.key should NOT be published.


5. Give the RootCA.pem and log_exporter.p12 read permissions:

chmod +r RootCA.pem
chmod +r log_exporter.p12


6. Update the secured target using:

cp_log_export set name <name> domain-server <domain-server> encrypted true ca-cert <path_to_CA_pem> client-cert <path_to_p12_certificate> client-secret <challenge_phrase_for _p12>

 

Create Server (target) Certificate

 

1. Generate server key - Do not pass to anyone:

openssl genrsa -out syslogServer.key 2048


2. Generate server certificate sign request:

openssl req -new -key syslogServer.key -out syslogServer.csr


3. Sign the certificate using the CA files:

openssl x509 -req -in syslogServer.csr -CA RootCA.pem -CAkey RootCA.key -CAcreateserial -out syslogServer.crt -days 2048 -sha256

 

After deploying a new instance of log exporter, all related files to that deployment can be found under $EXPORTERDIR/targets/<deployment name>

On an MDS/MLM server, EXPORTERDIR environment variable is per domain, and its value is changed automatically when switching between domain server contexts with the mdsenv command.

 

Target Configuration XML

The target configuration file, located under each deployment folder: $EXPORTERDIR/targets//targetConfiguration.xml

Note: You must restart the log exporter process for the new setting to take effect.

Listed below are some of the configuration options:

Parameter	Description	Possible/Default Values
<version></version>	Current Log Exporter version - used for upgrades.
<is_enabled></is_enabled>	Determines whether or not the process will be monitored by the watch dog.	true/false

Destination Parameters

Parameter	Description	Possible/Default Values
type	Reserved for future use.
<ip></ip>	The IP address of the target that will receive the logs	Any IPv4 address.
<port></port>	The port on which the target is listening to.	Any valid port number.
<protocol></protocol>	The protocol that will be used in the connection.	UDP / TCP

Security Parameters

Discussed in more detail in the "TLS Configuration" section.

Parameter	Description	Possible/Default Values
<security></security>	Determines weather or not the connection will be sent in clear text or encrypted.	clear [default] / tls
<pem_ca_file></pem_ca_file>	The location of the root CA pem file.
<p12_certificate_file></p12_certificate_file>	The location of the client key pair in p12 format.
<client_certificate_challenge_phrase></client_certificate_challenge_phrase>	The challenge phrase that was used in order to create the p12 certificate. Will be hashed after after restarting the process .

Source Parameters

Parameter	Description	Possible/Default Values
<folder></folder>	The path where the log files are located.	Default location is $FWDIR/log/.
<log_files></log_files>	Determines which log files will be exported & how far back.	read logs from [number] - (default=1) days back (recommended) | <specific file name> | on-line (no value=on-line)
<log_types></log_types>	Determines which log files (by extension) will be exported.	All [default] / log / audit
<read_mode></read_mode>	Determines how the log files will be read.	raw [default] | semi-unified (recommended for splunk format)

Resolver Parameters

Parameter	Description	Possible/Default Values
<mappingConfiguration></mappingConfiguration>	The XML file containing the log field mapping scheme. If left empty will use the default settings.	Default values are based on the 'format'.
<exportAllFields>true</exportAllFields>	When this field is set to 'true' all log fields will be sent regardless of whether or not they appear in the mapping scheme, , except for specifically black-listed fields in the relevant log format mapping file (<exported>false</exported>). When set to 'false' only those fields which appear in the relevant log format mapping file will be sent (with exported flag true: <exported>true</exported>)	true / false

Format Parameters

Parameter	Description	Possible/Default Values
<formatHeaderFile></formatHeaderFile>	The XML file contains the log header format scheme. If left empty will use the default settings.	Default values are based on the 'format'.

General Filter Configuration Path

Parameter	Description	Possible/Default Values
<dynamicFilter></dynamicFilter>	The XML file containing the filtering configuration. If left empty, default Configuration will be used.	Default path is conf/FilterConfiguration.xml

Filter out firewall connections Parameters

The Log Exporter solution supports several filtering options, as detailed in the section above. In this section, we will go over each option.

Filters logs based on blade

In the current release, we have a limited blade related filtering. This functionality will be expanded upon in future releases.

You can filter out firewall connection logs ('Firewall-1 & VPN-1', 'HTTPS Inspection' and 'Security Gateway/Management').

 

Parameter	Description	Possible/Default Values
<filter filter_out_by_connection="false">	Determines if the Access logs should be filtered out. When set to 'true' VPN-1 & Firewall-1 connection logs will be filtered out Note: No other blade filters are currently supported. This will be expanded upon in future releases.	true / false

Note: Firewall session logs will still be exported (Generated by tracking a firewall rule by per Session).  

          Limitation: HTTPS inspection logs, Non-rulebase generated Firewall logs & a few Firewall NAT update logs will still be exported.

 

Format Configuration XML

Body

Parameter	Description	Syslog	Splunk	CEF	LEEF	Generic
<start_message_body></start_message_body>	The character preceeding the log data payload.	[
<end_message_body></end_message_body>	The character following the log data payload.	]
<message_separator></message_separator>	The delimeter that separates logs.	&#10; (&#10;=='\n')	&#10; ('\n')	&#10; ('\n')	&#10; ('\n')	('\n')
<fields_separatator></fields_separatator>	The delimeter that separates log fields.	'; ' (semi colon, space)	| (pipe)	' ' (space)	&#09; (<TAB>)	' ' (space)
<field_value_separatator></field_value_separatator>	The assignment operator.	:	=	=	=	=
<value_encapsulation_start>&quot;</value_encapsulation_start>	The value encapsulation operator (start)	"			"	"
<value_encapsulation_end>&quot;</value_encapsulation_end>	The value encapsulation operator (end)	"			"	"
<escape_chars> <char> <orig></orig> <escaped></escaped> </char> </escape_chars>	Escaping unwanted characters. The escape functionality will replace the string that's encapsulated by the 'orig' tags with the one encapsulated by the 'escaped' tags	\ --> \\ " --> \" &#10; --> ' ' ] --> \]	| --> ; = --> \= &#10; --> ' '	\ --> \\ = --> \= &#10; --> ' ' | --> \|	= --> \= &#10; --> ' '	\ --> \\ " --> ' &#10; --> ' '

 

Header

Parameter	Description	Default values for syslog	Default values for splunk	Default values for CEF
<header_format></header_format>	The delimeter between the header values and the number of values. Every {} will be replaced with one value.	' ' (space)	|	|

 

Field Mapping Configuration XML

Parameter	Description	Values
<table>	Some fields will appear in tables depending on the log format. This information can be found in the elg log - one entry for every new field. A field can appear in multiple tables, each distinct instance is considered as a new field.
<exported></exported>	[optional] You can filter out specific fields by using the 'exported' true/false tag in the mapping configuration file. Alternatively, if the 'exportAllFields' tag in the 'targetConfiguration.xml' file is set to false, only those fields which are listed in the mapping file will be exported..	true \ false
<origName></origName>	The name of the field that will be mapped to <dstName>. refer to Log Fields Mapping for Advance Fields Configuration
<dstName></dstName>	The new mapping scheme name for the desired field.
<required></required>	[optional] When set to 'true' only logs which contain this field will be exported.	true \ false

The filter configuration file is located under each target folder: $EXPORTERDIR/targets/<target-name>/conf/FilterConfiguration.xml

Parameter	Description	Possible/Default Values
<filterGroup operator=""></filterGroup>	A group of fields that will determine what to export. The relation between the fields is determined by the operator value. refer to: Log Fields Mapping for Advance Fields Configuration	operator[and / or]
<field name="" operator=""><value operation=""></value> </field>	Declare a single field filter that will participate in the filter group. name: The name of the field to filter on. operator: Declares the operator (and \ or) between the various declared operations. operation: Declares the matching logic regarding the declared value. value: the specific value to filter on. Multiple values for a single operation is supported and should be added as a separate row.	operator[and / or] operation[eq - equal / neq  - not equal /gt - greater than  / lt - less than ]

Filtering Logic:

• The filtering feature allows to decide which logs will be exported based on values from the various fields on the raw log.

for example, export all logs from a specific origin (Security Gateway), or export all Threat prevention logs, or export all logs that are not from a specific source.


• The filtering can be declared both from cp_log_export command (only for action, blade, origin), or by editing the filtering configuration file manually.

Example for cp_log_export usages:

   cp_log_export set name <name> filter-action-in "value1,value2"
   cp_log_export set name <name> filter-origin-in "value1,value2"
   cp_log_export set name <name> filter-blade-in "value2"

Example of filter by raw field names of severity:"High" or "Critical" & blade/product:"IPS" or "Threat Emulation":

<filters>
        <filterGroup operator="and">
                <field name="action" operator="and">
                </field>
                <field name="origin" operator="and">
                </field>
                <field name="product" operator="or">
                        <value operation="eq">SmartDefense</value>
                        <value operation="eq">Threat Emulation</value>
                </field>
                <field name="severity" operator="or">
                        <value operation="eq">3</value>
                        <value operation="eq">4</value>
                </field>
        </filterGroup>
</filters>

Example of filter by mapped field names of cp_severity: "High" or "Critical" & blade/product: "IPS" or "Threat Emulation", for CEF format (severity -> cp_severity):

<filters>
        <filterGroup operator="and">
                <field name="action" operator="and">
                </field>
                <field name="origin" operator="and">
                </field>
                <field name="product" operator="or">
                        <value operation="eq">SmartDefense</value>
                        <value operation="eq">Threat Emulation</value>
                </field>
                <field name="cp_severity" operator="or">
                        <value operation="eq">3</value>
                        <value operation="eq">4</value>
                </field>
        </filterGroup>
</filters>

   * The name of the field to filter on should be the mapped name in case it is changed in the mapping XML, or the original raw name if it is not mapped.

   * The value of the field to filter on should always be the raw field values.

The predefined families for "product" field (filter-blade-in) are :

• TP for exporting Threat Prevention logs only (Anti-Bot,Anti Malware,Threat Emulation,IPS,IPS-1,SmartDefense,MTA,Anti-Virus,New Anti Virus,Anti Virus,Anti-Spam and Email Security,Threat Extraction,MTA).


• Access for exporting Access logs only (Security Gateway/Management,VPN-1 & FireWall-1,Firewall,Application Control,URL Filtering,Content Awareness,Connectra,Mobile Access,Compliance blade,Core,DDoS Protector,Identity Awareness,Identity Logging,UA WebAccess).


• Mobile for exporting Mobile logs only (WIFI Network,Mobile App,OS Exploits,Device,Network Security,Cellular Network,Network Access,iOS Profiles,Text Message,On-device Network Protection).


• EndPoint for exporting Endpoint logs only (Anti-Bot,Anti Malware,Threat Emulation,IPS,IPS-1,SmartDefense,MTA,Anti-Virus,New Anti Virus,Anti Virus,Anti-Spam and Email Security,Threat Extraction,MTA ).

 

Limitations:

• The relation between the values of the same operation is only OR.

Example:

cp_log_export set name <target-name> filter-action-in "accept,drop"

Only logs with action = "accept" OR action= "drop" will be exported.


• Filtering is not supported for any of the following fields : time, category, UUID,appi_name ,app_desc, app_category, matched_category, app_properties, app_rule_name, HTTPS_inspection_rule_name, cvpn_resource, cvpn_category, name, desc, properties.
  
  
• Filtering on a certain field with the condition: "not equal(value1) OR not equal(value2)" is not supported. When editing the filtering XML make sure to have a maximum of one line of "neq" operation in each field .

Check Point Infinity solution includes multiple log fields, representing the diversity of Check Point's products.
The log fields mapping will help you understand security threats, logs language, complex queries and SIEM.

For information on Check Point's Log Fields Mapping, refer to: sk144192.

Rsyslog

Rsyslog is not configured to use RFC5424 timestamp format by default, therefore you should manually change Rsyslog setting for it to be compliant with Log Exporter output format.

On the syslog server:

1. Open /etc/rsyslog.conf

2. If there’s an uncommented line: “$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat”, comment it.

3. Add the following line: $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format

4. Save and close the file.

5.Restart Rsyslog:
  service rsyslog restart

 

ArcSight

It is recommended by ArcSight to name the certificate 'syslog-ng'

1. Convert the key to p12 format:
    openssl pkcs12 -inkey syslogServer.key -in syslogServer.crt -export -out syslog-ng.p12 -name "syslogng-alias" -password pass:changeit

2. make sure the environment variable ARCSIGHT_HOME to be the connector install directory:

 

• Run the certificates manager on the Linux KDE console: ARCSIGHT_HOME/current/bin/arcsight agent keytoolgui
• From the File menu open the keystore: $ARCSIGHT_HOME/current/jre/lib/security/cacerts (password "changeit").
• From the menu select "Import Trusted Certificate"
• From the file dialog, select ca.pem and save it.
• Save and close the certificate manager.
• Edit the agent.properties file to enable mutual authentication by:

 

         vi $ARCSIGHT_HOME//current/user/agent/agent.properties

         1. Change this value to True:

             syslogng.mutual.auth.enabled=false -> true

         2. Add these line to the bottom:

             syslogng.tls.keystore.file=user/agent/syslog-ng.p12

             syslogng.tls.keystore.alias=syslogng-alias

• run /etc/init.d/arc_connector_name restart

 

Splunk

It is recommended to use Check Point App for Splunk when exporting logs to Splunk server.

For more information about installation and deployment, please see the Check Point App for Splunk User Guide.

In addition, in order to configure an encrypted connection, do the following:

1. Generate server pem file:
    cat syslogServer.crt syslogServer.key RootCA.pem > splunk.pem

2. Update the inputs.conf file on the Splunk server
    vi /opt/splunk/etc/apps/search/local/inputs.conf

    [SSL]
    serverCert = /etc/ssl/my-certs/splunk.pem
    sslPassword = <challenge password>
    requireClientCert = true

    [tcp-ssl://<port>]
    index = <index>

3. Update the server.conf file on the Splunk server
    vi /opt/splunk/etc/system/local/server.conf

    [sslConfig]
    sslRootCAPath = /etc/ssl/my-certs/RootCA.pem

4. Restart Splunk
    /opt/splunk/bin/splunk restart

 

QRadar

1. On the Authentication Mode field choose "TLS And Client Authentication"

    When using Client Authentication you need to provide the absolute path to the client certificate.

2. Upload the Check Point certificate and private key to QRadar and provide the absolute path to those under the "Provide Certificate" option.

1. Delete OPSEC application object from the GUI, if it is the only use for the OPSEC application, or alternatively remove the LEA client entity from it if it’s not :

2. If this is the only OPSEC LEA client ( otherwise skip this step) - Configure $FWDIR/conf/fwopsec.conf to not allow LEA:
    Comment out those lines:

From: # lea_server auth_port 18184 lea_server port 0 #

---->

To: # # lea_server auth_port 18184 # lea_server port 0 #

3. Install the log exporter according to the installation guide above.

1. Uninstall the CPLogToSyslog package using CPUSE. see instructions in sk92449 section 4-C
2. Install the log exporter according to the installation guide above.

If you want to change this environment, you must first consult with your Check Point partner or vendor. This release is built for the specific environment. Upgrades or other changes can overwrite Hotfix functionality and environment customizations.

Show / Hide the section

Show / Hide the section