How to generate a log parser for third party syslog server
Generating the Syslog Parser
Note - If not done properly, this could prevent the syslog process from running, so follow the below instructions carefully.
To generate the Syslog parser:
- Install the Eventia Log Parsing Editor tool on your Windows machine. This tool allows you to easily define Parsing files for third party syslogs based on syslog samples.
- Open the Eventia Log Parsing Editor application.
- Read syslog samples from a file or the clipboard. They will then be analyzed by the Syslog Parser GUI.
Log samples are analyzed and combined into patterns. Many text sections that are recognized as variable (dates, numbers, IP addresses) are automatically detected.
- Define variable sections which will add data to the log fields and will also help combine these patterns. To manually define the sections, highlight them with the mouse and then click Define Highlighted Text to bring up the Selection Definition window.
After selecting the field, the field type and its regular expression pattern is automatically identified. You can associate it with a log field. In our example, the log identifies a type of alert so you associate it with a new log field "AlertType".
- Some fields (like action or protocol) may have different names in the syslogs than is normally expected in the logs. For instance, our log contains the values "permitted" and "denied" instead of the standard CheckPoint values "accept" and "reject". A dictionary maps these syslog values into expected values.
The dictionary editor allows you to add new mappings from the syslog value to the log server value. Most common dictionaries are already defined.
- This is how it should look like after defining the all the relevant sections:
- The product field is special since it is key to interpreting a log's values. There is a section on the bottom of the screen to define the product. If the logs contain identifying text, then it is used for product identification. Otherwise, the fact that you have successfully parsed a log is sufficient to identify the product.
- Click "Simulate Current Parsing File" to simulate the results with a new sample file to verify that your definitions are correct. Unsuccessfully parsed samples can be added back to the pattern list to improve parsing.
- Click "Generate" to generate the set of parsing files that will be copied to the Eventia log server machine. The files that will be generated are ParserName.C and ParserName-dict.ini (when dictionaries are used), where ParserName is the name you choose for the parser file.
- Save your project in a project file (Menu > Save). This way you can modify the definitions in the future.
Note: It is not possible to create projects from existing parsing files automatically, so keep this valuable information.
Installing the Syslog parser on the relevant Log server:
Once the Syslog parsing file is created, deploy it in the Log Server by following these instructions:
- To install or remove parser files and dictionary files on the log server, you must first install the Log Server File Utility on the log server.
The Log Server File Utility is supplied as a file called addParsingFile, and comes in different versions according to the Log Server's operating system. Take the utility file from Eventia Log Parsing Editor's installation directory, under the directory named like your Log Server's operating system. For example, if your Log Server runs on Windows Server, and you installed Eventia Log Parsing Editor in the default location, take:
C:\Program Files\Eventia Log Parsing Editor\Windows\addParsingFile.exe
To install it on the log server, save it on the log server.
If the log server is on a LINUX / UNIX platform, such as SecurePlatform OS / Solaris OS, save to:
If the log server is on a Windows platform, save to:
- Install the Parsing and Dictionary files on your log server:
- Copy the parsing file, and, if relevant, the dictionary file, to the log server.
- Run: addParsingFile -p <ParsingFile> [-d <DictionaryFile>]
- Activate Syslog parsing on the Log Server:
- Open SmartDashboard
- Edit the Log Server network object > Additional Logging Configuration tab
- Check the "Accept Syslog messages" checkbox
- Install Database on the Log Server (if it's separate from Security Management)
- Restart the Log Server:
- In R80.10, need to restart syslogd daemon on the log server manually.
Verifying that the Syslog parser works properly
The files are now installed on the log server, and received syslogs will be parsed and translated according to the files' instructions. Check in SmartLog or SmartView Tracker that the syslog entries are indeed being parsed properly to the desired fields.