Support Center > Search Results > SecureKnowledge Details
SmartLog does not index logs that existed prior to SmartLog installation
Symptoms
  • SmartLog does not index logs that existed prior to SmartLog installation.
Cause

The time-stamp in the SmartLog settings file is marked to index only from the point of its installation.


Solution

Note: This article does not apply to R80.x. R80 index backwards until it reach the SmartConsole backwards index limit (the default is 14 days).  For R80.x follow sk111766

Background

SmartLog parameters that are relevant to this issue can be configured in the following file on Security Management Server:

  • in pre-R76:
    $SMARTLOGDIR/conf/smartlog_settings.txt

  • in R76 - R77.x:
    $SMARTLOGDIR/smartlog_settings.txt

    Note: In R76 - R77.x, SmartLog parameters are located in two files:
    • $SMARTLOGDIR/conf/smartlog_settings.conf (replaced during upgrades)
    • $SMARTLOGDIR/smartlog_settings.txt

 

Configuring the start date/time for logs indexing

Follow these steps on Security Management Server:

  1. Backup the current file:

    • in pre-R76:
      [Expert@HostName]# cp $SMARTLOGDIR/conf/smartlog_settings.txt  $SMARTLOGDIR/conf/smartlog_settings.txt_ORIGINAL

    • in R76 - R77.x:
      [Expert@HostName]# cp $SMARTLOGDIR/smartlog_settings.txt  $SMARTLOGDIR/smartlog_settings.txt_ORIGINAL


  2. Edit the current file:

    • in pre-R76:
      [Expert@HostName]# vi $SMARTLOGDIR/conf/smartlog_settings.txt

    • in R76 - R77.x:
      [Expert@HostName]# vi $SMARTLOGDIR/smartlog_settings.txt


  3. Delete these lines:

    time_restriction_for_fetch_all (<existing_data>)
    time_restriction_for_fetch_all_disp (<existing_data>)

  4. Backup and edit $SMARTLOGDIR/conf/smartlog_settings.conf and change the number of days of logs to re-index:

    [Expert@HostName]# cp $SMARTLOGDIR/conf/smartlog_settings.conf $SMARTLOGDIR/conf/smartlog_settings.conf_ORIGINAL
    [Expert@HostName]# vi $SMARTLOGDIR/conf/smartlog_settings.conf

:num_days_restriction_for_fetch_all (<days>)
:num_days_restriction_for_fetch_all_integrated (<days>)

<days> is the last number of days of logs to be indexed by the SmartLog server. For example, to re-index logs from the last 30 days of logs, give a value of 30.

Note - To reduce the performance impact while re-indexing, we recommend that you import only the number of days of logs that you need.



  1. Backup and remove $SMARTLOGDIR/data/FetchedFiles:

    [Expert@HostName]# cp $SMARTLOGDIR/data/FetchedFiles $SMARTLOGDIR/data/FetchedFiles_ORIGINAL
    [Expert@HostName]# rm -i $SMARTLOGDIR/data/FetchedFiles
     
  2. Restart SmartLog services:

    [Expert@HostName]# smartlogstop
    [Expert@HostName]# smartlogstart

  3. Verify smartlog_settings.txt shows the earliest date to re-index.

    • in pre-R76:
      [Expert@HostName]# grep restriction $SMARTLOGDIR/conf/smartlog_settings.txt

    • in R76 - R77.x:
      [Expert@HostName]# grep restriction $SMARTLOGDIR/smartlog_settings.txt

 

Notes for Multi-Domain Management:

  • The same configuration steps should be implemented in the context of each CMA.

  • Manually navigate to each CMA's Smartlog directory, as $SMARTLOGDIR does not currently work in a Multi-Domain Management / Provider-1 environment.
    For SecurePlatform the $SMARTLOGDIR is /var/opt/CPmds-R77/customers/<CMA name>/CPSmartLog-R77/
    For other OS: /var/log/opt/CPmds-R77/customers/<CMA name>/CPSmartLog-R77/

 


Give us Feedback
Please rate this document
[1=Worst,5=Best]
Comment