This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Installing Log Forwarder on Linux and Windows

The Log Forwarder is the log processing tool that collects the logs from the protectors and forwards them to the Audit Store.

1: Configuring the disk space on the Log Forwarder

The following section describes the steps to install the Log Forwarder on Linux or Windows.

Installing Log Forwarder on Linux

The following section describes the steps to install the Log Forwarder on a Linux platform using the Interactive or Silent mode of installation.

By default, the Log Forwarder is installed in the /opt/protegrity directory. You can choose to install the Log Forwarder in a different directory by specifying the -d or --dir argument in the installation command. If you change the base installation directory, then the installation path will also change accordingly.

Use the --help argument if you need any help with installing the Log Forwader, as shown in the following command.

./LogforwarderSetup_Linux_x64_<version>.sh --help

The following snippet displays the output.

Install:

  ./LogforwarderSetup_Linux_x64_<version>.sh -e <host[:port]...> [-d <dir>]

Options:
  -e, --endpoint    Host of the target audit store endpoint(s).
                    Repeat this option to specify multiple endpoints to balance the load on audit store endpoints.
                    Each endpoint may specify a port. When no port is specified, 9200 is used.
  -d, --dir         Path to base directory for installation (default: /opt/protegrity)

Installing Log Forwarder on Linux using Interactive Mode

To preserve all the configurations while upgrading the Log Forwarder, ensure that you backup all the files present under the /opt/protegrity/logforwarder/data/config.d directory.

To install the Log Forwarder on a Linux platform using the Interactive mode:

Run the Log Forwarder installer using the following command.

./LogforwarderSetup_Linux_x64_<version>.sh

The prompt to enter the Audit Store endpoint appears.

Enter the audit store endpoint (host),
alternative (host:port) to use another port than the default port 9200 :

Enter the Audit Store endpoint that is the Audit Store IP address and the Audit Store port number where the Log Forwarder sends the logs. The default port number is 9200. If you are using the default port, then do not specify the port number.
Press ENTER.
The added Audit Store endpoint appears on the screen.
The prompt to enter an additional Audit Store appears.
```
Do you want to add another audit store endpoint? [y/n]:
```
If you want to add more than one Audit Store endpoint, then type y otherwise type n. If you need to add additional Audit Store endpoints, then repeat both Step 2 and Step 3 for each additional endpoint to add.
Type the y key to install into the destination directory.
The Log Forwarder is installed in the /opt/protegrity/logforwarder/ directory.
Start the Protegrity Log Forwarder service by using the following command.
```
/opt/protegrity/logforwarder/bin/logforwarderctrl start
```
The Log Forwarder is successfully installed.

If you want to modify the number of Audit Stores, then perform the following steps after the installation completes.

i. Edit the upstream.cfg file to add the audit stores.

ii. Navigate to the /opt/protegrity/logforwarder/data/config.d directory, and edit the upstream.cfg file as follows. The [Node] block must be added for each new Audit Store.

 [NODE]
   Name node-1
   Host 10.37.4.150
   Port 9200
   tls on
   tls.verify off
   Pipeline logs_pipeline
 [NODE]
   Name node-2
   Host 10.37.4.158
   Port 9200
   tls on
   tls.verify off
   Pipeline logs_pipeline

The following parameters need to be added for a new node.

Parameter	Description
Name	Set a name for the Audit Store.
Host	IP address or host name of the Audit Store.
Port	Set the port number. The default port number is 9200.
tls	Enable or disable the TLS support. Set this parameter to on to enable the TLS support and off to disable the TLS support. The default tls setting is on.
tls.verify	Force certificate validation. Set this parameter to on to enforce certificate validation and off to disable certificate verification. The default tls.verify setting is off.
Pipeline	Set a filter for the Audit Store. The default pipeline setting is logs_pipeline.

iii. Use the following command to restart the Protegrity Log Forwarder service after editing the file.

/opt/protegrity/logforwarder/bin/logforwarderctrl start

Installing Log Forwarder on Linux using Silent Mode

To preserve all the configurations while upgrading the Log Forwarder, ensure that you backup all the files present under the /opt/protegrity/logforwarder/data/config.d directory.

You can also execute the Log Forwarder installer without any manual intervention, which is also known as the Silent mode of installation. The following parameters must be provided to execute the installer in the Silent mode.

Parameter	Description
`-e` or `--endpoint`	The IP address and port number of the Audit Store instance. You can add multiple Audit Store endpoints. If you add multiple Audit Store points, then you need to provide the `-e` or `--endpoint` argument for each endpoint. The default port number is 9200. If you are using the default port, then do not specify the port number.
`-d` or `--dir`	Installation directory of the Log Forwarder, which is optional. If the installation directory is not specified, then the installation path is the default directory, which is the /opt/protegrity directory.

At the command prompt, type the following command from the installer directory.

./LogforwarderSetup_Linux_x64_<version>.sh -e <ip address:port number> [-e <ip address:port number>]

If you want to install the Log Forwarder in a directory other than the default directory, add the -d or --dir argument to the command to specify the Log Forwarder installation directory

The following snippet displays a sample command.

./LogforwarderSetup_Linux_x64_<version>.sh -e <ip address:port number> [-e <ip address:port number>] -d <Log Forwarder installation directory>

Uninstalling the Log Forwarder on Linux

Navigate to the /opt/protegrity/logforwarder/bin directory.
Stop the Log Forwarder by using the following command.
```
./logforwarderctrl stop
```
Delete the logforwarder directory.
The Log Forwarder and all its components are uninstalled.

Installing Log Forwarder on Windows

The following section describes the steps to install the Log Forwarder on a Windows platform using the Windows wizard or through silent installation.

When you install the Log Forwarder, the system automatically sets up a directory structure with the required files in the ..\Protegrity\logforwarder directory.

Installing Log Forwarder on Windows using the Windows Wizard

To install the Log Forwarder on a Windows platform using the Windows wizard:

Double-click or run the LogforwarderSetup_<OS>_<version>.exe file.
The Setup Wizard appears.
Click Next.
The Audit Store Connectivity Information screen appears.
Select the number of audit stores that are needed, and then click Next.
The screen to specify the Audit Store location appears.
Enter the Audit Store endpoint (IP address:port number).
The default port number is 9200.
Click Next.
The Select Destination Location screen appears.
Browse to the directory in which you want to install the Log Forwarder, or retain the default location.
It is recommended to retain the default location.
Click Next.
The Ready to Install screen appears.
Click Install.
The Windows wizard installs the Log Forwarder on your machine.
Click Finish to close the Log Forwarder Setup Wizard and complete the installation. The directories are created under the installation directory that was defined and the installation files are installed in these directories.

If you want to modify the number of Audit Stores or if you have selected an incorrect number of Audit Stores in step 3, then perform the following steps after the installation completes.

i. Edit the upstream.cfg file to add the audit stores.

ii. Navigate to the ..\Protegrity\logforwarder\data\config.d directory, and edit the upstream.cfg file as follows. The [Node] block must be added for each new Audit Store.

[NODE]
  Name node-1
  Host 10.37.4.150
  Port 9200
  tls on
  tls.verify off
  Pipeline logs_pipeline
[NODE]
  Name node-2
  Host 10.37.4.158
  Port 9200
  tls on
  tls.verify off
  Pipeline logs_pipeline

The following parameters need to be added for a new node.

Parameter	Description
Name	Set a name for the Audit Store.
Host	IP address or host name of the Audit Store.
Port	Set the port number. The default port number is 9200.
tls	Enable or disable the TLS support. Set this parameter to on to enable the TLS support and off to disable the TLS support. The default tls setting is on.
tls.verify	Force certificate validation. Set this parameter to on to enforce certificate validation and off to disable certificate verification. The default tls.verify setting is off.
Pipeline	Set a filter for the Audit Store. The default pipeline setting is logs_pipeline.

iii. Restart the Log Forwarder service from the Windows Task Manager after editing the file.

Installing Log Forwarder on Windows using Silent Mode

Parameter	Description
`-endpoint1`, `-endpoint2`, `-endpoint3`	Audit Store IP address and the Port number where the Log Forwarder sends the logs. The default port number is 9200. The parameters `-endpoint2` and `-endpoint3` are optional.
`-dir`	Installation directory of the Log Forwarder, which is optional. If the installation directory is not specified, then the installation path is the default directory, which is the ..\Protegrity\logforwarder directory.

At the command prompt, type the following command from the installation directory.

.\LogforwarderSetup_<OS>_<version>.exe -endpoint1 <ip address:port number> [-endpoint2 <ip address:port number>] [-endpoint3 <ip address and port number>]

To install the Log Forwarder in a directory other than the default directory, add the -dir parameter to the command to specify the Log Forwarder installation directory. The following snippet displays a sample command.

.\LogforwarderSetup_<OS>_<version>.exe -endpoint1 <ip address:port number> [-endpoint2 <ip address:port number>] [-endpoint3 <ip address and port number>] -dir <Log Forwarder installation directory>

Uninstalling the Log Forwarder

Navigate to the \Protegrity\logforwarder directory.
Double-click the unins000.exe file.
The Log Forwarder Uninstall dialog box appears. A message appears asking you to confirm whether you want to uninstall the Log Forwarder.
Click Yes.
The Log Forwarder and all its components are uninstalled.

1 - Configuring the disk space on the Log Forwarder

The Log Forwarder collects logs from the protectors and forwards them to Insight. Insight stores the logs in the Audit Store. If the Audit Store is not reachable due to network issues, then the Log Forwarder caches the undelivered logs locally on the hard disk.

If the incoming logs are cached faster than they are sent to Insight, then a back pressure arises.

The following formula can be used to calculate the disk space on the Log Forwarder. The formula requires the estimated audit rate and time to sustain the audit rate, without logs being sent to Insight. Modify the values in this example as required. The default value of the disk space is 256 MB.

Disk Space in Mega bytes = (Audit Rate X Time in Seconds X 5.9 ) / 1024.

Audit Rate = Number of policy audits generated per second
Time in Seconds = Time duration for which the disk can sustain the audit rate without the logs being sent to Insight.

If the default or the configured value of the storage.total_limit_size setting is reached, then the Log Forwarder discards the oldest audits to create disk space for new audits.

Perform the following steps to configure the storage.total_limit_size setting in the out.conf file on the protector machine.

Log in and open a CLI on the protector machine.
Navigate to the config.d directory using the following command.
```
cd /opt/protegrity/logforwarder/data/config.d
```
Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/data/config.d path. Use the /opt/protegrity/fluent-bit/data/config.d path for protectors v9.1.0.0 and earlier.
Back up the existing out.conf file using the following command.
```
cp out.conf out.conf_backup
```
Open the out.conf file using a text editor.

Update the value of storage.total_limit_size setting in the output blocks. The default value of the storage.total_limit_size is 256 MB. The following snippet shows the extract of the code.

[OUTPUT]
    Name opensearch 
    Match logdata 
    Retry_Limit False
    Index pty_insight_audit
    Type  _doc
    Time_Key ingest_time_utc
    Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
    storage.total\_limit\_size 256M

[OUTPUT]
    Name opensearch 
    Match flulog
    Retry_Limit 1
    Index pty_insight_audit
    Type  _doc
    Time_Key ingest_time_utc
    Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
    storage.total\_limit\_size 256M

[OUTPUT]
    Name opensearch 
    Match errorlog
    Retry_Limit 1
    Index pty_insight_audit
    Type  _doc
    Time_Key ingest_time_utc
    Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
    storage.total\_limit\_size 256M

Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/data/config.d path. Use the /opt/protegrity/fluent-bit/data/config.d path for protectors v9.1.0.0 and earlier.

Save and close the file.
Restart the Log Forwarder on the protector using the following commands.
```
/opt/protegrity/logforwarder/bin/logforwarderctrl stop
/opt/protegrity/logforwarder/bin/logforwarderctrl start
```
Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/bin path. Use the /opt/protegrity/fluent-bit/bin path for protectors v9.1.0.0 and earlier.
If required, complete the configurations on the remaining protector machines.