If the incoming logs are cached faster than they are sent to the Audit Store, then a back pressure arises.
The following formula can be used to calculate the disk space on the Log Forwarder. The formula requires the estimated audit rate and time to sustain the audit rate, without logs being sent to the Audit Store. Modify the values in this example as required. The default value of the disk space is 256 MB.
Disk Space in Mega bytes = (Audit Rate X Time in Seconds X 5.9 ) / 1024.
If the default or the configured value of the storage.total_limit_size setting is reached, then the Log Forwarder discards the oldest audits to create disk space for new audits.
Perform the following steps to configure the storage.total_limit_size setting in the out.conf file on the protector machine.
Log in and open a CLI on the protector machine.
Navigate to the config.d directory using the following command.
cd /opt/protegrity/logforwarder/data/config.d
Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/data/config.d path. Use the /opt/protegrity/fluent-bit/data/config.d path for protectors v9.1.0.0 and earlier.
Back up the existing out.conf file using the following command.
cp out.conf out.conf_backup
Open the out.conf file using a text editor.
Update the value of storage.total_limit_size setting in the output blocks. The default value of the storage.total_limit_size is 256 MB. The following snippet shows the extract of the code.
[OUTPUT]
Name opensearch
Match logdata
Retry_Limit False
Index pty_insight_audit
Type _doc
Time_Key ingest_time_utc
Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
**storage.total\_limit\_size 256M**
[OUTPUT]
Name opensearch
Match flulog
Retry_Limit 1
Index pty_insight_audit
Type _doc
Time_Key ingest_time_utc
Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
**storage.total\_limit\_size 256M**
[OUTPUT]
Name opensearch
Match errorlog
Retry_Limit 1
Index pty_insight_audit
Type _doc
Time_Key ingest_time_utc
Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
**storage.total\_limit\_size 256M**
Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/data/config.d path. Use the /opt/protegrity/fluent-bit/data/config.d path for protectors v9.1.0.0 and earlier.
Save and close the file.
Restart the Log Forwarder on the protector using the following commands.
/opt/protegrity/logforwarder/bin/logforwarderctrl stop
/opt/protegrity/logforwarder/bin/logforwarderctrl start
Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/bin path. Use the /opt/protegrity/fluent-bit/bin path for protectors v9.1.0.0 and earlier.
If required, complete the configurations on the remaining protector machines.