Auditing and logging

Auditing and logging for DSG.

For every configuration change that occurs on the DSG, such as, creation of tunnels, Rulesets, deployinging of the configuration, and so on, the DSG generates an audit log. Though most of the logs are forwarded to the ESA and are visible on Forensics, some DSG logs serve a specific purpose and are available only on the individual DSG nodes.

Discover

The log management mechanism for Protegrity products forwards the logs to Insight on the ESA. Insight stores the logs in the Audit Store. The following services forwards the logs to Insight:

  • td-agent service installed. The td-agent forwards the appliance logs to Insight on the ESA.
  • Log Forwarder service forwards the data security operations-related logs, namely protect, unprotect, and reprotect and the PEP server logs to Insight on the ESA.

Note: Before you can access Discover, you must configure the DSG to forward logs to Insight on the ESA. You must also verify that the td-agent and the Log Forwarder services are running on the DSG. To verify the service status, navigate to System > Services on the DSG Web UI.

For more information about configuring the DSG to forward appliance logs to the ESA, refer to Forwarding Appliance Logs to Insight.

For more information about configuring Log Forwarder to forward the audit logs, refer to Forwarding Audit Logs to Insight.

Note: The Log Forwarder configuration can be modified in the pepserver.cfg file. If the Log Forwarder mode in the pepserver.cfg file is modified to error mode or drop mode, then the Pepserver service and the Cloud Gateway* service should be restarted.

To restart the services, login to the DSG Web UI, navigate to System > Services, restart the Pepserver service, and then restart the Cloud Gateway service.

For more information about logging configuration in the pepserver.cfg file, refer to the section PEP Server Configuration File in the Protegrity Installation Guide 9.1.0.0.

To access the Discover logs, on the ESA Web UI, navigate to Audit Store > Dashboard > Discover. The Discover page displays audit logs for the following events:

  • Tunnel creation, deletion, and updates
  • Ruleset creation, deletion, and updates
  • Certificate upload, deletion, and downloads
  • Key upload
  • Deploying the DSG configurations
  • Configuration changes made in the Global Settings tab
  • Data security operations, such as, protect, unprotect, and re-protect
  • System logs
  • PEP server logs

Log Viewer

The Log Viewer displays the aggregation of the gateway logs for all the DSG nodes in the cluster. To access the Log Viewer file, on the ESA Web UI, navigate to **Cloud Gateway > 3.3.0.0 {build number} > Log Viewer.

Important: The gateway logs are not forwarded to Insight.

The Log Viewer file details log messages that can be used to analyze the following events:

  • UDF-related compilation errors
  • Transaction metrics logs
  • Stack traces to debug exceptions

Note: The gateway.log file can also be forwarded to any log forwarding system, such as, a Security Information and Event Management (SIEM) system or AWS CloudWatch utility.

For more information about log forwarding, refer to Forwarding logs to SIEM systems.

Audit log representation

The DSG has the Log Forwarder service that forwards the logs related to the data security operations, such as, protect, unprotect, reprotect, and the PEP server logs to Insight on the ESA. The logs generated from the DSG are collected and forwarded to Insight. Insight stores the logs in the Audit Store.

The Audit Store holds the logs and these log records are used in various areas, such as, Insight , forensics, alerts, reports, dashboards, and so on. Insight is a component which provides the interface for viewing the data from the Audit Store. When the data security operations are performed to protect the sensitive data, an aggregated audit log is generated, and displayed on the Discover page in the Audit Store Dashboards.

Before you begin:

  • Ensure that the Analytics component is initialized on the ESA Web UI. On the ESA Web UI, you can access the logs on the Discover page only after initializing the Analytics component.

    For more information about initializing the Analytics component, refer to Initializing the Audit Store Cluster on the ESA.

To understand auditing and logging for the DSG, consider the following example that will be processed using the CSV codec to extract the sensitive data.

firstname,lastname,city,country
John,Smith,London,Uk
Adam,Martin,London,Uk
Jae,Crowder,London,Uk
John,Holland,Bern,Switzerland
Marcus,Smart,Paris,France
Johnson,Mickey,Ottawa,Canada

For more information about extracting the CSV payload, refer to the section CSV Payload.

The CSV extract rule is defined to process all the rows and columns. When a request is sent, the DSG processes the request and the data is protected.

firstname,lastname,city,country
5HnMc6vZ,G8bcRG7J1X,SQSsyxEgBKw,ATJuBh
CMgcxkSL,dlyfZKMIt5H,SQSsyxEgBKw,ATJuBh
Iqj0jjq,RgbFVD6GnOjT,SQSsyxEgBKw,ATJuBh
5HnMc6vZ,SQtul5Lqymz0,1dC18Ciy,jTFgvSyjjROCx9QZOw
6Tz3mgUy3aD,pqDuxmLouR,49HA83v7PO,Jb3kzS8gcyk
4iILZXVL06xs,nXhtMyK6vx8,TiRDIPY1Ik5,Elc5GhObzFF

After the protection operation is completed, a log is generated on the Forensics page on the ESA Web UI. An aggregated log is generated for all the protect operations performed by the CSV codec. In versions prior to the DSG 2.6.0.0, 24 different audit records were generated for each protect operation. Logging is now enhanced on the DSG and a single log entry with the count 24 is generated for the example. A log with the count is only displayed when the protect operation is completed successfully. In case of failure, the individual audit records will be displayed on the Forensics page on the ESA Web UI.

The following figure illustrates the audit log representation for the protect operation.

Last modified February 7, 2025