This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Enterprise Security Administrator (ESA)

ESA handles the management of policies, keys, monitoring, auditing, and reporting of protected systems in the enterprise. ESA appliances have the Insight, and the td-agent service installed and are capable of hosting Docker containers.

The detailed Architecture diagram for ESA v10.0.1 is shown below.

ESA v10.0.1 Architecture

1 - Infrastructure Requirements

2 - Installing and Configuring ESA

The ESA can be installed on-premise or a cloud platform such as AWS, GCP, or Azure. When upgrading from a previous version, the ESA installer is available as a patch.

Assumptions

  • This section assumes that there is no prior installation of protegrity product and installation is happening from scratch.

  • GTM and LTM are provisioned and installed.

    For more information about the prescribed configurations, refer Recommended Traffic Manager.

This section explains about Installation and Configuration of ESAs as per Architecture Diagram in Deployment with Default Audit logging flow to ESA.

For more information about installing ESA on on-premise or cloud platforms, refer Installation.

Prerequisites

Before proceeding with the installation, ensure the following prerequisites are met.

  • Sites: Ensure that two sites are available — one designated for the Primary Site and another for the Disaster Recovery (DR) Site.

  • Network Connectivity: Ensure there is reliable network connectivity between the Primary and DR sites.

Installing and Configuring ESA

  1. Install ESA (ESA P1) in the Primary Site.

    For more information about detailed installation instructions for on-premise and cloud installation, refer Installing ESA.

  2. Initialize PIM in ESA P1.

    For more information about initializing PIM in ESA, refer Initializing the Policy Management.

  3. Initialize Analytics in the Insight in ESA P1.

    For more information about initializing Analytics, refer Creating the Audit Store Cluster on the ESA.

  4. Upload and apply Custom Certificates. Apply custom certificates for the following components:

    • Management Server/Client
    • Consul Server
    • Audit Store Server/Client
    • Audit Store REST Client
    • Insight Analytics Client
    • PLUG Client

    For more information about recommendations related to certificates for various components, refer Certificate Requirements.

    For more information about steps to upload certificates in ESA, refer Uploading Certificates.

    For more information about steps to apply certificates in ESA, refer Changing Certificates.

  5. Define policies for Data Security. On ESA P1, define the necessary data elements, policies and configure external member source.

  6. Configure Proxy Authentication and External LDAP. On ESA P1, configure proxy authentication for configuration with External LDAP for managing ESA Administration. Refer below sections:

  7. Configuring ESA with External Keystore.

    For more information about setting ESA to External Keystore, refer Section Switching HSM Modules from HSM Integration Guide for ESA.

  8. Configure Rollover Index Insight Scheduler Parameters. Set the rollover index scheduler parameters according to specific requirements.

    For more information about recommended configurations, refer Index Rollover.

  9. Configure Information Lifecycle Management (ILM) Export Insight Scheduler Parameters. Adjust the ILM export scheduler settings based on the requirements.

    For more information about recommendation configurations for ILM Export, refer ILM Export.

  10. Configure ILM Multi Delete Insight Scheduler Parameters. Set the ILM Multi Delete insight scheduler parameters according to specific requirements.

For more information about recommendation configurations for ILM Multi Delete, refer [ILM Delete](/docs/model_arch/model_arch/esa/insight/#ilm-delete). <!-- fix link here -->
  1. Configure Alerts. Set up the required alerts to monitor system health and events.
For more information about recommendations related to configuration of alerts, refer [Alerting](/docs/model_arch/model_arch/esa/insight/#alerting). <!-- fix link here -->
  1. Install additional ESAs.

    1. In the Primary Site, install two additional ESAs: ESA S1 and ESA S2.

    2. In the DR Site, install three ESAs: ESA S3, ESA S4, and ESA S5.

  2. Create Trusted Appliances Cluster (TAC) between all ESAs in Primary and DR site. Create a TAC between ESAs P1, S1,S2 in Primary and ESAs S3,S4,S5 in DR Site.

For more information about creating TAC, refer [Creating a TAC using the Web UI](/docs/aog/trusted_appliances_cluster/aog_creating_tac_ui/).
  1. Form an Insight Cluster: Join ESA S1 and ESA S2 to ESA P1 to form a robust and redundant Insight Cluster. Form Insight cluster in DR site between ESA S3,S4 and S5.
For more information about creating Insight cluster, refer [Creating an Audit Store cluster](/docs/installation/upg_creating_audit_store_cluster/).
  1. Create Replication Tasks. Establish a replication task on ESA P1 to replicate all ESA data—excluding SSH settings—to ESA S1, S2, S3, S4, and S5.

    For recommendations, refer point 1 ESA Primary to Secondary Replication Job in the Scheduler Tasks.

    For more information about configuring scheduled task for cluster export, refer Scheduling Configuration Export to Cluster Tasks.

  2. Enable ILM Multi Export Scheduler task. By default, ILM Multi Export Scheduled task is disabled. So, it is necessary to enable this task and configure as per requirements.

    For more information about ILM Multi Export, refer Exporting logs.

  3. Create Scheduled Tasks for copying ILM Exported Files from Primary Site to DR Site. Create a scheduled task in each of the ESAs in the primary site, that is, ESA P1, ESA S1, ESA S2 to copy ILM exported files from ESA P1,ESA S1, and ESA S2 to ESAs in DR site, that is, ESA S3, ESA S4, and ESA S5 in the DR Site.

    For recommendations, refer point 2 ESA Exported Indexes Purge in Scheduler Tasks.

    For more information, refer Creating a scheduled task.

  4. Create Scheduled task for taking backup of ESA P1 to a file.

    For recommendations, refer point 3 Back up Primary ESA data to file in Scheduler Tasks.

Keep a detailed log and documentation of each step performed for future reference and troubleshooting.
Meticulously following these steps help establish a resilient and secure ESA infrastructure that aligns with the specified model architecture diagram.

4 - External Keystore Configuration

To enhance security, Protegrity products can be integrated with external Key Stores. For more information to configure ESA with External Key Store, refer section Configuring ESA Appliance or DSG Appliance to Communicate with Safenet HSM in the HSM Integration Guide the My.Protegrity portal.

5 - Scheduler Tasks

To maintain the resiliency, efficiency, and reliability of the ESA, it is essential to set up scheduled tasks.

The recommended scheduled tasks to configure on the ESA is provided here.

Task Summary and Description

1. ESA Primary to Secondary Replication Job

  • Description: This task must be configured in primary ESA P1. It performs replication of data from the Primary ESA P1 to all the secondary ESAs, that is, ESA S1, ESA2, ESA S3, ESA S4, ESA S5.

  • Frequency: Recommended to run daily at midnight. It is important that this task must be executed immediately after a change in policy or rulesets or in case of changes made to any of the keys under Key Management menu in ESA Web UI.

Important: Ensure only the following items are selected as part of this Cluster Export task.

Appliance OS Configuration: Export OS configuration

  • Web Settings
  • Firewall Settings
  • Appliance Authentication Settings
  • Appliance JWT Configuration
  • Appliance AZURE AD Configuration (If Azure AD is being used)
  • Time-zone and NTP settings
  • OS Services Status
  • Appliance FIM Policies and Settings
  • User custom list of files (If custom files are present)

Directory Server And Settings: Export directory services and related settings

  • LDAP Server

Export Consul Configuration and Data

  • Import Consul Certificates and keys

Cloud Utility AWS: Cloud Utility AWS CloudWatch configuration files

  • Import Cloud Utility AWS CloudWatch configuration files

Backup Policy-Management for Trusted Appliances Cluster: Backup All Policy-Management Configs, Keys, Certs and Data for Trusted Appliance Cluster

  • Import All Policy-Management Configs and Data for Trusted Appliance Cluster

Policy Manager Web UI Settings: Export policy manager web ui settings

  • Import policy manager web ui settings

Export Gateway Configuration Files: Gateway Config Files Export

  • Gateway Config Files Import

Do not select the below options as part of this cluster export task

  • SSH settings
  • Management and WebServices Certificates
  • Certificates
  • Import All Policy-Management Configs, Keys, Certs, Data but without Key Store files for Trusted Appliance Cluster

Important: Scheduled tasks are not replicated as part of cluster export.

2. ESA Exported Indexes Move

  • Description: This task must be configured in all the ESAs in primary site, that is, ESA P1, ESA S1, and ESA2. It moves the ILM exported index files from respective ESAs, that is, ESA P1, ESA S1 and ESA S2 to ESA S5 in DR site.

  • Frequency: Recommended to run weekly.

3. Back up Primary ESA data to file.

  • Description: This task must be configured in primary ESA P1. It backs up data from Primary ESA to a file and moves it to a remote machine.

  • Frequency: Recommended to run weekly.

    Important: Ensure only the following items are selected as part of this File Export task.

  • Appliance OS Configuration

  • Directory Server And Settings

  • Export Consul Configuration and Data

  • Cloud Utility AWS (if AWS Cloud Utility is configured and being used)

  • Backup Policy-Management

  • Policy Manager Web UI Settings

  • Export Gateway Configuration Files

Do not select the below option as part of this file export task

  • Backup Policy-Management for Trusted Appliances Cluster without Key Store.

Important: Scheduled tasks are not backed up as part of file backup.

6 - Backup and Restore

Effective backup and restore procedures are crucial for ensuring data integrity and system reliability.

Below are the key backup and restore strategies to implement in the ESA environment:

Full OS

  • Description: Backup of the complete operating system including all configurations, applications, and user data in ESA. This is applicable only for on-prem environment.

  • Frequency: Recommended weekly or before and after significant changes such as upgrade.

  • Restore Steps: Boot into a System Restore Mode to restore to last backed up state.

For more information to perform full OS backup and restore, refer Working with OS Full Backup and Restore.

Cloud

  • Description: Create snapshots of ESA instances for disaster recovery. This is applicable only for cloud environment.

  • Frequency: Recommended weekly or before and after significant changes such as upgrade or changes to policy in ESA.

  • Restore Steps: Restore the snapshot by following the cloud provider-specific restore procedure.

File

  • Description: Ensure that ESA data is regularly backed up and stored securely. Automating the process using scheduled task in ESA will help maintain consistency and reduce manual intervention. The restore steps allow easy recovery of data, ensuring minimal disruption in case of failure.

  • Frequency: Recommended weekly or before and after significant changes such as upgrade or changes to policy in ESA or in case of changes made to any of the keys under Key Management menu in ESA Web UI.

  • Tools: Configure Scheduled task in ESA to use tools like rsync, scp to copy exported backup files from ESA to an external machine. Ensure backup files are created along with timestamp. Refer point 3 in Scheduler Tasks for achieving this.

  • Restore Steps: Upload the required backed up matching the required timestamp to the ESA and initiate import. For more information about steps to import the backed up file, refer Importing Data/Configurations from a File.

Important: Scheduled tasks are not backed up as part of file backup.

Custom Files

The following custom files should be backed up and included as part of customer.custom to ensure they are replicated to all ESAs in the cluster during replication jobs:

  • Any custom files for td-agent.

  • Any custom files related to External HSM configurations, shared library files.

  • Any changes to rsyslog.

For more information about adding the required custom files as mentioned above, refer Working with the custom files.

7 - Insight

Insight Analytics and Audit (Insight) leverages OpenSearch and OpenSearch Dashboard to perform analytics on audit events and log messages aggregated in the Audit Store. Both are distributed as Docker containers that can be hosted on ESAs.

ILM Export

This section outlines the ILM export configuration for various log and metric indices. The objective is to manage the lifecycle of these indices efficiently, ensuring data is archived or deleted as required.

IndexILM Export Configuration
Audit Log IndexMaximum size: 50 GB
Maximum doc count: 50 million
Maximum index age: 30 days
Protector Status Logs IndexMaximum size: 150 GB
Maximum Doc count: 1 billion
Maximum index Age: 365 days
Troubleshooting Logs IndexMaximum size: 150 GB
Maximum Doc count: 1 billion
Maximum index Age: 365 days
Policy Logs IndexMaximum size: 150 GB
Maximum doc count: 1 billion
Maximum index age: 365 days
Miscellaneous IndexMaximum size: 200 MB
Maximum doc count: 3.5 million
Maximum index age: 7 days
DSG Transaction Metrics IndexMaximum size: 1 GB
Maximum doc count: 10 million
Maximum index age: 1 day
DSG Error Metrics IndexMaximum size: 1 GB
Maximum doc count: 3.5 million
Maximum index age: 1 day
DSG Usage Metrics IndexMaximum size: 1 GB
Maximum doc count: 3.5 million
Maximum index age: 1 day

ILM Export Configuration Considerations

  • Maximum size: Defines the maximum size an index can reach before getting exported.

  • Maximum doc count: Defines the maximum doc count an index can reach before getting exported.

  • Maximum index age: Defines the maximum age an index can reach before getting exported.

  • Conditions: ILM Export occurs when either of the above limits is reached where index entries from the index are removed and archived to a file.

For more information about configuring ILM Export, refer ILM Multi Export.

ILM Delete

This section outlines the ILM export configuration for various log and metric indices. The objective is to manage the lifecycle of these indices efficiently, ensuring data is archived or deleted as required.

IndexILM Export Configuration
Miscellaneous IndexMaximum size: 1 GB
DSG Transaction Metrics IndexMaximum size: 14 GB
Maximum doc count: 100 million
Maximum index age: 30 days
DSG Error Metrics IndexMaximum size: 6 GB
Maximum doc count: 50 million
Maximum index age: 30 days
DSG Usage Metrics IndexMaximum size: 10 GB
Maximum doc count: 75 million
Maximum index age: 30 days

ILM Export Configuration Considerations

  • Maximum size: Defines the maximum size an index can reach before getting deleted.

  • Maximum doc count: Defines the maximum doc count an index can reach before getting deleted.

  • Maximum age: Defines the maximum age an index can reach before getting deleted.

  • Conditions: Deletion occurs when either of the above limits is reached.

For more information about configuring ILM Delete, refer ILM Multi Export.

Index Rollover

This section details the index rollover settings for various log and metric indices. Efficient rollover policies ensure high performance and manageability of the indices.

IndexIndex Rollover Configuration
Audit Log IndexMaximum size: 50 GB
Maximum doc count: 50 million
Maximum index age: 1 day
Protector Status Logs IndexMaximum size: 5 GB
Maximum doc count: 200 million
Maximum index age: 30 days
Troubleshooting Logs IndexMaximum size: 5 GB
Maximum doc count: 200 million
Maximum index age: 30 days
Policy Logs IndexMaximum size: 5 GB
Maximum doc count: 200 million
Maximum index age: 30 days
Miscellaneous IndexMaximum size: 200 MB
Maximum doc count: 3.5 million
Maximum index age: 7 days
DSG Transaction Metrics IndexMaximum size: 1 GB
Maximum doc count: 10 million
Maximum index age: 1 day
DSG Error Metrics IndexMaximum size: 1 GB
Maximum doc count: 3.5 million
Maximum index age: 1 day
DSG Usage Metrics IndexMaximum size: 1 GB
Maximum doc count: 3.5 million
Maximum index age: 1 day

Index Deletion Configuration Considerations

  • Maximum size: Defines the maximum size an index can reach before rolling over.

  • Maximum doc count: Defines the maximum doc count an index can reach before rolling over.

  • Maximum age: Defines the maximum age an index can reach before rolling over.

  • Conditions: Index Rollover occurs when either of the limits is reached.

For more information about configuring Audit Index Rollover, refer Audit Index Rollover.

Alerting

Create a scheduled task in all the ESAs to monitor and log spikes in CPU, memory, or disk usage that exceed configured thresholds. These logs must be systematically forwarded to the Insight within the ESA.

For more information about configuring alerts, refer Working with alerts.

Requirements

  1. Monitoring Metrics: The task should observe the following system metrics:

    1. CPU Usage
    2. Memory Usage
    3. Disk Usage
  2. Threshold Configuration

    1. Define specific thresholds for CPU, memory, and disk usage.
    2. Ensure these thresholds can be adjusted as needed.
  3. Log Generation

    Generate detailed logs whenever a spike in CPU, memory, or disk usage exceeds the configured threshold.

  4. Log Forwarding

    Implement mechanisms to forward these logs to the Insight within the ESA.

Implementation Steps

  1. Script Development

    • Develop a script to monitor CPU, memory, and disk usage.
    • Incorporate threshold parameters into the script.
  2. Schedule the task using Task Scheduler.

    For more information about creating scheduled task using task scheduler, refer Creating a scheduled task.

  3. Logging Mechanism

    Use logger library to write logs to syslog.

  4. Test and Validate

    • Conduct thorough testing to ensure the script accurately detects and logs spikes.

    • Validate that logs are correctly forwarded to and received by the Insight.

By implementing this scheduled task, the ability to monitor system health and respond proactively to potential issues is enhanced, thereby improving overall system stability and security compliance.

Audit Store Dashboards

It is recommended that default dashboards in ESAs are not modified or deleted.

For more information on Protegrity provided dashboards, refer Working with Protegrity dashboards.