Managing Configuration Files in DSG

• 1: Configuring the gateway.json file
• 2: Configuring the features.json file
• 3: Configuring the pycore.ini file

1 - Configuring the gateway.json file

This section describes global configuration settings that apply across all DSG nodes in a cluster. These settings extend the options available in the Global Settings tab by providing additional system‑level controls.

The gateway.json file includes configurations, such as, setting the log levels, enabling learn mode, and so on.

Note: It is recommended that configuration changes are made on the ESA and then deployed to the DSG nodes in the cluster

Accessing the gateway.json file

1. Login to the ESA Web UI.

2. Navigate to the Settings > System.

3. Go to Cloud Gateway - Settings area, access the gateway.json file.

Sample gateway.json file

The sample configuration is illustrated below.

{
    "log": {
        "logLevel": "Warning",
        "logFacility": [
            {
                "enabled": false, 
                "facilityName": "Tunnel", 
                "logLevel": "Information"
            }, 
            {
                "enabled": false,
                "facilityName": "DiskBuffer", 
                "logLevel": "Warning"
            }, 
            {
                "enabled": false,
                "facilityName": "Admin", 
                "logLevel": "Warning"
            }, 
            {
                "enabled": false,
                "facilityName": "RuleSet", 
                "logLevel": "Verbose"
            }, 
            {
                "enabled": false,
                "facilityName": "Service", 
                "logLevel": "Warning"
            }
        ]
    },
    "mountManager": {
        "enabled": true,
        "interval" : "*/3 * * * *"
    },
    "admin": {
        "listenAddress": "ethMNG",
        "listenPort": 8585,
        "certificateFilename": "admin.pem",
        "certificateKeyFilename": "admin.key",
        "ciphers": "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS!SSLv2:!SSLv3!TLSv1!TLSv1.1",
        "clientCACertificateFileName" : "ca.pem",
        "clientCertificateFileName" : "admin_client.pem",
        "clientCertificateKeyFileName" : "admin_client.key",
        "commonName" : "protegrityClient",
        "ssl_options":"{\"cert_reqs\":\"CERT_REQUIRED\"}"
    },
    "learnModeDefault": {
        "enabled": false,
        "excludeResource": "\\.(css|png|gif|jpg|ico|woff|ttf|svg|eot)(\\?|\\b)",
        "excludeContentType": "\\bcss|image|video|svg\\b",
        "freeDiskSpaceThreashold": 1024000000
    },
    "globalUDFSettings" : {
        "allowed_modules":["bs4", "common.logger", "re", "gzip", "fromstring", "cStringIO","struct", "traceback"] ,
        "allowed_methods" : ["BeautifulSoup", "find_all", "fromstring", "format_exc", "list", "dict", "str", "warning"]
    },
    "globalProtocolStackSettings": {
        "http": {
            "max_clients": 100,
            "connection_cache_ttl": -1,
            "max_body_size": 4194304,
            "max_streaming_body_size": 52428800,
            "include_hostname_in_header": true
        }
    },
    "longRunningRoutinesTracing": {
        "enabled": false,
        "timeout": 20
    },
    "pdf_codec_default_font":{
        "name": "OpenSans-Regular.ttf"
    },
    "stats" :{
        "enabled" : true
    }
}

Structure of the Configuration File

The configuration file is organised into logical sections, with each section defining a specific area of system‑level behavior for DSG.

log

Controls logging behaviour for DSG, including global log levels and log facilities that determine how gateway activity is recorded. The following configurations are available for the log configuration:

• logLevel: Set the logging level. The available logging levels are as follows:

  • Warning (default)
  • Info
  • Debug
  • Verbose

• logFacility: Set the logging level for the following modules:

  • Ruleset
  • Services
  • Tunnel
  • DiskBuffer
  • Admin

• checkErrorLogAfterCount: Decide the trimming factor that is a part of the error metrics. You can set this value in the range of -1 to 1000.

  • If the value set is greater than -1 and the log size of the error metrics is greater than 4k, then it will trim the error_metrics in such a way that all the parameters will be displayed accurately and only the row number information will be trimmed.
  • If the log size is not exceeding 4k, then the error metrics will be displayed as is.
  • If the value is set to -1 and the log size of error metrics is greater than 4k, then all the characters after the 4k limit will be trimmed from the log file.
  • If the logs are not repetitive, additional rows will be reported in separate logs. This parameter is not present in the gateway.json file. Add the checkErrorLogAfterCount parameter to enable this feature.

mountManager

Settings related to NFS mounts. The following configurations are available for the log configuration:

• enabled: Enable or disable mount management.

• interval: Time in seconds when the DSG node will poll the NFS shares for pulling files. You can also specify a cron job expression. The cron job format is also supported to schedule jobs. If you use the cron job expression “* * * * *”, then the DSG will poll the NFS shares at the minimum interval of one minute.

admin

Settings related to admin tunnel are listed. DSG uses this tunnel for internal communication with ESA and other DSG nodes.

• listenAddress: Listening interface name, typically ethMNG.

• listenPort: Port on which the interface listens to.

• certificateFilename: Admin tunnel certificate file name with the .pem extension. The default certificates and keys are set after the DSG is installed.

• certificateKeyFilename: Admin tunnel key file name with the .key.

• ciphers: Colon separated list of Ciphers.

• clientCACertificateFilename: Admin tunnel CA certificate filename with the .pem extension.

• clientCertificateFilename: Admin tunnel client certificate filename with the .pem extension.

• clientCertificateKeyFilename: Admin tunnel Client key file name with the .key extension.

• commonName: Common name as defined while creating the admin tunnel client certificates.

• ssl_options: Set the SSL options to be enforced. For a secure communication between DSG and ESA, it is recommended not to modify this option. Default value is "cert_reqs":"CERT_REQUIRED".

learnModeDefault

Settings for the Learn Mode.

• enabled: Enable or disable Learn Mode on the DSG node. Default value it true.

• excludeResources: Values in the field are excluded from the Learn Mode logging. Default value is \.(css|png|gif|jpg|ico|woff|ttf|svg|eot)(\?|\b).

• excluedContentType: Content type specified in the field is excluded from the Learn Mode logging. Default value is \bcss|image|video|svg\b.

• freeDiskSpaceThreshold: Minimum free disk space required so that the Learn Mode feature remains enabled. The feature is automatically disabled, if free disk space falls below this threshold. If the setting is disabled, then you must enable this feature manually. Default value is 1024000000.

globalUDFSettings

Settings that apply to any rules defined with custom UDF implementation for a DSG node.

• allowed_modules: List of modules that can be used in the UDF. Default value it bs4, common.logger, re, gzip, fromstring, cStringIO,struct, traceback.

• allowed_methods: List of methods that can be used in the UDF. Default value is BeautifulSoup, find_all, fromstring, format_exc, list, dict, str, warning.

globalProtocolStackSettings (http)

Settings for incoming HTTP requests management.

• max_clients: Set the maximum number of concurrent outbound connections every gateway process can establish with each host. Default value is 100.

• include_hostname_in_header: By default, the hostname will be visible in response header. Set the parameter to false, to remove the hostname from the response header. Default value is true.

• connection_cache_ttl: Timeout value that you must configure up to which an HTTP request connection persists. Following values can be set.

  • -1: Set to enable caching (default).
  • 0: Set to disable caching.
  • : Set a value in seconds.

• max_body_size: Maximum bytes for the HTTP request body. The datatype for this option is bytes. Default value is 4194304.

• max_streaming_body_size: Maximum bytes for the HTTP request body when REST with streaming is enabled. The datatype for this option is bytes. Default value is 52428800.

longRunningRoutinesTracing

• enabled: Enable or disable tracing. Default value is false.

• timeout: Define the value in seconds to log a stack trace of processes that do not process easily in the given timeout interval. You can set the parameter to false, to remove the hostname from the response header. Default value is 20.

pdf_codec_default_font

Specifies the default font used by the Enhanced Adobe PDF codec when processing PDF content.

• name: Set the default font file to process the PDF file under the Enhanced Adobe PDF codec extract rule. Default value is OpenSans-Regular.ttf.

stats

• enabled: Enable or disable the usage metrics. Default value is true.

2 - Configuring the features.json file

The features.json file is one of the files in the Protegrity Data Security Gateway (DSG) configuration. By adding or removing flags to this file, users can enable or disable specific behavior in the product.

Sample features.json file

The following snippet shows the default parameters configured in the features.json file.

{
    "features": [
        "enhanced-http-transaction-metrics",
        "disable-sftp-client-key-check",
        "normalize-time-labels",
        "enhanced-lock-filename"
    ]
}

The following table provides descriptions of the feature flags that can be configured in the features.json file.

• DSG 4.0.0
• DSG 4.0.1

Adding Feature Flags to the features.json File

1. Login to the ESA Web UI.
2. Navigate to the Settings > System.
3. Go to Cloud Gateway - Settings area, access the features.json file.
4. Open the features.json file for editing.
5. Add the required feature flags in the features.json file.

   Note: Ensure the file is valid JSON after modification.

6. Deploy the configurations from the Cluster page on the ESA to apply the changes across all DSGs.

3 - Configuring the pycore.ini file

Accessing the pycore.ini File

1. Login to the DSG Web UI.
2. Navigate to Settings > System > Files.
3. Open the pycore.ini file.

Sample pycore.ini file

The following represents a sample pycore.ini file.

###############################################################################
# Resilient Package Sync Config
# -----------------------------
# Protector configuration
# ----------------------------- 
[protector]

# Cadence determines how often the protector connects with ESA / proxy to fetch the policy updates in background.
# Default is 60 seconds. So by default, every 60 seconds protector tries to fetch the policy updates.
# If the cadence is set to "0", then the protector will get the policy only once.
#
# Default 60.
cadence = 60
###############################################################################
[sync]

# Protocol to use when communicating with the service providing Resilient Packages.
# Use 'https' for ESA or 'shmem' for local shared memory.
protocol = https

# Host/IP to the service providing Resilient Packages
host = <hostname>

# Path to CA certificate
ca = /mnt/ramdisk/certificates/mng/CA.pem

# Path to client certificate
cert = /mnt/ramdisk/certificates/mng/client.pem

# Path to client certificate key
key = /mnt/ramdisk/certificates/mng/client.key

# Path to a secret file that is used to decrypt the client certificate key.
# When using a custom certificate bundle, the 'secretcommand' can instead be 
# used to execute an external command that obtains the secret.
#secretfile = REPLACE_SYNC_SECRET_FILE

###############################################################################
# Log Provider Config
###############################################################################
[log]

# In case that connection to fluent-bit is lost, set how audits/logs are handled
# 
# drop  : (default) Protector throws logs away if connection to the fluentbit is lost
# error : Protector returns error without protecting/unprotecting 
#         data if connection to the fluentbit is lost
mode = drop

# Host/IP to fluent-bit where audits/logs will be forwarded from the protector
#
# Default localhost
host = localhost
port = 15780

The following table helps you to understand the usage of the parameters listed in the pycore.ini configuration file for DSG.

Important: It is recommended that only the parameters listed in the following table are edited as per your requirement.