Additional configurations in gateway.json

Multiple settings for setting up the DSG.

The Global Settings tab provides information about the different configurations, that when set, are enforced across all the DSG nodes. In addition to the setting that are part of the Global Settings tab, the gateway.json file also includes additional settings.

The gateway.json file includes configurations, such as, setting the log levels, enabling learn mode, and so on. The sample configuration is illustrated below:

{
    "log": {
        "logLevel": "Warning",
        "logFacility": [
            {
                "enabled": false, 
                "facilityName": "Tunnel", 
                "logLevel": "Information"
            }, 
            {
                "enabled": false,
                "facilityName": "DiskBuffer", 
                "logLevel": "Warning"
            }, 
            {
                "enabled": false,
                "facilityName": "Admin", 
                "logLevel": "Warning"
            }, 
            {
                "enabled": false,
                "facilityName": "RuleSet", 
                "logLevel": "Verbose"
            }, 
            {
                "enabled": false,
                "facilityName": "Service", 
                "logLevel": "Warning"
            }
        ]
    },
    "mountManager": {
        "enabled": true,
        "interval" : "*/3 * * * *"
    },
    "admin": {
        "listenAddress": "ethMNG",
        "listenPort": 8585,
        "certificateFilename": "admin.pem",
        "certificateKeyFilename": "admin.key",
        "ciphers": "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS!SSLv2:!SSLv3!TLSv1!TLSv1.1",
        "clientCACertificateFileName" : "ca.pem",
        "clientCertificateFileName" : "admin_client.pem",
        "clientCertificateKeyFileName" : "admin_client.key",
        "commonName" : "protegrityClient",
        "ssl_options":"{\"cert_reqs\":\"CERT_REQUIRED\"}"
    },
    "learnModeDefault": {
        "enabled": false,
        "excludeResource": "\\.(css|png|gif|jpg|ico|woff|ttf|svg|eot)(\\?|\\b)",
        "excludeContentType": "\\bcss|image|video|svg\\b",
        "freeDiskSpaceThreashold": 1024000000
    },
    "globalUDFSettings" : {
        "allowed_modules":["bs4", "common.logger", "re", "gzip", "fromstring", "cStringIO","struct", "traceback"] ,
        "allowed_methods" : ["BeautifulSoup", "find_all", "fromstring", "format_exc", "list", "dict", "str", "warning"]
    },
    "globalProtocolStackSettings": {
        "http": {
            "max_clients": 100,
            "connection_cache_ttl": -1,
            "max_body_size": 4194304,
            "max_streaming_body_size": 52428800,
            "include_hostname_in_header": true
        }
    },
    "longRunningRoutinesTracing": {
        "enabled": false,
        "timeout": 20
    },
    "pdf_codec_default_font":{
        "name": "OpenSans-Regular.ttf"
    },
    "stats" :{
        "enabled" : true
    }
}

It is recommended that any settings that must be changed, are edited on the ESA and then pushed to the DSG nodes in the cluster. To access the gateway.json file, on the ESA Web UI, navigate to Settings > System.

log

The snippet for ’log’ is as follows:

"log": {
        "logLevel": "Warning",
        "logFacility": [
            {
                "enabled": false, 
                "facilityName": "Tunnel", 
                "logLevel": "Information"
            }, 
            {
                "enabled": false,
                "facilityName": "DiskBuffer", 
                "logLevel": "Warning"
            }, 
            {
                "enabled": false,
                "facilityName": "Admin", 
                "logLevel": "Warning"
            }, 
            {
                "enabled": false,
                "facilityName": "RuleSet", 
                "logLevel": "Verbose"
            }, 
            {
                "enabled": false,
                "facilityName": "Service", 
                "logLevel": "Warning"
            }
        ]
    },

Settings to control logging level. The following configurations are available for the log configuration:

  • logLevel: Set the logging level. The available logging levels are as follows:

    • Warning (default)
    • Info
    • Debug
    • Verbose

  • logFacility: Set the logging level for the following modules:

    • Ruleset
    • Services
    • Tunnel
    • DiskBuffer
    • Admin

  • checkErrorLogAfterCount: Decide the trimming factor that is a part of the error metrics. You can set this value in the range of -1 to 1000.

    • If the value set is greater than -1 and the log size of the error metrics is greater than 4k, then it will trim the error_metrics in such a way that all the parameters will be displayed accurately and only the row number information will be trimmed.
    • If the log size is not exceeding 4k, then the error metrics will be displayed as is.
    • If the value is set to -1 and the log size of error metrics is greater than 4k, then all the characters after the 4k limit will be trimmed from the log file.
    • If the logs are not repetitive, additional rows will be reported in separate logs. This parameter is not present in the gateway.json file. Add the checkErrorLogAfterCount parameter to enable this feature.

mountManager

Settings related to NFS mounts. The following configurations are available for the log configuration:

  • enabled: Enable or disable mount management.

  • interval: Time in seconds when the DSG node will poll the NFS shares for pulling files. You can also specify a cron job expression. The cron job format is also supported to schedule jobs. If you use the cron job expression “* * * * *”, then the DSG will poll the NFS shares at the minimum interval of one minute.

admin

Settings related to admin tunnel are listed. DSG uses this tunnel for internal communication with ESA and other DSG nodes.

  • listenAddress: Listening interface name, typically ethMNG.

  • listenPort: Port on which the interface listens to.

  • certificateFilename: Admin tunnel certificate file name with the .pem extension. The default certificates and keys are set after the DSG is installed.

  • certificateKeyFilename: Admin tunnel key file name with the .key.

  • ciphers: Colon separated list of Ciphers.

  • clientCACertificateFilename: Admin tunnel CA certificate filename with the .pem extension.

  • clientCertificateFilename: Admin tunnel client certificate filename with the .pem extension.

  • clientCertificateKeyFilename: Admin tunnel Client key file name with the .key extension.

  • commonName: Common name as defined while creating the admin tunnel client certificates.

  • ssl_options: Set the SSL options to be enforced. For a secure communication between DSG and ESA, it is recommended not to modify this option. Default value is "cert_reqs":"CERT_REQUIRED".

learnModeDefault

Settings for the Learn Mode.

  • enabled: Enable or disable Learn Mode on the DSG node. Default value it true.

  • excludeResources: Values in the field are excluded from the Learn Mode logging. Default value is \.(css|png|gif|jpg|ico|woff|ttf|svg|eot)(\?|\b).

  • excluedContentType: Content type specified in the field is excluded from the Learn Mode logging. Default value is \bcss|image|video|svg\b.

  • freeDiskSpaceThreshold: Minimum free disk space required so that the Learn Mode feature remains enabled. The feature is automatically disabled, if free disk space falls below this threshold. If the setting is disabled, then you must enable this feature manually. Default value is 1024000000.

globalUDFSettings

Settings that apply to any rules defined with custom UDF implementation for a DSG node.

  • allowed_modules: List of modules that can be used in the UDF. Default value it bs4, common.logger, re, gzip, fromstring, cStringIO,struct, traceback.

  • allowed_methods: List of methods that can be used in the UDF. Default value is BeautifulSoup, find_all, fromstring, format_exc, list, dict, str, warning.

globalProtocolStackSettings (http)

Settings for incoming HTTP requests management.

  • max_clients: Set the maximum number of concurrent outbound connections every gateway process can establish with each host. Default value is 100.

  • include_hostname_in_header: By default, the hostname will be visible in response header. Set the parameter to false, to remove the hostname from the response header. Default value is true.

  • connection_cache_ttl: Timeout value that you must configure up to which an HTTP request connection persists. Following values can be set.

    • -1: Set to enable caching (default).
    • 0: Set to disable caching.
    • : Set a value in seconds.

  • max_body_size: Maximum bytes for the HTTP request body. The datatype for this option is bytes. Default value is 4194304.

  • max_streaming_body_size: Maximum bytes for the HTTP request body when REST with streaming is enabled. The datatype for this option is bytes. Default value is 52428800.

longRunningRoutinesTracing

  • enabled: Enable or disable tracing. Default value is false.

  • timeout: Define the value in seconds to log a stack trace of processes that do not process easily in the given timeout interval. You can set the parameter to false, to remove the hostname from the response header. Default value is 20.

pdf_codec_default_font

  • name: Set the default font file to process the PDF file under the Enhanced Adobe PDF codec extract rule. Default value is OpenSans-Regular.ttf.

stats

  • enabled: Enable or disable the usage metrics. Default value is true.
Last modified February 7, 2025