Tunnels

• 1: Manage a Tunnel
• 2: Amazon S3 Tunnel
• 3: HTTP Tunnel
• 4: SFTP Tunnel
• 5: SMTP Tunnel
• 6: NFS/CIFS
• 6.1: NFS/CIFS

1 - Manage a Tunnel

Create a tunnel

You can create tunnels for custom ports that are not predefined in the DSG using the Create Tunnel option in the Tunnels tab. The Create Tunnel screen is as seen in the following figure.

The following table provides the description for each option available on the UI.

The following types of tunnels can be created.

• HTTP
• SFTP
• SMTP
• Amazon S3
• CIFS/NFS

Edit a tunnel

Edit an existing tunnel configuration using the Edit option in the Tunnels tab. The Edit Tunnel screen is as seen in the following figure.

After editing the required field, click Update to save your changes.

Delete a tunnel

Delete an existing tunnel using the Delete option in the Tunnels tab. The Delete Tunnel screen is shown in the following figure.

The following table provides the description for each option available on the UI.

2 - Amazon S3 Tunnel

Amazon Simple Storage Service (S3) is an online file storage web service. It lets you manage files through browser-based access as well as web services APIs. In DSG, the S3 tunnel is used to communicate with Amazon S3 cloud storage over the Amazon S3 REST API. The higher-layer S3 Service object, which sits above the tunnel object, configured at the RuleSet level is used to process file contents retrieved from S3.

A sample S3 tunnel configuration is shown in the following figure.

Amazon S3 uses buckets to store data and data is classified as objects. Each object is identified with a unique key ID. Consider an example that john.doe is the bucket and incoming is a folder under john.doe bucket. Assuming the requirement is that files landing in the incoming folder should be picked up and processed by DSG nodes. The data pulled from the AWS online storage is available in the incoming folder under the source bucket. The Amazon S3 Service is used to perform data security operation on this data in the source bucket.

Note: The DSG supports four levels of nested folders in an Amazon S3 bucket.

After the rules are executed, the processed data may be stored in a separate bucket (e.g. the folder named outgoing under the same john.doe bucket), which is the target bucket. When the DSG nodes poll AWS for a file uploaded, whichever node accesses the file first places a lock on the file. You can specify if the lock files must be stored in a separate bucket or under the source bucket. If the file is locked, the other DSG nodes will stop trying to access the file.

If the data operation on a locked file fails, the lock file can be viewed for detailed log and error information. The lock files are automatically deleted if the processing completes successfully.

Consider the scenario where an incoming bucket contains two directories Folder1 and Folder2.

The DSG allows multiprocessing of files that are place in the bucket. The lock files are created for every file processed. In the scenario mentioned, the lock files are created as follows:

• If the abc.csv file of Folder1 is processed, the lock file is created as Folder1.abc.csv.<hostname>.<Process ID>.lock.
• If the pqr.csv file of Folder2 is processed, the lock file is created as Folder1.pqr.csv.<hostname>.<Process ID>.lock.

Consider the following figure where files are nested in the S3 bucket.

The lock files are created as follows:

• If the abc.csv file of Folder1 is processed, the lock file is created as Folder1.abc.csv.<hostname>.<Process ID>.lock.
• If the pqr.csv file of Folder2 is processed, the lock file is created as Folder1.Folder2.pqr.csv.<hostname>.<Process ID>.lock.
• If the abc.csv file of Folder3 is processed, the lock file is created as Folder1.Folder2.Folder3.abc.csv.<hostname>.<Process ID>.lock.

If the multiprocessing of files is to be discontinued, remove the enhanced-lock-filename flag from the features.json file available in the System > Files on the DSG Web UI.

The following image illustrates the options available for an S3 tunnel.

The options specific to the S3 Protocol type are described as follows:

Bucket list settings

1 Source Bucket Name: Bucket name as defined in AWS where the files that need to be processed are available.

2. Source File Name Pattern: Regex pattern for the filenames to be processed. For example, .csv.

Rename Processed Files: Regex logic for renaming processed file.

3. Match Pattern: Regex logic for renaming processed file.

4. Replace Value: Value to append or name that will be used to rename the original source file based on the pattern provided and grouping defined in regex logic.

5. Overwrite Target Object: Select to overwrite a file in the bucket with a newly processed file of the same name. Refer to Amazon S3 Object.

6. Lock Files Bucket: Name of the lock files folder, if you want the lock files to be stored in a separate bucket. If not defined, lock files are placed in the source bucket.

7. Interval: Time in secs when the DSG node will poll AWS for pulling files. You can also specify a cron job expression. Refer to Cron documentation. The default value is 5. If you use the cron job expression “* * * * *”, DSG will poll AWS at the minimum interval of one minute.
Cron job format is also supported to schedule jobs.

AWS Settings

8. AWS Access Key Id: Access key id used to make secure protocol request to an AWS service API. Refer to Amazon Web Service documentation.

9. AWS Secret Access Key: Secret access key related to the access key id. The access key id and secret access key work together to sign into AWS and provide access to resources. Refer to Amazon Web Service documentation.

10. AWS Endpoint URL: Specify the endpoint URL if it is other than the amazon S3 bucket. This parameter should only be configured if the user is using DSG to connect to other endpoint than amazon S3 bucket i.e. On-Premise S3, Google Cloud Bucket, and so on.If not defined, the DSG will connect to Amazon S3 bucket.

11. Path to CA Bundle: Specify the path to CA bundle if the endpoint is other than Amazon S3 bucket. If the user has installed the S3 on-premise using the self signed certificate, then specify that path to CA bundle in this parameter. If the endpoint URL is Amazon S3 bucket, then by default it uses SSL certificate to connect to S3 bucket.

Advanced Settings

12. Advanced settings: Set additional advanced options for tunnel configuration, if required, in the form of JSON in the following textbox. In a scenario where an ESA and two DSG nodes are in a cluster, by using the Selective Tunnel Loading functionality, you can load specific tunnel configurations on specific DSG nodes.

The advanced settings that can be configured for S3 Protocol.

Using S3 tunnel to access files on Google Cloud Storage

Similar to AWS buckets, data is also stored on the Google Cloud Storage can also be protected. You can use the S3 tunnel to access the files on the GCP storage. The incoming and processed file has to be placed in the same storage in separate folders. For example, a storage named john.doe bucket contains a folder incoming that contains files to be picked and processed by DSG nodes. This acts as the source bucket. After the rules are executed, the data is stored in the processed bucket. Ensure the following points are considered.

• AWS Endpoint URL contains the URL of the Google Cloud storage.
• AWS Access Key ID and AWS Secret Access Key contain the secret ID and HMAC keys.

Refer to Google docs for information about Access ID and HMAC keys.

3 - HTTP Tunnel

Based on the protocol selected, the dependent fields in the Tunnel screen vary. The following image illustrates the settings that are specific to the HTTP protocol.

The options for the Inbound Transport Settings field in the Tunnel Details screen specific to the HTTP Protocol type are described in the following table.

Network Settings

1 Listening Interface: IP address through which sensitive data enters the DSG. The following Listening Interface options are available:

• ethMNG: The management interface on which the DSG Web UI is accessible.
• ethSRV0: The service interface for communicating with an untrusted service.
• 127.0.0.1: The local loopback adapter.
• 0.0.0.0: The broadcast address for listening to all the available network interfaces.
• Other: Manually add a listening address based on your requirements.

2 Port:Port linked to the listening address.

TLS/SSL Security Settings

3 TLS Enabled: Select to enable TLS features.

4 Certificate: Certificate applicable for a tunnel.

5 Cipher Suites: Colon separated list of Ciphers.

6 TLS Mutual Authentication: CERT_NONE is selected as default. Use CERT_OPTIONAL to validate if a client certificate is provided or CERT_REQUIRED to process a request only if a client certificate is provided. If TLS mutual authentication is set to CERT_OPTIONAL or CERT_REQUIRED, then the CA certificate must be provided.

7 CA Certificates: A CA certificate chain. This option is applicable only if the value client certificate is set to 1 (optional) or 2 (required). Client certificates can be requested at the tunnel and the RuleSet level for authentication. On the Tunnels screen, you can configure the ca_reqs parameter in the Inbound Transport Settings field to request the client certificate. Similarly, on the Ruleset screen, you can toggle the Required Client Certificate checkbox to enable or disable client certificates. Based on the combination of the options in the tunnel and the RuleSet, the server executes the transaction. If the certificate is incorrect or not provided, then server returns a 401 error response.

The following table explains the combinations for the client certificate at the tunnel and the RuleSet level.

8 DH Parameters: The .pem filename that includes the DH parameters. Upload the .pem file from the Certificate/Key Material screen. The Diffie-Hellman (DH) parameters define the way OpenSSL performs the DH Key exchange.

9 ECDH Curve Name: Supported curve names for the ECDH key exchange.The Elliptic curve Diffie–Hellman (ECDH) protocol allows key agreement and leverages elliptic-curve cryptography (ECC) properties for enhanced security.

10 Certificate Revoke List: Path of the Certificate Revocation List (CRL) file. For more information about CRL error message that appears when a revoked certificate is sent, refer to the CRL error. The ca.crl.pem file includes a list of certificates that are revoked. Based on the flags that you provide in the verify_flags setting, SSL identifies certificate verification operations that need to performed. The CRL verification operations can be VERIFY_CRL_CHECK_LEAF or VERIFY_CRL_CHECK_CHAIN.

When you try to access the DSG through HTTPS using such a revoked certificate, the DSG returns the following error message.

11 Verify Flags Set to one of the following operations to verify the CRL:

• VERIFY_DEFAULT
• VERIFY_X509_TRUSTED_FIRST
• VERIFY_CRL_CHECK_LEAF
• VERIFY_CRL_CHECK_CHAIN

12 SSL Options|Set the required flags that reflect the TLS behavior at runtime. A single flag or multiple flags can be used.It is used to define the supported SSL options in the JSON format. The DSG supports TLS v1.2.|For example, in the following JSON, TLSv1 and TLSv1.1 are disabled.
{
“options”: [“OP_NO_SSLv2”,
“OP_NO_SSLv3”,
“OP_NO_TLSv1”,
“OP_NO_TLSv1_1”]
}|

13 Advanced Settings Set additional advanced options for tunnel configuration, if required, in the form of JSON.|In a scenario where an ESA and two DSG nodes are in a cluster, by using the Selective Tunnel Loading functionality, you can load specific tunnel configurations on specific DSG nodes.

Generating ECDSA certificate and key

The dh_params parameter points to a .pem file. The .pem file includes the DH parameters that are required to enable DH key exchange for improved protection without compromising computational resources required at each end. The value accepted by this field is the file name with the extension (.pem). The DSG supports both RSA certificates and Elliptic Curve Digital Signature Algorithm (ECDSA) certificates for the ECDHE protocol. The RSA certificates are available as default when the DSG is installed, while to use ECDSA certificates in the DSG, you must generate an ECDSA certificate and the related key. The following procedure explains how to generate the ECDSA certificate and key.

To generate dhparam.pem file:

1. Set the SSL options in the Inbound Transport settings as given in the following example.

   • DH Parameters: /opt/protegrity/alliance/config/dhparam/dhparam.pem
   • ECDH Curve Name: prime256v1
   • SSL Options: OP_NO_COMPRESSION

2. From the ESA CLI Manager, navigate to Administration > OS Console.

3. Execute the following command to generate the dhparam.pem file.

   openssl dhparam -out /opt/protegrity/alliance/config/dhparam/dhparam.pem 2048
   

Note: Ensure that you create the dhparam directory in the given path. The path /opt/protegrity/alliance/config/dhparam is the location where you want to save the .pem file. The value 2048 is the key size.

4. Execute the following command to generate the key.

openssl genpkey -paramfile dhparam.pem -out dhkey.pem

The ecdh_curve_name parameter is the curve type that is required for the key exchange. The OpenSSL curves that are supported by DSG are listed in Supported OpenSSL Curve Names.

You can configure additional inbound settings that apply to HTTP from the Global Settings page on the DSG Web UI.

4 - SFTP Tunnel

Based on the protocol selected, the dependent fields in the Tunnel screen vary. The following image illustrates that settings specific to SFTP protocol.

The options specific to the SFTP Protocol type are described in the following table.

**-The advanced settings that can be configured for SFTP Protocol.

The following snippet describes the example format for the SFTP Advanced settings:

{
   "idle_connection_timeout": 30,
   "default_window_size": 2097152,
   "default_max_packet_size": 32768,
   "use_compression": true,
   "ciphers": [
      "aes128-ctr",
      "aes256-ctr",
      "3des-cbc"
   ],
   "kex": [
      "diffie-hellman-group14-sha1",
      "diffie-hellman-group-exchange-sha1"
   ],
   "digests": [
      "hmac-sha1"
   ]
}

5 - SMTP Tunnel

The DSG can perform data security operations on the sensitive data sent by an Simple Mail Transfer Protocol (SMTP) client before the data reaches the destination SMTP server.

Over the internet, SMTP is an Internet standard for sending emails. When an email is sent to anyone, the email is sent using an SMTP client to the SMTP server. For example, if an email is sent from john.doe@xyz.com to jane.smith@abc.com, the email first reaches the xyz’s SMTP server, then reaches abc’s SMTP server, before it finally reaches the recipient, jane.smith@abc.com.

The DSG intercepts the communication between the SMTP client and server and performs data security operations on sensitive data. The sensitive data residing in the email elements, such as subject of an email, body of an email, attachments, filename, and so on, are supported for the SMTP protocol:

When the DSG is used as an SMTP gateway, the Rulesets must use the SMTP service and the first child Extract rule must be SMTP Message.

The following image illustrates how the SMTP protocol is handled in the DSG. Consider an example where, john.doe@xyz.com is sending an email to jane.smith@xyz.com. The xyz SMTP server is the same for the sender and the recipient.

1. The sender, john.doe@xyz.com, sends an email to the recipient, jane.smith@xyz.com. The Subject of the email contains sensitive data that must be protected before it reaches the recipient.

2. The DSG is configured with an SMTP tunnel such that it listens for incoming requests on the listening ports. The DSG is also configured with Rulesets such that an Extract rule extracts the Subject from the request. The Extract rule also defines a regex that extracts the sensitive data and passes it to the Transform rule. The Transform rule performs data security operations on the sensitive data.

   The DSG forwards the email with the protected data in the Subject to the SMTP server.

3. The recipient SMTP client polls the SMTP server for any emails. The email is received and the sensitive data in the Subject appears protected.

The following image illustrates the settings specific to the SMTP protocol.

The options specific to the SMTP Tunnel are described in the following table.

The ssl_options supported for the SMTP Tunnel are described in the following table.

* The following Listening Interface options are available:

• ethMNG: The management interface on which the DSG Web UI is accessible.
• ethSRV0: The service interface where the DSG listens for the incoming SMTP requests.
• 127.0.0.1: The local loopback adapter.
• 0.0.0.0: The broadcast address for listening to all the available network interfaces.
• Other: Manually add a listening address based on your requirements.

**-The advanced settings that can be configured for SMTP Protocol.

6 - NFS/CIFS

Though the files are accessed remotely, the behavior is same as when files are accessed locally. The NFS file system follows a client/server model, where the server is responsible for authentication and permissions, while the client accesses data through the local disk systems.

A sample NFS/CIFS tunnel configuration is shown in the following figure.

Note: The Address format for an NFS tunnel is <ip address/hostname>:<mount_path> and for a CIFS tunnel is \\<ip address or hostname>\<share_path>.

Consider an example NFS/CIFS server with folder structure that includes folders namely, /input, /output, and /lock. When a client accesses the NFS/CIFS server, the files are stored in the input folder. The Mounted File System Out-of-Band Service is used to perform data security operation on the files in the /input folder. A source file is processed only when a corresponding trigger file is created and found in the /input folder.

Note: Ensure that the trigger file time stamp is greater than or equal to the source file time stamp.

After the rules are executed, the processed files can be stored in a separate folder, such as in this example, /output. When DSG nodes poll NFS/CIFS server for a file uploaded, whichever node accesses the file first places a lock on the file. You can specify if the lock files must be stored in a separate bucket, such as /lock or under the source folder. If the file is locked, the other DSG nodes will stop trying to access the file.

If the data operation on a locked file fails, the lock file can be viewed for detailed log and error information. The lock files are automatically deleted if the processing completes successfully.

6.1 - NFS/CIFS

The options for the NFS tunnel as illustrated in the following figure

Mount settings

1 Mount Point - The Address format for an NFS tunnel is <IP address/hostname>:<mount_path>

2 Input Directory - The mount tunnel forwards the files present in this directory for further processing. This directory structure will be defined in the NFS/CIFS share.

3 Source File Name Pattern - Regex logic for identifying the source files that must be processed.

4 Overwrite Target File - Select to overwrite a file in the bucket with the newly processed file with the same name.

Rename processed files

Regex logic for renaming original source files after processed files are generated

5 Match Pattern - Exact pattern to match and filter the file.

6 Replace Value - Value to append or name that will be used to rename the original source file based on the pattern provided and grouping defined in regex logic.

Trigger File

File that triggers the rule. The rule will be triggered, only if this file is found to exist in the input directory. Files in the NFS/CIFS Share directory are not processed until the trigger criteria is met. Ensure that the trigger file is sent only after the files that need to be processed are placed in the source directory. After the trigger file is placed, you must touch the trigger file.

7 Trigger File Name Pattern - Identifier that will be appended to each source file to create a trigger control file. Consider a source file abc.csv, if you define the identifier as %.ctl, you must create a trigger file abc.csv.ctl to ensure that the source file is processed.

It is mandatory to provide a trigger file for each source file to ensure that it is processed. Files without a corresponding trigger file will not be processed.

The *, [, and ] characters are not accepted as part of the trigger file pattern.

8 Delete Trigger File - Enable to delete the trigger file after the source file is processed.

9 Lock Files Directory - Directory where the lock files will be stored. If this value is not provided as per the directory structure in the NFS/CIFS share, then the lock files will be stored in the mount point. Ensure that the lock directory name does not include spaces. The DSG will not process files under the lock directory that includes spaces.

10 Error Files directory - Files that fail to process are moved to this directory. The lock files generated for such files are also moved to this directory. For example, the file is moved from the /input directory to the /error directory.

11 Error Files Extension - Extension that will be appended to each error file. If you do not specify an extension, then the .err extension will be used.

Mount Options

Parameters that will be used to mount the share.

12 Mount Type - Specify Soft if you want the mount point to report an error, if the server is unreachable after wait time crosses the Mount Timeout value. If you select Hard, ensure that the Interrupt Timeout checkbox is selected.

13 Mount Timeout - Number in seconds after which an error is reported. Default value is 60.

14 Options - Additional NFS options that can be provided as inbound settings. If the lock directory is not defined, then the lock files are automatically placed in the /input directory. For example, {"port":1234, "nolock" "nfsvers": 3}. To enable enhanced security for the mounted share, it is recommended that the following options are set:

noexec,nosuid,nodev

where:

• noexec: Disallow execution of executable binaries on the mounted file system.
• nosuid: Disallow creation of set user id files on the file system.
• nodev: Disallow mounting of special devices, such as, USB, printers, etc.

15 Advanced Settings - Set additional advanced options for tunnel configuration, if required, in the form of JSON in the Advanced Settings textbox. For example, {"interval":5, "fileChunkSize": 4096}. In a scenario where an ESA and two DSG nodes are in a cluster, by using the Selective Tunnel Loading functionality, you can load specific tunnel configurations on specific DSG nodes.

Note: Ensure that the NFS share options are configured in the exports configuration file for each mount that the DSG will access. The all_squash option must be set to specify the anonuid and anongid with the user ID and group ID of the non-root user respectively.

This prevents the DSG from changing user and group permissions of the mount directories on the NFS server.