Protectors

• 1: Configuration Parameters for Protector
• 2: Installing Resilient Package (RP) Agent on Linux or Unix and Windows
• 3: Installing Resilient Package (RP) Proxy on Linux or Unix and Windows
• 4: Configuration Parameters for RPP outside of the ESA
• 5: Installing Log Forwarder on Linux and Windows
• 5.1: Configuring the disk space on the Log Forwarder
• 6: Memory Utilization in Resilient Protectors
• 7: Verification of Signed Protector Build
• 8: Protection Method Reference
• 8.1: Protegrity Tokenization
• 8.1.1: Tokenization Support by Protegrity Products
• 8.1.2: Delimiters
• 8.1.3: Tokenization Properties
• 8.1.3.1: Data Type and Alphabet
• 8.1.3.2: Static Lookup Table (SLT) Tokenizers
• 8.1.3.3: From Left and From Right Settings
• 8.1.3.4: Internal Initialization Vector (IV)
• 8.1.3.5: Minimum and Maximum Input Length
• 8.1.3.5.1: Calculating Token Length
• 8.1.3.6: Length Preserving
• 8.1.3.7: Short Data Tokenization
• 8.1.3.8: Case-Preserving and Position-Preserving Tokenization
• 8.1.3.8.1: Case-Preserving Tokenization
• 8.1.3.8.2: Position-Preserving Tokenization
• 8.1.3.9: External Initialization Vector (EIV)
• 8.1.3.9.1: Tokenization Model with External IV
• 8.1.3.9.2: External IV Tokenization Properties
• 8.1.3.10: Truncating Whitespaces
• 8.1.4: Tokenization Types
• 8.1.4.1: Numeric (0-9)
• 8.1.4.2: Integer (0-9)
• 8.1.4.3: Credit Card
• 8.1.4.4: Alpha (A-Z)
• 8.1.4.5: Upper-Case Alpha (A-Z)
• 8.1.4.6: Alpha-Numeric (0-9, a-z, A-Z)
• 8.1.4.7: Upper-Case Alpha-Numeric (0-9, A-Z)
• 8.1.4.8: Lower ASCII
• 8.1.4.9: Datetime (YYYY-MM-DD HH:MM:SS)
• 8.1.4.10: Decimal
• 8.1.4.11: Unicode Gen2
• 8.1.4.12: Binary
• 8.1.4.13: Email
• 8.1.4.14: Printable
• 8.1.4.15: Date (YYYY-MM-DD, DD/MM/YYYY, MM.DD.YYYY)
• 8.1.4.16: Unicode
• 8.1.4.17: Unicode Base64
• 8.1.5:
• 8.1.6:
• 8.2: Protegrity Format Preserving Encryption
• 8.2.1: FPE Properties
• 8.2.2: Code Points
• 8.2.3: Tweak Input
• 8.2.4: Left and Right Settings
• 8.2.5: Handling Special Numeric Credit Card Data
• 8.3: Protegrity Encryption
• 8.3.1: Encryption Algorithms
• 8.3.1.1: AES-128 and AES-256
• 8.3.1.2: CUSP
• 8.3.1.3: 3DES
• 8.3.2: Encryption Properties - IV, CRC, Key ID
• 8.3.3: Data Length and Padding in Encryption
• 8.3.4:
• 8.3.5:
• 8.3.6:
• 8.4: No Encryption
• 8.5: Monitoring
• 8.6: Masking
• 8.7: Hashing
• 8.8: ASCII Character Codes
• 8.9: Examples of Column Sized Calculation for AES and 3DES Encryption
• 8.10: Empty String Handling by Protectors
• 8.11: Hashing Functions and Examples
• 8.11.1: Hash Data column size
• 8.11.2: Using Hashing Triggers and View
• 8.12: Codebook Re-shuffling in the Data Security Gateway
• 8.13:
• 8.14:
• 8.15:
• 8.16:
• 9: Application Protector
• 9.1: Application Protector Java
• 9.1.1: Understanding the Architecture
• 9.1.2: System Requirements
• 9.1.3: Preparing the Environment
• 9.1.4: Installing the AP Java Protector
• 9.1.5: Configuring the Protector
• 9.1.6: Application Protector Java APIs
• 9.1.6.1: Using the AP Java APIs
• 9.1.7: Additional Topics
• 9.1.7.1: Uninstalling the Application Protector
• 9.1.7.2: Memory Usage of the AP Java
• 9.1.7.3: DevOps Approach for Application Protector Java
• 9.1.7.4: Application Protector API Return Codes
• 9.1.7.5: Config.ini file for Application Protector
• 9.1.7.6: Multi-node Application Protector Architecture
• 9.2: Application Protector Python
• 9.2.1: Understanding the Architecture
• 9.2.2: System Requirements
• 9.2.3: Preparing the Environment
• 9.2.4: Installing the AP Python Protector
• 9.2.5: Configuring the Protector
• 9.2.6: Application Protector Python APIs
• 9.2.6.1: Using the AP Python APIs
• 9.2.7: Additional Topics
• 9.2.7.1: Uninstalling the Application Protector
• 9.2.7.2: Memory Usage of the AP Python
• 9.2.7.3: Setting Up AP Python on Linux in a Development Environment
• 9.2.7.4: DevOps Approach for Application Protector Python
• 9.2.7.5: Application Protector API Return Codes
• 9.2.7.6: Config.ini file for Application Protector
• 9.2.7.7: Multi-node Application Protector Architecture
• 9.3: Application Protector .Net
• 9.3.1: Architecture and Workflow
• 9.3.2: System Requirements
• 9.3.3: Preparing the Environment
• 9.3.4: Installing the AP .Net Protector
• 9.3.5: Configuring the Protector
• 9.3.6: Using the AP .Net APIs
• 9.3.6.1: Application Protector .Net APIs
• 9.3.7: Additional Topics
• 9.3.7.1: Uninstalling the Application Protector
• 9.3.7.2: Memory Usage of the AP .Net
• 9.3.7.3: Setting Up AP .Net Mock on Windows in a Development Environment
• 9.3.7.4: DevOps Approach for Application Protector
• 9.3.7.5: Application Protector API Return Codes
• 9.3.7.6: Config.ini file for Application Protector
• 9.3.7.7: Multi-node Application Protector Architecture
• 10: Big Data Protector
• 10.1: Amazon EMR
• 10.1.1: Understanding the architecture
• 10.1.1.1: Bootstrap installer architecture
• 10.1.1.2: Static installer architecture
• 10.1.2: Preparing the environment
• 10.1.2.1: Setting up for the Bootstrap Installer
• 10.1.2.1.1: Verifying the prerequisites
• 10.1.2.1.2: Extracting the Big Data Protector Package
• 10.1.2.1.3: Executing the Configurator Script
• 10.1.2.2: Setting up for the Static Installer
• 10.1.2.2.1: Verifying the prerequisites for Static Installer
• 10.1.2.2.2: Extracting the Installation Package
• 10.1.2.2.3: Updating the BDP.Config File
• 10.1.3: Installing the protector
• 10.1.3.1: Using the Bootstrap Installer
• 10.1.3.1.1: Creating a Cluster
• 10.1.3.1.2: Managing the Cluster Nodes
• 10.1.3.1.3: Verifying the Parameters
• 10.1.3.2: Using the Static Installer
• 10.1.3.2.1: Installing the Protector on all the Nodes
• 10.1.3.2.2: Installing the Protector on Specific Nodes
• 10.1.3.2.3: Verifying the Parameters
• 10.1.4: Configuring the protector
• 10.1.5: Working with Cluster Utilities
• 10.1.5.1: RPAgent Control Script
• 10.1.5.2: Log Forwarder Control Script
• 10.1.5.3: Sync Config.ini
• 10.1.5.4: Sync Log Forwarder Configuration
• 10.1.5.5: Sync RPAgent Configuration
• 10.1.6: Uninstalling the protector
• 10.1.6.1: Uninstalling the Big Data Protector when Bootstrap is used
• 10.1.6.2: Uninstalling the Big Data Protector when Static installer is used
• 10.1.6.2.1: From all the Nodes
• 10.1.6.2.2: From Specific Nodes
• 10.2: AWS Databricks
• 10.2.1: Understanding the architecture
• 10.2.1.1: For the Application Protector REST Approach
• 10.2.1.2: For the Cloud Protector Approach
• 10.2.2: System Requirements
• 10.2.2.1: For the Application Protector REST Approach
• 10.2.2.2: For the Cloud Protector Approach
• 10.2.3: Preparing the Environment
• 10.2.3.1: Extracting the Installation Package
• 10.2.3.2: Working with the Configurator Script
• 10.2.3.3: Retrieving the IP Address
• 10.2.3.4: Uploading the Secrets
• 10.2.4: Installing the Protector
• 10.2.4.1: Creating the User Defined Functions
• 10.2.5: Configuring the Protector
• 10.2.5.1: Editing the Cluster Configuration
• 10.2.6: Uninstalling the Protector
• 10.2.6.1: Dropping the User Defined Functions
• 10.3: CDP-PVC-Base
• 10.3.1: Understanding the architecture
• 10.3.2: System Requirements
• 10.3.3: Preparing the Environment
• 10.3.3.1: Extracting the installation package
• 10.3.3.2: Running the configurator script
• 10.3.3.3: Distributing the parcels
• 10.3.3.4: Activating the parcels
• 10.3.4: Installing the Protector
• 10.3.4.1: Installing the parcel on a new node
• 10.3.4.2: Starting the Big Data Protector service
• 10.3.4.3:
• 10.3.5: Configuring the Protector
• 10.3.5.1: Updating the Parcels
• 10.3.5.1.1: Updating the Certifcate Parcels with a Restart
• 10.3.5.1.2: Updating the Certificate Parcels without a Restart
• 10.3.5.1.3: Updating the Log Forwarder Parcel
• 10.3.5.2: Setting the Configurations
• 10.3.5.2.1: Updating Parameters in the config.ini file
• 10.3.5.2.2: Updating Parameters for the RP Agent
• 10.3.5.2.3: Updating Parameters for the Log Forwarder
• 10.3.5.2.4: Adding a new configuration parameter
• 10.3.5.2.5: Setting the Big Data Protector configuration
• 10.3.5.2.6: Enabling the application log file
• 10.3.5.3: Creating the User Defined Functions
• 10.3.5.3.1: Registering and dropping the Hive UDFs
• 10.3.5.3.2: Registering the Spark UDFs
• 10.3.5.3.3: Registering the Impala UDFs
• 10.3.5.3.4: Installing the Impala UDFs
• 10.3.6: Uninstalling the Protector
• 10.3.6.1: Dropping the User Defined Functions
• 10.3.6.2: Uninstalling the Impala UDFs
• 10.3.6.3: Restoring the Big Data Protector configuration
• 10.3.6.4: Removing the Big Data Protector Services
• 10.3.6.5: Deactivating the parcels
• 10.3.6.6: Removing the parcels
• 10.3.6.7: Deleting the parcels from the local repository
• 10.3.6.8: Deleting the CSD files
• 10.4: CDP AWS DataHub
• 10.4.1: Understanding the architecture
• 10.4.2: System Requirements
• 10.4.3: Preparing the Environment
• 10.4.3.1: Extracting the package
• 10.4.3.2: Executing the configurator script
• 10.4.3.3: Registering the Recipe Scripts
• 10.4.3.4: Creating and Registering the Custom Cluster Template
• 10.4.4: Installing the Big Data Protector
• 10.4.4.1: Creating the DataHub Cluster
• 10.4.5: Configuring the Big Data Protector
• 10.4.5.1: Setting the Parameters Manually
• 10.4.5.2: Setting the Parameters using the Helper Script
• 10.4.5.3: Installing the UDFs using the Helper Script
• 10.4.5.4: Updating the Configuration Parameters
• 10.4.5.5: Updating the Certificates Parcel
• 10.4.5.6: Updating the Log Forwarder Parcel
• 10.4.6: Uninstalling the Big Data Protector
• 10.4.6.1: Drop the UDFs using the Helper Script
• 10.4.6.2: Restoring the Parameters using the Helper Script
• 10.5: User Defined Functions and APIs
• 10.5.1: MapReduce APIs
• 10.5.2: Hive UDFs
• 10.5.3: Pig UDFs
• 10.5.4: HBase Commands
• 10.5.5: Impala UDFs
• 10.5.6: Spark Java APIs
• 10.5.7: Spark SQL UDFs
• 10.5.8: PySpark - Scala Wrapper UDFs
• 10.5.9: Unity Catalog Batch Python UDFs
• 10.6: Additional Information
• 10.6.1: Migrating Tokenized Unicode Data
• 10.6.2: Return Codes for the Big Data Protector
• 11: Data Warehouse Protectors
• 11.1: Deploying the Data Warehouse Protectors
• 11.2: Teradata Data Warehouse Protector
• 11.2.1: Understanding the Architecture
• 11.2.2: System Requirements
• 11.2.3: Preparing the Environment
• 11.2.3.1: Extracting the Teradata Installation Package
• 11.2.3.2: Installing the Log Forwarder
• 11.2.3.3: Installing the Resilient Package Agent
• 11.2.4: Installing the Protector
• 11.2.4.1: Installing the Teradata Objects
• 11.2.4.2: Creating the Teradata User Defined Functions (UDFs)
• 11.2.4.3: Installing the Teradata User Defined Types (UDTs)
• 11.2.4.4: Creating the Teradata User Defined Types (UDTs)
• 11.2.5: Configuring the Protector
• 11.2.5.1: Working with the config.ini file
• 11.2.5.1.1: Accessing the config.ini File
• 11.2.5.1.2: Understanding the Parameters in the config.ini File
• 11.2.5.2: Updating the Output Buffer for the Unicode UDFs
• 11.2.5.2.1: Updating the dbpuserconf.ini file
• 11.2.5.2.2: Updating the createvarcharunicode.sql file
• 11.2.6: Uninstalling the Protector
• 11.2.6.1: Uninstalling the Log Forwarder
• 11.2.6.2: Uninstalling the RPAgent
• 11.2.6.3: Uninstalling the User Defined Functions (UDFs)
• 11.2.6.4: Removing the Installation Directory
• 11.3: User Defined Functions and APIs
• 11.3.1: Teradata UDFs
• 11.3.1.1: General UDFs
• 11.3.1.2: Access Check UDFs
• 11.3.1.3: Varchar Latin UDFs
• 11.3.1.4: Varchar Unicode UDFs
• 11.3.1.5: Float UDFs
• 11.3.1.6: Small Integer UDFs
• 11.3.1.7: Integer UDFs
• 11.3.1.8: Big Integer UDFs
• 11.3.1.9: Date UDFs
• 11.3.1.10: 8-Byte and 16-Byte Decimal UDFs
• 11.3.1.11: JSON UDFs
• 11.3.1.12: XML UDFs
• 11.3.1.13: Float UDFs for No Encryption
• 11.3.1.14: Date UDFs for No Encryption
• 11.3.1.15: 8-Byte AND 16-Byte Decimal UDFs for No Encryption
• 11.3.2: Trino User Defined Functions and Procedures
• 11.3.2.1: General UDFs
• 11.3.2.2: VarChar UDFs
• 11.3.2.3: BigInt UDFs
• 11.3.2.4: SmallInt UDFs
• 11.3.2.5: Integer UDFs
• 11.3.2.6: Date UDFs
• 11.3.2.7: DateTime UDFs
• 11.3.2.8: VarChar Encryption UDFs
• 11.3.2.9: Unicode UDFs
• 11.3.2.10: Decimal UDFs
• 11.3.2.11: Double UDFs
• 11.3.2.12: VarBinary Encryption UDFs
• 11.4: Appendix
• 11.4.1: Additional references for the Protectors
• 11.4.1.1: Additional references for the Teradata Protector
• 11.4.1.1.1: Configuring access to execute queries
• 11.4.1.1.2: Teradata Query Bands and Trusted Sessions
• 11.4.2: Data Warehouse Sample Scripts
• 11.4.2.1: Teradata Database Data Warehouse Sample Scripts
• 11.4.3: Return Codes for Data Warehouse Protectors
• 11.4.4: Supported Data Warehouse Protectors Matrix
• 11.5: Trino Data Warehouse Protector
• 11.5.1: Understanding the Architecture
• 11.5.2: System Requirements
• 11.5.3: Preparing the Environment
• 11.5.3.1: Extracting the Files from the Installation Package
• 11.5.3.2: Executing the Configurator Script
• 11.5.4: Installing the Trino Protector
• 11.5.5: Configuring the Trino Protector
• 11.5.5.1: Working with Cluster Utilities
• 11.5.5.1.1: Log Forwarder Control Script
• 11.5.5.1.2: RPAgent Control Script
• 11.5.5.1.3: Sync Config.ini
• 11.5.5.1.4: Sync RPAgent
• 11.5.5.1.5: Sync Log Forwarder
• 11.5.6: Uninstalling the Trino Protector
• 12: Database Protector
• 12.1: Oracle Database Protector
• 12.1.1: Understanding the Architecture
• 12.1.2: System Requirements
• 12.1.3: Preparing the Environment
• 12.1.3.1: Extracting the Installation Package
• 12.1.3.2: Installing the Log Forwarder
• 12.1.3.3: Installing the RPAgent
• 12.1.4: Installing the Oracle Database Protector
• 12.1.4.1: Installing the Policy Enforcement Point (PEP)
• 12.1.4.2: Creating the User Defined Functions (UDFs)
• 12.1.5: Configuring the Oracle Database Protector
• 12.1.5.1: User Impersonation
• 12.1.5.2: Enterprise User Security (EUS) in the Oracle Database
• 12.1.6: Uninstalling the Oracle Database Protector
• 12.2: User Defined Functions and APIs
• 12.2.1: Oracle User Defined Functions and APIs
• 12.2.1.1: General UDFs
• 12.2.1.2: Access Check Procedures
• 12.2.1.3: Insert Encryption UDFs
• 12.2.1.4: Insert No-Encryption, Token, and FPE UDFs
• 12.2.1.5: Multiple Insert Encryption Procedures
• 12.2.1.6: Select Decryption UDFs
• 12.2.1.7: Select No-Encryption, Token, and FPE UDFs
• 12.2.1.8: Update Encryption UDFs
• 12.2.1.9: Update No-Encryption, Token, and FPE UDFs
• 12.2.1.10: Multiple Update Encryption Procedures
• 12.2.1.11: Hash UDFs
• 12.2.1.12: Blob UDFs
• 12.2.1.13: Clob UDFs
• 12.2.1.14: Bulk UDFs
• 12.2.1.15: Oracle Input Datatype to UDF Mapping
• 13: REST Container
• 13.1: Understanding the Architecture
• 13.1.1: Architecture and Components using Dynamic-based Deployment
• 13.1.2: Architecture and Components using Static Deployment
• 13.2: System Requirements
• 13.2.1: Software Requirements
• 13.2.2: Hardware Requirements
• 13.3: Preparing the Environment
• 13.3.1: Initializing the Jump Box
• 13.3.2: Extracting the Installation Package
• 13.3.3: Creating Certificates
• 13.3.4: Uploading the Images to the Container Repository
• 13.3.5: Creating the AWS Environment
• 13.3.5.1: Creating the AWS Setup for Static Mode
• 13.3.5.1.1: Creating an Data Encryption Key (DEK)
• 13.3.5.1.2: Creating an AWS S3 Bucket
• 13.3.5.1.3: Creating an AWS EFS
• 13.3.5.2: Creating a Kubernetes Cluster
• 13.4: Installing the Protector
• 13.4.1: Deploying REST Container for Dynamic Method
• 13.4.1.1: Deploying Log Forwarder
• 13.4.1.2: Deploying Resilient Package Proxy (RPP)
• 13.4.1.3: Deploying the REST Container
• 13.4.2: Deploying REST Product in Static Mode
• 13.4.2.1: Retrieving the Policy Package from the ESA
• 13.4.2.2: Deploying Log Forwarder
• 13.4.2.3: Deploying KMSProxy Container
• 13.4.2.4: Deploying REST Container Using Static Method
• 13.4.2.5: Updating the Policy Package
• 13.5: Application Protector API on REST
• 13.5.1: Version 4 (V4) Application Protector API on REST
• 13.5.1.1: List of REST APIs
• 13.5.1.1.1: HTTP GET version
• 13.5.1.1.2: HTTP POST protect
• 13.5.1.1.3: HTTP POST unprotect
• 13.5.1.1.4: HTTP POST reprotect
• 13.5.1.1.5: HTTP GET doc
• 13.5.1.1.6: HTTP Headers
• 13.5.1.2: V4 AP REST HTTP Response Codes
• 13.5.2: Version 1 (V1) Application Protector API on REST
• 13.5.2.1: List of REST APIs
• 13.5.2.1.1: HTTP GET version
• 13.5.2.1.2: HTTP POST protect
• 13.5.2.1.3: HTTP POST unprotect
• 13.5.2.1.4: HTTP POST reprotect
• 13.5.2.1.5: HTTP Headers
• 13.5.2.2: Error Handling for v1 API
• 13.5.2.3: V1 AP REST HTTP Response Codes
• 13.6: Using Samples
• 13.7: Running the Autoscaling Script
• 13.8: Using Dockerfiles to Build Custom Images
• 13.9: Appendix - Deploying the Helm Charts by Using the Set Argument
• 14: Application Protector Java Container
• 14.1: Understanding the Architecture
• 14.1.1: Architecture and Components using Dynamic-based Deployment
• 14.1.2: Architecture and Components using Static Deployment
• 14.2: System Requirements
• 14.2.1: Software Requirements
• 14.2.2: Hardware Requirements
• 14.3: Preparing the Environment
• 14.3.1: Initializing the Jump Box
• 14.3.2: Extracting the Installation Package
• 14.3.3: Creating Certificates
• 14.3.4: Uploading the Images to the Container Repository
• 14.3.5: Creating the AWS Environment
• 14.3.5.1: Creating the AWS Setup for Static Mode
• 14.3.5.1.1: Creating a Data Encryption Key (DEK)
• 14.3.5.1.2: Creating an AWS S3 Bucket
• 14.3.5.1.3: Creating an AWS EFS
• 14.3.5.2: Creating a Kubernetes Cluster
• 14.4: Installing the Protector
• 14.4.1: Deploying AP Java Container for Dynamic Method
• 14.4.1.1: Deploying Log Forwarder
• 14.4.1.2: Deploying Resilient Package Proxy (RPP)
• 14.4.1.3: Deploying the AP Java Container with Dynamic Method
• 14.4.2: Deploying AP Java Container in Static Mode
• 14.4.2.1: Retrieving the Policy Package from the ESA
• 14.4.2.2: Deploying Log Forwarder
• 14.4.2.3: Deploying KMSProxy Container
• 14.4.2.4: Deploying AP Java Container Using Static Method
• 14.4.2.5: Updating the Policy Package
• 14.5: Running Security Operations
• 14.6: Using Dockerfiles to Build Custom Images
• 14.7: Appendix - Deploying the Helm Charts by Using the Set Argument
• 15: Protegrity AWS EC2 Protector
• 15.1: Understanding the Architecture
• 15.2: System Requirements
• 15.3: Preparing the Environment
• 15.3.1: Initializing the Jump Box
• 15.3.2: Extracting the Installation Package
• 15.3.3: Creating a JAR for the Sample Application
• 15.3.4: Creating a Linux AMI for the Sample Application
• 15.3.5: Creating Certificates and Keys for TLS Authentication
• 15.3.6: Uploading the Server Certificates to the AWS Identity and Access Management
• 15.3.7: Uploading the RPSyncConfig Package to the AWS S3 Bucket
• 15.3.8: Preparing the AWS Requirements
• 15.4: Installing the Protector
• 15.5: Running Security Operations
• 15.6: Upgrading the Protector from Version 9.x to 10.x
• 15.7: Upgrading the Protector from Version 10.x to 10.y

1 - Configuration Parameters for Protector

The protectors can be configured by specifying the parameters in a configuration file or by specifying the parameters as environment variables. The parameters specified as environment variables have a higher priority than the parameters specified in the configuration file. The value specified in the environment varibles overrides the value specified in the configuration file for the same parameter.

The configuration parameters can be categorized as follows:

• Core - Responsible for performing URP operations and generating the logs.
• Resilient Package Sync - Responsible for retrieving the package from the ESA, RPP, or the shared memory at regular intervals.
• Resilient Package Status - Responsible for sending the status of the resilient package to Insight at regular intervals.
• Forwarding Audits and Logs - Responsible for forwarding the audits and logs to the Log Forwarder or other outputs.

The configuration parameters can be mandatory and optional. Mandatory parameters are the ones where users are expected to modify the values as per their requirement. Optional parameters are the ones where users are recommended to retain the default values. Users should exercise extreme caution while updating the values for any optional parameter.

Core Configuration Parameter

The following is the Core configuration parameter.

In the configuration file, specify this parameter under the section [core].
For example:

[core]
flushinterval = <value>

Resilient Package Sync Configuration Parameters

The following are the configuration parameters for Resilient Package Sync.

In the configuration file, specify this parameter under the section [sync].
For example:

[sync]
interval = <value>
protocol = <value>
host = <value>
port = <value>
ca = <value>
cert = <value>
key = <value>
secretfile = <value>
requesttimeout = <value>
verify = <value>
setlocalip = <value>
channel = <value>

Resilient Package Status Configuration Parameter

The following is the configuration parameter for Resilient Package Status.

In the configuration file, specify this parameter under the section [status].
For example:

[status]
interval = <value>

Configuration Parameters for Forwarding Audits and Logs

The following are the configuration parameters for forwarding the audits and logs.

In the configuration file, specify this parameter under the section [log].
For example:

[log]
output = <value>
mode = <value>
host = <value>
port = <value>

2 - Installing Resilient Package (RP) Agent on Linux or Unix and Windows

The following section describes the steps to install the RP Agent on Linux or Unix and Windows.

Installing RP Agent on Linux or Unix

The following section describes the steps to install the RP Agent on a Linux or Unix platform using the Interactive or Silent mode of installation.

By default, the RP Agent is installed in the /opt/protegrity directory. You can choose to install the RP Agent in a different directory by specifying the -d or --dir argument in the installation command. If you change the base installation directory, then the installation path will also change accordingly.

Use the --help argument if you need any help with installing the RP Agent, as shown in the following command.

./RPAgentSetup_Linux_x64_<version>.sh --help

The following snippet displays the output.

Install:
./RPAgentSetup_Linux_x64_<version>.sh -e <host[:port]...> [-d <dir>]
 
Options:
 -h, --host      Host or IP of the upstream
 -u, --user      ESA username. Must be a user with the Export Certificates role.
 -p, --password  ESA password. Must be a user with the Export Certificates role.
 -t, --token     ESA token. Must be a token with the Export Certificates role.
     --port      Port to connect to on the upstream (default: 25400)
 -d, --dir       Path to base directory for installation (default: /opt/protegrity)
   , --nocert    Disable certificates

Important: The -t or --token parameter is currently not applicable.

Installing RP Agent on Linux or Unix using Interactive Mode

To install the RP Agent on a Linux or Unix platform using the Interactive mode:

1. Run the RP Agent installer using the following command.

   ./RPAgentSetup_Linux_x64_<version>.sh
   

   The prompt to enter the upstream hostname or IP address appears.

   Please enter upstream host name or IP address
   []:
   

2. Enter the ESA Host Name or IP Address.

3. Press ENTER.

   The prompt to enter the username for downloading certificates appears.

   Please enter the user name for downloading certificates
   []:
   

4. Enter the username for downloading the certificates.

5. Press ENTER.

   The prompt to enter the password for downloading the certificates appears.

   Please enter the password for downloading certificates
   []:
   

6. Press ENTER to install into the destination directory.

   Directories are created under /opt/protegrity/rpagent by default, and the required installation files are installed in these directories.

   Ensure that the ESA is up and running with the HubController service in running status to enable automatic downloading of certificates.

7. If you have installed the RP Agent using the --nocert parameter, then the ESA certificates are not downloaded during the installation. To manually install the certificates to the /opt/protegrity/rpagent/data directory of the RP Agent, perform the following steps.

   i. Navigate to the /opt/protegrity/rpagent/bin directory and run the following command.

   ./GetCertificates -u <ESA User with the Export Certificates role> [-h <ESA host name or IP address>] [-p portno] [-d directory]
   

   This initiates a secure communication between the RP Agent and the ESA.

   ii. Enter the password for the ESA user.

   iii. Verify that the following files have been copied to the /opt/protegrity/rpagent/data directory:

   • CA.pem
   • cert.key
   • cert.pem
   • rpagent.cfg
   • secret.txt

8. Start the RP Agent by using the following command.

   /opt/protegrity/rpagent/bin/rpagentctrl start
   

   The RP Agent is successfully installed.

Installing RP Agent on Linux or Unix using Silent Mode

You can also execute the RP Agent installer without any manual intervention, which is also known as the Silent mode of installation. The following parameters must be provided to execute the installer in the Silent mode.

Important: The -t or --token parameter is currently not applicable.

At the command prompt, type the following command from the installer directory.

./RPAgentSetup_Linux_x64_<version>.sh (-u <user> -p <password>) [-h <host>] [--port <port>] 

If you want to install the RP Agent in a directory other than the default directory, then you can add the -d parameter to the command to specify the directory. The following command displays a sample snippet.

./RPAgentSetup_Linux_x64_<version>.sh (-u <user> -p <password>) [-h <host>] [--port <port>] [-d <dir>]

Uninstalling the RP Agent on Linux or Unix

1. Navigate to the /opt/protegrity/rpagent/bin directory.

2. Stop the RP Agent by using the following command.

   ./rpagentctrl stop
   

3. Delete the rpagent directory.

   The RP Agent and all its components are uninstalled.

Installing RP Agent on Windows

The following section describes the steps to install the RP Agent on a Windows platform using the Windows wizard or through silent installation.

When you install the RP Agent, the system automatically sets up a directory structure with the required files in the ..\Protegrity\rpagent directory.

Installing RP Agent on Windows using the Windows Wizard

To install the RP Agent on a Windows platform using the Windows wizard:

1. Double-click or run the RPAgentSetup_<OS>_<version>.exe file.

   The Setup Wizard appears.

2. Click Next.

   The Upstream Connectivity Information screen appears.

3. Specify the following details.

   Parameter	Description
   Address	Host name or IP address of the upstream server that is providing the resilient packages.
   Port	Port number of the upstream server that is providing the resilient packages. The default value is 25400.
   Certificate download user	Name of the ESA user for downloading the certificates from the ESA. This is an optional parameter.
   Certificate download password	Password of the ESA user required for downloading the certificates from the ESA. This is an optional parameter.

   Note: Ensure that the ESA is up and running with the HubController service in running status to enable automatic downloading of certificates.

4. Click Next.

   The screen to specify the RP Agent location appears.

5. Browse to the directory in which you want to install the RP Agent, or retain the default location.

   It is recommended to retain the default location.

6. Click Next.

   The Ready to Install screen appears.

7. Click Install.

   The Windows wizard installs the RP Agent on your machine.

8. Click Finish to close the Resilient Package Agent Setup Wizard and complete the installation.

   The directories are created under the installation directory that was defined and the installation files are installed in these directories.

9. If you have not specified the Certificate download user name and password in the Upstream Connectivity Information screen, then the ESA certificates are not downloaded during the installation. To manually install the certificates to the \Protegrity\rpagent\data directory of the RP Agent, perform the following steps.

   i. Navigate to the \Protegrity\rpagent\bin directory and run the following command.

   .\GetCertificates.bat -u <ESA User with the Export Certificates role> -p <Password of the ESA user> [-h <ESA host name or IP address>] [--port portno] [-d directory]
   

   This initiates a secure communication between the RP Agent and the ESA.

   ii. Verify that the following files have been copied to the \Protegrity\rpagent\data directory:

   • CA.pem
   • cert.key
   • cert.pem
   • rpagent.cfg
   • secret.txt

   iii. Restart the rpagent service from the Windows Task Manager after editing the file.

Installing RP Agent on Windows using Silent Mode

You can also execute the RP Agent installer without any manual intervention, which is also known as the Silent mode of installation. The following parameters must be provided to execute the installer in the Silent Mode.

Important: The --token parameter is currently not applicable.

At the command prompt, type the following command from the installation directory.

.\RPAgentSetup_<OS>_<version>.exe --host <ip address> --port <port number> --user <ESA user name> --password <ESA user password>

To install the RP Agent in a directory other than the default directory, add the --dir parameter to the command to specify the RP Agent installation directory. The following snippet displays a sample command.

.\RPAgentSetup_<OS>_<version>.exe --host <ip address> --port <port number> --user <ESA user name> --password <ESA user password> --dir <RP Agent installation directory>

Uninstalling the RP Agent on Windows

1. Navigate to the \Protegrity\rpagent directory.

2. Double-click the unins000.exe file.

   The Resilient Package Agent Uninstall dialog box appears. A message appears asking you to confirm whether you want to uninstall the RP Agent.

3. Click Yes. The RP Agent and all its components are uninstalled.

3 - Installing Resilient Package (RP) Proxy on Linux or Unix and Windows

The following section describes the steps to install the RP Proxy on Linux or Unix and Windows.

Installing RP Proxy on Linux or Unix

The following section describes the steps to install the RP Proxy on a Linux or Unix platform using the Interactive or Silent mode of installation.

By default, the RP Proxy is installed in the /opt/protegrity directory. You can choose to install the RP Proxy in a different directory by specifying the -d or --dir argument in the installation command. If you change the base installation directory, then the installation path will also change accordingly.

Use the --help argument if you need any help with installing the RP Proxy, as shown in the following command.

./pty-rpproxy-setup-linux64-<version>.sh --help

The following snippet displays the output.

Install:
./pty-rpproxy-setup-linux64-<version>.sh (-u <user> -p <password> | -t <token> | --nocert) [-h <hostname>] [--port <port>] [-d <dir>]
 
Options:
 -h, --host      Host or IP of the upstream
     --port      Port to connect to on the upstream (default: 25400)
 -u, --user      ESA user. Must be a user with the Export Certificates role.
 -p, --password  ESA user's password.
 -t, --token     ESA token. Must be a token with the Export Certificates role.
 -d, --dir       Path to base directory for installation (default: /opt/protegrity)
   , --nocert    Disable certificates

Important: The -t or --token parameter is currently not applicable.

Installing RP Proxy on Linux or Unix using Interactive Mode

To install the RP Proxy on a Linux or Unix platform using the Interactive mode:

1. Run the RP Proxy installer using the following command.

   ./pty-rpproxy-setup-linux64-<version>.sh
   

   The prompt to enter the upstream hostname or IP address appears.

   Please enter upstream host name or IP address
   []:
   

2. Type the ESA Host Name or IP Address, and press ENTER.

   The prompt to enter the username for downloading certificates appears.

   Please enter the user name for downloading certificates
   []:
   

3. Type the username for downloading the certificates, and press ENTER.

   The prompt to enter the password for downloading the certificates appears.

   Please enter the password for downloading certificates
   []:
   

4. Type the password, and press ENTER to install into the destination directory.

   Directories are created under /opt/protegrity/rpproxy by default, and the required installation files are installed in these directories.

   Ensure that the ESA is up and running with the HubController service in running status to enable automatic downloading of certificates.

5. If you have installed the RP Proxy using the --nocert parameter, then the ESA certificates are not downloaded during the installation. To manually install the certificates to the /opt/protegrity/rpproxy/data directory of the RP Proxy, perform the following steps.

   i. Navigate to the /opt/protegrity/rpproxy/bin directory and run the following command.

   ./GetCertificates -u <ESA User with the Export Certificates role> [-h <ESA host name or IP address>] [-p portno] [-d directory]
   

   This initiates a secure communication between the RP Proxy and the ESA.

   ii. Enter the password for the ESA user.

   iii. Verify that the following files have been copied to the /opt/protegrity/rpproxy/data directory:

   • CA.pem
   • cert.key
   • cert.pem
   • secret.txt

   Start the RP Proxy by using the following command.

   /opt/protegrity/rpproxy/bin/rpproxyctrl start
   

   The RP Proxy is successfully installed.

Installing RP Proxy on Linux or Unix using Silent Mode

You can also execute the RP Proxy installer without any manual intervention, which is also known as the Silent mode of installation. The following parameters must be provided to execute the installer in the Silent mode.

At the command prompt, type the following command from the installer directory.

./pty-rpproxy-setup-linux64-<version>.sh (-u <user> -p <password>) [-h <host>] [--port <port>] 

If you want to install the RP Proxy in a directory other than the default directory, then you can add the -d parameter to the command to specify the directory. The following command displays a sample snippet.

./pty-rpproxy-setup-linux64-<version>.sh (-u <user> -p <password>) [-h <host>] [--port <port>] [-d <dir>]

Installing RP Proxy on Windows

The following section describes the steps to install the RP Proxy on a Windows platform using the Windows wizard or through silent installation.

When you install the RP Proxy, the system automatically sets up a directory structure with the required files in the ..\Protegrity\rpproxy directory.

Installing RP Proxy on Windows using the Windows Wizard

To install the RP Proxy on a Windows platform using the Windows wizard:

1. Double-click or run the pty-rpproxy-setup-<OS>-<version>.exe file. The Setup Wizard appears.

2. Click Next. The screen to specify the RP Proxy location appears.

3. Browse to the directory in which you want to install the RP Proxy, or retain the default location. It is recommended to retain the default location.

4. Click Next.

   The Proxy config screen appears.

5. Specify the following details.

   Parameter	Description
   IP/hostname	Host name or IP address of the upstream server that is providing the resilient packages.
   Port	Port number of the upstream server that is providing the resilient packages. The default value is 25400.
   User	Name of the ESA user for downloading the certificates from the ESA. This is an optional parameter.
   Password	Password of the ESA user required for downloading the certificates from the ESA. This is an optional parameter.

   Note: Ensure that the ESA is up and running with the HubController service in running status to enable automatic downloading of certificates.

6. Click Next.

   The Ready to Install screen appears.

7. Click Install.

   The Windows wizard installs the RP Proxy on your machine.

8. Click Finish to close the rpproxy Setup Wizard and complete the installation.

   The directories are created under the installation directory that was defined and the installation files are installed in these directories.

9. If you have not specified the Certificate download user name and password in the Proxy config screen, then the ESA certificates are not downloaded during the installation. To manually install the certificates to the \Protegrity\rpproxy\data directory of the RP Proxy, perform the following steps.

   i. Navigate to the \Protegrity\rpproxy\bin directory and run the following command.

   .\GetCertificates.bat -u <ESA User with the Export Certificates role> -p <Password of the ESA user> [-h <ESA host name or IP address>] [--port portno] [-d directory]
   

   This initiates a secure communication between the RP Proxy and the ESA.

   ii. Verify that the following files have been copied to the \Protegrity\rpproxy\data directory:

   • CA.pem
   • cert.key
   • cert.pem
   • secret.txt

   iii. Restart the rpproxy service from the Windows Task Manager after editing the file.

Installing RP Proxy on Windows using Silent Mode

You can also execute the RP Proxy installer without any manual intervention, which is also known as the Silent mode of installation. The following parameters must be provided to execute the installer in the Silent Mode.

At the command prompt, type the following command from the installation directory.

.\pty-rpproxy-setup-<OS>-<version>.exe --host <ip address> --port <port number> --user <ESA user name> --password <ESA user password>

To install the RP Proxy in a directory other than the default directory, add the --dir parameter to the command to specify the RP Proxy installation directory. The following snippet displays a sample command.

.\pty-rpproxy-setup-<OS>-<version>.exe --host <ip address> --port <port number> --user <ESA user name> --password <ESA user password> --dir <RP Proxy installation directory>

Uninstalling the RP Proxy

1. Navigate to the \Protegrity\rpproxy\bin directory.

2. Double-click the uninstall.exe file.

   The Uninstall rpproxy dialog box appears. A message appears asking you to confirm whether you want to uninstall the RP Proxy.

3. Click Yes. The RP Proxy and all its components are uninstalled.

4 - Configuration Parameters for RPP outside of the ESA

The Resilient Package Proxy (RPP) is a HTTP cache and it enables Dynamic Resilient Package Deployment to scale. The RPP:

• Leads to faster response time on protectors.
• Reduces network traffic.

The configuration parameters for RPP include log level, host and port, and service configuration. These parameters are explained in detail in the following section.

Service Logging Configuration

Log Level

The following is the log level parameter.

Host and Port

The following is the host and port where the logs are forwarded.

Service Configuration

Limit Request

The following is the limit request parameter.

Cache TTL

The following is the cache Time to live (TTL) parameter.

Listener Configuration

The following are the listener configuration parameters.

Authentication Server Configuration

The authentication (auth) server can be another RPP or ESA. The following are the auth server configuration parameters.

Upstream Server Configuration

The upstream server can be another RPP or ESA. The following are the auth server configuration parameters.

Note: The parameters listed here are for configuring RPP outside of the ESA. RPP running on the ESA does not require configuration changes and any modifications would not be maintained during ESA upgrades.

5 - Installing Log Forwarder on Linux and Windows

The following section describes the steps to install the Log Forwarder on Linux or Windows.

Installing Log Forwarder on Linux

The following section describes the steps to install the Log Forwarder on a Linux platform using the Interactive or Silent mode of installation.

By default, the Log Forwarder is installed in the /opt/protegrity directory. You can choose to install the Log Forwarder in a different directory by specifying the -d or --dir argument in the installation command. If you change the base installation directory, then the installation path will also change accordingly.

Use the --help argument if you need any help with installing the Log Forwader, as shown in the following command.

./LogforwarderSetup_Linux_x64_<version>.sh --help

The following snippet displays the output.

Install:

  ./LogforwarderSetup_Linux_x64_<version>.sh -e <host[:port]...> [-d <dir>]

Options:
  -e, --endpoint    Host of the target audit store endpoint(s).
                    Repeat this option to specify multiple endpoints to balance the load on audit store endpoints.
                    Each endpoint may specify a port. When no port is specified, 9200 is used.
  -d, --dir         Path to base directory for installation (default: /opt/protegrity)

Installing Log Forwarder on Linux using Interactive Mode

To preserve all the configurations while upgrading the Log Forwarder, ensure that you backup all the files present under the /opt/protegrity/logforwarder/data/config.d directory.

To install the Log Forwarder on a Linux platform using the Interactive mode:

1. Run the Log Forwarder installer using the following command.

   ./LogforwarderSetup_Linux_x64_<version>.sh
   

   The prompt to enter the Audit Store endpoint appears.

   Enter the audit store endpoint (host),
   alternative (host:port) to use another port than the default port 9200 :
   

2. Enter the Audit Store endpoint that is the Audit Store IP address and the Audit Store port number where the Log Forwarder sends the logs. The default port number is 9200. If you are using the default port, then do not specify the port number.

3. Press ENTER.

   The added Audit Store endpoint appears on the screen.

   The prompt to enter an additional Audit Store appears.

   Do you want to add another audit store endpoint? [y/n]:
   

4. If you want to add more than one Audit Store endpoint, then type y otherwise type n. If you need to add additional Audit Store endpoints, then repeat both Step 2 and Step 3 for each additional endpoint to add.

5. Type the y key to install into the destination directory.

   The Log Forwarder is installed in the /opt/protegrity/logforwarder/ directory.

6. Start the Protegrity Log Forwarder service by using the following command.

   /opt/protegrity/logforwarder/bin/logforwarderctrl start
   

   The Log Forwarder is successfully installed.

7. If you want to modify the number of Audit Stores, then perform the following steps after the installation completes.

   i. Edit the upstream.cfg file to add the audit stores.

   ii. Navigate to the /opt/protegrity/logforwarder/data/config.d directory, and edit the upstream.cfg file as follows. The [Node] block must be added for each new Audit Store.

    [NODE]
      Name node-1
      Host 10.37.4.150
      Port 9200
      tls on
      tls.verify off
      Pipeline logs_pipeline
    [NODE]
      Name node-2
      Host 10.37.4.158
      Port 9200
      tls on
      tls.verify off
      Pipeline logs_pipeline
   

   The following parameters need to be added for a new node.

   Parameter	Description
   Name	Set a name for the Audit Store.
   Host	IP address or host name of the Audit Store.
   Port	Set the port number. The default port number is 9200.
   tls	Enable or disable the TLS support. Set this parameter to on to enable the TLS support and off to disable the TLS support. The default tls setting is on.
   tls.verify	Force certificate validation. Set this parameter to on to enforce certificate validation and off to disable certificate verification. The default tls.verify setting is off.
   Pipeline	Set a filter for the Audit Store. The default pipeline setting is logs_pipeline.

   iii. Use the following command to restart the Protegrity Log Forwarder service after editing the file.

   /opt/protegrity/logforwarder/bin/logforwarderctrl start
   

Installing Log Forwarder on Linux using Silent Mode

To preserve all the configurations while upgrading the Log Forwarder, ensure that you backup all the files present under the /opt/protegrity/logforwarder/data/config.d directory.

You can also execute the Log Forwarder installer without any manual intervention, which is also known as the Silent mode of installation. The following parameters must be provided to execute the installer in the Silent mode.

At the command prompt, type the following command from the installer directory.

./LogforwarderSetup_Linux_x64_<version>.sh -e <ip address:port number> [-e <ip address:port number>]

If you want to install the Log Forwarder in a directory other than the default directory, add the -d or --dir argument to the command to specify the Log Forwarder installation directory

The following snippet displays a sample command.

./LogforwarderSetup_Linux_x64_<version>.sh -e <ip address:port number> [-e <ip address:port number>] -d <Log Forwarder installation directory> 

Uninstalling the Log Forwarder on Linux

1. Navigate to the /opt/protegrity/logforwarder/bin directory.

2. Stop the Log Forwarder by using the following command.

   ./logforwarderctrl stop
   

3. Delete the logforwarder directory.

   The Log Forwarder and all its components are uninstalled.

Installing Log Forwarder on Windows

The following section describes the steps to install the Log Forwarder on a Windows platform using the Windows wizard or through silent installation.

When you install the Log Forwarder, the system automatically sets up a directory structure with the required files in the ..\Protegrity\logforwarder directory.

Installing Log Forwarder on Windows using the Windows Wizard

To install the Log Forwarder on a Windows platform using the Windows wizard:

1. Double-click or run the LogforwarderSetup_<OS>_<version>.exe file.

   The Setup Wizard appears.

2. Click Next.

   The Audit Store Connectivity Information screen appears.

3. Select the number of audit stores that are needed, and then click Next.

   The screen to specify the Audit Store location appears.

4. Enter the Audit Store endpoint (IP address:port number).

   The default port number is 9200.

5. Click Next.

   The Select Destination Location screen appears.

6. Browse to the directory in which you want to install the Log Forwarder, or retain the default location.

   It is recommended to retain the default location.

7. Click Next.

   The Ready to Install screen appears.

8. Click Install.

   The Windows wizard installs the Log Forwarder on your machine.

9. Click Finish to close the Log Forwarder Setup Wizard and complete the installation. The directories are created under the installation directory that was defined and the installation files are installed in these directories.

10. If you want to modify the number of Audit Stores or if you have selected an incorrect number of Audit Stores in step 3, then perform the following steps after the installation completes.

    i. Edit the upstream.cfg file to add the audit stores.

    ii. Navigate to the ..\Protegrity\logforwarder\data\config.d directory, and edit the upstream.cfg file as follows. The [Node] block must be added for each new Audit Store.

    [NODE]
      Name node-1
      Host 10.37.4.150
      Port 9200
      tls on
      tls.verify off
      Pipeline logs_pipeline
    [NODE]
      Name node-2
      Host 10.37.4.158
      Port 9200
      tls on
      tls.verify off
      Pipeline logs_pipeline
    

    The following parameters need to be added for a new node.

    Parameter	Description
    Name	Set a name for the Audit Store.
    Host	IP address or host name of the Audit Store.
    Port	Set the port number. The default port number is 9200.
    tls	Enable or disable the TLS support. Set this parameter to on to enable the TLS support and off to disable the TLS support. The default tls setting is on.
    tls.verify	Force certificate validation. Set this parameter to on to enforce certificate validation and off to disable certificate verification. The default tls.verify setting is off.
    Pipeline	Set a filter for the Audit Store. The default pipeline setting is logs_pipeline.

    iii. Restart the Log Forwarder service from the Windows Task Manager after editing the file.

Installing Log Forwarder on Windows using Silent Mode

You can also execute the Log Forwarder installer without any manual intervention, which is also known as the Silent mode of installation. The following parameters must be provided to execute the installer in the Silent Mode.

At the command prompt, type the following command from the installation directory.

.\LogforwarderSetup_<OS>_<version>.exe -endpoint1 <ip address:port number> [-endpoint2 <ip address:port number>] [-endpoint3 <ip address and port number>]

To install the Log Forwarder in a directory other than the default directory, add the -dir parameter to the command to specify the Log Forwarder installation directory. The following snippet displays a sample command.

.\LogforwarderSetup_<OS>_<version>.exe -endpoint1 <ip address:port number> [-endpoint2 <ip address:port number>] [-endpoint3 <ip address and port number>] -dir <Log Forwarder installation directory>

Uninstalling the Log Forwarder

1. Navigate to the \Protegrity\logforwarder directory.

2. Double-click the unins000.exe file.

   The Log Forwarder Uninstall dialog box appears. A message appears asking you to confirm whether you want to uninstall the Log Forwarder.

3. Click Yes.

   The Log Forwarder and all its components are uninstalled.

5.1 - Configuring the disk space on the Log Forwarder

If the incoming logs are cached faster than they are sent to Insight, then a back pressure arises.

The following formula can be used to calculate the disk space on the Log Forwarder. The formula requires the estimated audit rate and time to sustain the audit rate, without logs being sent to Insight. Modify the values in this example as required. The default value of the disk space is 256 MB.

Disk Space in Mega bytes = (Audit Rate X Time in Seconds X 5.9 ) / 1024.

• Audit Rate = Number of policy audits generated per second
• Time in Seconds = Time duration for which the disk can sustain the audit rate without the logs being sent to Insight.

If the default or the configured value of the storage.total_limit_size setting is reached, then the Log Forwarder discards the oldest audits to create disk space for new audits.

Perform the following steps to configure the storage.total_limit_size setting in the out.conf file on the protector machine.

1. Log in and open a CLI on the protector machine.

2. Navigate to the config.d directory using the following command.

   cd /opt/protegrity/logforwarder/data/config.d
   

   Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/data/config.d path. Use the /opt/protegrity/fluent-bit/data/config.d path for protectors v9.1.0.0 and earlier.

3. Back up the existing out.conf file using the following command.

   cp out.conf out.conf_backup
   

4. Open the out.conf file using a text editor.

5. Update the value of storage.total_limit_size setting in the output blocks. The default value of the storage.total_limit_size is 256 MB. The following snippet shows the extract of the code.

   [OUTPUT]
       Name opensearch 
       Match logdata 
       Retry_Limit False
       Index pty_insight_audit
       Type  _doc
       Time_Key ingest_time_utc
       Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
       storage.total\_limit\_size 256M
   
   [OUTPUT]
       Name opensearch 
       Match flulog
       Retry_Limit 1
       Index pty_insight_audit
       Type  _doc
       Time_Key ingest_time_utc
       Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
       storage.total\_limit\_size 256M
   
   [OUTPUT]
       Name opensearch 
       Match errorlog
       Retry_Limit 1
       Index pty_insight_audit
       Type  _doc
       Time_Key ingest_time_utc
       Upstream /opt/protegrity/logforwarder/data/config.d/upstream.cfg
       storage.total\_limit\_size 256M
   

   Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/data/config.d path. Use the /opt/protegrity/fluent-bit/data/config.d path for protectors v9.1.0.0 and earlier.

6. Save and close the file.

7. Restart the Log Forwarder on the protector using the following commands.

   /opt/protegrity/logforwarder/bin/logforwarderctrl stop
   /opt/protegrity/logforwarder/bin/logforwarderctrl start
   

   Protectors v9.2.0.0 and later use the /opt/protegrity/logforwarder/bin path. Use the /opt/protegrity/fluent-bit/bin path for protectors v9.1.0.0 and earlier.

8. If required, complete the configurations on the remaining protector machines.

6 - Memory Utilization in Resilient Protectors

The 10.0.x Protectors support a Resilient Deployment architecture, which uses dynamic memory allocation to store the resilient packages downloaded from the ESA or the upstream server. In comparison, the protectors prior to version 10.0.x support a PEP Server based architecture, which uses fixed memory allocation to store the policies downloaded from the ESA. As a result, the Resilient Deployment architecture offers a memory efficient solution as compared to the PEP Server architecture.

The new Resilient Deployment architecture offers the following advantages over the PEP Server architecture:

• Scalability:
  • Easy to scale the protectors by increasing the hardware.
  • No limits on the policy size.
• Lower costs: Less network and memory requirements thereby reducing the hardware costs.
• Secure: Enables configuration to only store policy in process memory.
• Improved performance: Takes less time in distributing the resilient packages.

Memory Consumption during Policy Enforcement

The following example shows how much memory is consumed by an 10.0.x Protector when a policy is successfully loaded in heap memory. An application log is generated specifying the memory used.

{"additional_info":{"description":"Policy successfully loaded","memoryUsed":"13"}}

In this example, the memory used is 13 MB. This application log is then sent to Insight, where you can view the memory utilization for all the protectors.

For more information about Protegrity Dashboards, refer to the section Viewing the dashboards.

For more information about application logs, refer to the section Application Logs.

The following table shows how the number of data elements in combination with number of users affect the memory consumption.

The average length of user names is 16 characters.

For more information about capacity planning, contact Protegrity Support.

7 - Verification of Signed Protector Build

Verifying the signed protector build on the Linux platform

The digital signature of the signed protector build on the Linux platform, using the GNU Privacy Guard (GPG) encryption key, must be verified.

To verify the digital signature on the Linux platform:

1. Download the protector to any location on the machine where you want to install the protector.

2. To get the GPG encryption key from the ESA, which is located in the /opt/verification_keys/ directory, run the following command on the protector machine.

   sshpass -p <ESA root password> scp -r root@<ESA ip>:/opt/verification_keys/10.0.gpg <location of 10.0.gpg file>
   

3. Import the key to the GPG utility using the following command.

   gpg --import <location for 10.0.gpg file>
   

4. Extract the protector build using the following command.

   tar -xvf <protector build (.tgz)>
   

5. Verify the signature using the following command.

   gpg --verify signatures/<signature of protector build (.sig)> <protector build (.tgz)>
   

6. Extract the protector build again using the following command to obtain the various components to install the protector.

   tar -xvf <protector build (.tgz)>
   

Verifying the signed protector build on the Windows platform

The digital signature of the signed protector build on the Windows platform, using the GNU Privacy Guard (GPG), must be verified.

To verify the digital signature on the Windows platform:

1. Download the protector to any location on the machine where you want to install the protector.

2. From 10.0 ESA, get the gpg encryption key which is located in /opt/verification_keys folder to the protector machine.

3. Run the following command on the protector machine to transfer the file in the current directory.

   scp -r root@<ESA_IP>:/opt/verification_keys/10.0.gpg .
   

   Note: Alternatively, login to the ESA using ssh. Navigate to the /opt/verification_keys/ folder and download the 10.0.gpg file manually. Then upload it to the protector machine.

4. Install gpg4win on your Windows protector machine.
   a. Navigate to https://files.gpg4win.org/.
   b Download the latest https://files.gpg4win.org/gpg4win-4.4.1.exe installer.
   c. Run the downloaded installer.
   d. Verify the installed version using the following command.
   gpg --version
   The gpg version gets displayed. For example; gpg (GnuPG) 2.4.8.

5. Import the key to the GPG utility using the following command.

   gpg --import <location for 10.0.gpg file>
   

6. Extract the protector build file.
   a. Right-click the protector build file and click Extract All.
   b. Click Extract.

7. Verify the signature using the following command.

   gpg --verify <full path of signature file (.sig)> <full path of protector build (.zip)>
   

   The signature file “.sig” is present in the signatures folder.

8. Extract the protector build file to get other components.
   a. Navigate to the protector build folder.
   b. Right-cick the protector build file and click Extract All.
   c. Click Extract.

8 - Protection Method Reference

Protegrity products can protect sensitive data with the following protection methods:

• Tokenization
• Format Preserving Encryption (FPE)
• Encryption
• Hashing
• No Encryption
• Monitoring
• Masking

The following table describes the protection methods for structured and unstructured data security policy types.

Table: Protection Methods by Data Security Policy Type

The following table describes the deprecated protection methods for structured and unstructured data security policy types.

Table: Deprecated Protection Methods by Data Security Policy Type

Protegrity protection methods, including tokenization, encryption, monitoring, masking, and hashing, support various input formats. This enables you to protect sensitive data using these methods. Some examples of input formats are as follows:

• Social Security Numbers (SSNs)
• Credit Card Numbers (CCNs)
• Electronic Personal Health Information (ePHI), which is controlled by Health Insurance Portability and Accountability Act (HIPPA) and Health Information Technology for Economic and Clinical Health (HITECH)
• Personally identifiable information (PII)

The following table shows different types of sensitive data that can be protected using different protection methods. It demonstrates input values and their corresponding protected values.

Table: Examples of Protected Data

You can combine Protegrity protection methods to obtain the required level of data access control within the enterprise.

For example, a Security Officer can use a data security policy to control what is delivered to different roles in the policy. The following figure shows how Social Security Number access can vary by different users and applications.

In the figure, the tokenized SSN is stored in the database. However, there are four roles defined in the policy:

Table: Different Roles in the Policy

Each role can receive a different form of the SSN based on its need. The Security Officer determines the SSN form by role.
Protegrity tokenization maintains a separation of duties by way of the data security policy.
The DBA, Developers, and System Administrators do not have direct access to the data. Everything goes through the data security policy, regardless of who manages the system.
For more information about data security policies, refer to Managing policies.

8.1 - Protegrity Tokenization

Tokenization is the process of replacing sensitive data with tokens that has no worth to someone who gains unauthorized access to the data. With tokenization, specific pieces of original data can be preserved, while the system tokenizes data according to design. Tokens can be set up and deployed directly on the protectors, depending on your enterprise configuration and data security needs. Once tokenization is deployed, operational systems continually work with the tokens. If the operational systems experience a security breach, then only the tokens are at risk of being compromised. Protegrity tokenization is transparent to end-users. Data integrity is strongly enforced by way of the data security policy.

Protegrity tokenization can be configured to preserve different parts of the original value in the token, such as the last 4 digits. It also recognizes and preserves delimiters, which are often used in SSNs, dates, etc.

Protegrity tokenization enables the user to tokenize various input data types, such as payment card industry (PCI), personally identifiable information (PII), and protected health information (PHI).

With Protegrity tokenization, there is a 1:1 relationship between the real data value and its token value. This enables token values to be used as alternative unique IDs that can be used for joining related information.

The following table describes the token types supported by Protegrity tokenization.

Table: Tokenization Types

The following table describes the deprecated token types supported by Protegrity tokenization.

8.1.1 - Tokenization Support by Protegrity Products

Protegrity offers various types of protectors which helps to protect data in different software and platforms. For example, we can use:

• Application Protectors: To protect data in C, C++, Python, Java, .Net, and Go programming languages.
• Big Data Protectors: To protect data in Big Data at various component levels, such as, Hive, Pig, MapReduce, etc.
• Data Warehouse Protectors: To protect data in the Teradata Data Warehouses.
• Gateway Protectors: To protect data in Gateway Protectors like Data Security Gateway (DSG).
• Cloud Protectors: To protect data in Cloud Protectors.

Each protector has certain tokenization types which are listed in the following sections.

Application Protector

The Protegrity Application Protector (AP) is a high-performance, versatile solution that provides a packaged interface to integrate comprehensive, granular security and auditing into enterprise applications.

Application Protectors support all types of tokens.

Table: Supported Tokenization Types by Application Protector

^*1 - If the input and output types of the API are BYTE[], then the customer application should convert the input to and output from the byte array, before calling the API.

Table: Deprecated Tokenization Types supported by Application Protector

^*1 - If the input and output types of the API are BYTE[], then the customer application should convert the input to and output from the byte array, before calling the API.

For more information about Application protectors, refer to Application Protector.

Big Data Protector

Protegrity supports MapReduce, Hive, Pig, HBase, Spark, and Impala, which utilizes Hadoop Distributed File System (HDFS) or Ozone as the data storage layer. The data is protected from internal and external threats, and users and business processes can continue to utilize the secured data. Protegrity protects data inside the files using tokenization and strong encryption protection methods.

The following table shows the tokenization types supported for Big Data Protectors.

Table: Supported Tokenization Types for Big Data Protectors

^*1 - The customer application should convert the input into a byte array and generate the output from the byte array in the required data type.
^*2 - The Datetime tokenization will only work with VARCHAR data type.
^*3 - The Char tokenization UDFs only support Numeric, Alpha, Alpha Numeric, Upper-case Alpha, Upper Alpha-Numeric, and Email data elements, and with length preservation selected. Using any other data elements with Char tokenization UDFs is not supported. Using non-length preserving data elements with Char tokenization UDFs is not supported.

The following table shows the deprecated tokenization types supported for Big Data Protectors.

Table: Deprecated Tokenization Types supported for Big Data Protectors

^*1 - The customer application should convert the input into a byte array and generate the output from the byte array in the required data type.

For more information about Big Data protectors, refer to Big Data Protector.

Data Warehouse Protector

The Protegrity Data Warehouse Protector is an advanced security solution designed to protect sensitive data at the column level. This enables you to secure your data, while still permitting access to authorized users. Additionally, the Data Warehouse Protector integrates seamlessly with existing database systems using the User-Defined Functions for an enhanced security. Protegrity protects data inside the data warehouses using various tokenization and encryption methods.

Table: Supported Tokenization Types for Data Warehouse Protector

Table: Deprecated Tokenization Types supported by Data Warehouse Protector

For more information about Data Warehouse protectors, refer to Data Warehouse Protector.

• If you have fixed-length data fields and the input data is shorter than the length of the field, then truncate the leading and trailing white spaces before passing the input to the respective Protect and Unprotect UDFs.
• The truncation of whitespaces ensures consistent data output for the protect and unprotect operations. This consistency holds true across all Protegrity products.
• For more information, refer to Truncating Whitespaces.

Database Protector

The Database Protector is a comprehensive data security solution designed to protect sensitive data directly within relational databases. It enables data protection using high‑performance, while allowing applications and authorized users to continue accessing the data transparently.

The following table shows the tokenization types supported for Database Protectors.

Table: Supported Tokenization Types for Database Protectors

For more information about Database protectors, refer to Database Protectors

8.1.2 - Delimiters

Protegrity tokenization can generate the same token regardless of how the data is formatted. Any character in the input that does not comply with the token types in the Tokenization Types is generally treated as a delimiter and remains unchanged during tokenization.

The following table shows how the Protegrity Token types handles delimiters and spaces as compared to plain numerical data.

Table: Tokenization with Delimiters

Note: Some tokenizers can tokenize delimiters. Unicode Gen2, lower ASCII, printable, and binary are examples of tokenizers that can tokenize delimiters.

8.1.3 - Tokenization Properties

Table: Common Tokenization Properties

The following table shows what properties can be set for the token types.

Table: Tokenization Properties for Token Types

• X - means that Property is disabled and cannot be specified.
• √ - means that Property is enabled or can be specified.

The following table shows what properties can be set for the deprecated token types.

Table: Tokenization Properties for deprecated Token Types

• X - means that Property is disabled and cannot be specified.
• √ - means that Property is enabled or can be specified.

8.1.3.1 - Data Type and Alphabet

An alphabet contains all characters considered for tokenization, it is derived from the tokenization type. Characters outside the alphabet are considered delimiters.

Note: This is applicable only for Unicode Gen2 token.

Refer to Tokenization Types for the full list of supported token types.

8.1.3.2 - Static Lookup Table (SLT) Tokenizers

A static lookup table (SLT) contains a pre-generated list of all possible values from a given set of characters. An alphabetic lookup table for instance might contain all values from “Aa” to “Zz”. All entries are then shuffled so that they are in random order.

SLT tokenizer uses multiple SLTs to generate tokens. This is done by first dividing the input value into smaller pieces, called token blocks, which correspond to entries in the lookup tables. The token blocks are then substituted with values from the SLTs and chained together to form the final token value. This means that the token is a result of multiple lookups in multiple SLTs.

Another benefit of SLT tokenizers is that tokenization can be done locally on the protector. With this solution, tokenization is performed locally within the protector environment.

For more information, refer to Working with Data Elements.

There are several types of SLT tokenizers from which you can choose. They are distinguished by their block size and the number of lookup tables.

Table: SLT Tokenizer with block size and lookup tables

*1 - For the SLT_X_1 tokenizer, the number of lookup tables used for the security operations is determined during the creation of the data elements.

The following table describes the types of SLT tokenizers and compares their characteristics.

Table: SLT Tokenizer Memory Footprint for Token Types

Note: The amount of memory used in the protector is twice the size of the token tables (kB) because an inverted SLT is stored in the memory, in addition to the original SLT.

Table: SLT Tokenizer Characteristics for Deprecated Token Types

8.1.3.3 - From Left and From Right Settings

This property indicates the number of characters from left and right that will remain in the clear and hence be excluded from tokenization. Not all token types will allow the end-user to specify these values. The From Left and From Right settings can be configured in the Tokenize Options during the Data Element creation on the ESA Web UI.

For example;
Input Value: 5511309239934975
Credit Card Token: Left=0 Right=4
Output Value: 8278278929904975

When processing input data, you must check the From Left and From Right settings. Validate the input data based on the From Left and From Right settings before applying the Allow Short Data settings.

For more information about how From Left and From Right settings work together with short data settings, refer to Calculating Token Length.

8.1.3.4 - Internal Initialization Vector (IV)

Internal IV is automatically applied to the input value when the token element’s left and right properties are non-zero, designating some characters to remain in the clear. An Internal IV provides an additional security during the tokenization process.

Data to tokenize can be logically divided into three components: left, middle, and right. If an IV is used, then the left and right components are concatenated to form the IV. This IV is then added to the middle component before the value is tokenized.

Table: Examples of Tokenization with Internal IV

8.1.3.5 - Minimum and Maximum Input Length

In Protegrity tokenization only the Decimal token type allows for defining the Minimum and Maximum length of the token element when created. Some token types, such as Datetime, have a fixed length. For the remainder, Minimum and Maximum length depends on token type, tokenizer, length preservation, and short token setting.

The following table illustrates length settings by token type.

Table: Minimum and Maximum Input Length for Token Types

• The minimum and maximum length validation on input data is done on the characters to tokenize.
• The From Left and From right clear characters are not counted. Additionally, characters outside of the alphabet for the selected token type are also not counted.
• The NULL values are accepted but not tokenized.

Table: Minimum and Maximum Input Length for Deprecated Token Types

8.1.3.5.1 - Calculating Token Length

For a Numeric token type, non-numeric values are considered as delimiters. The unsupported characters will be treated as delimiters and left un-tokenized. This occurs when the input value does not contain tokenizable characters with the selected token type.

The number of characters to tokenize is calculated as described on the following image:

If the input value does not contain characters to tokenize, then it is considered a zero-length token. The tokenization of a zero-length input value will not produce an error during the tokenization, and input value will be returned as output.

If the input value has at least one character and short data tokenization is enabled, then the source data can be tokenized. If short data tokenization is not enabled, then the source data will be returned as it is. Alternatively, an appropriate error will appear due to tokenization.

For more information on short data tokenization, refer to Short Data Tokenization.

If the input value contains more characters than the maximum for tokenization, then the value of tokenization is considered too long. The tokenization process provides an appropriate error message.

If the input value has a sufficient number of characters, the tokenization process is successful. This occurs when the character count falls between the minimum and maximum settings.

Table: Token Length Examples

8.1.3.6 - Length Preserving

With the Preserve Length flag enabled, the length of the input data and protected token value is the same.

For data elements with the Preserve Length flag available, you have an option to generate token values that are of the same length as the input data.

Note: The Unicode Gen2 token element is Code Point length preserving when this option is enabled. The length in bytes can vary depending on the alphabet selected during data element creation.

As an extension to this flag, the Allow Short Data flag provides multiple options to manage short input data handling. If the Preserve Length property is not set, then short input protected will not keep its original length. Generated tokens will at least have the minimum length defined for the token type.

For more information about short data tokenization, refer to Short Data Tokenization.

A check for maximum input length is performed regardless of the preservation setting. This check ensures that the input is within the allowed length limit.

If Preserve Length is not selected, then tokenized data may be longer than the input value up to +5%, or at least +1 symbol on a very small initial value (1-2 symbols). Here, symbol can represent a character or a code point.

If Preserve Length is not selected, then for applying protection in database columns, column length of the resulting protected table should be bigger than length of the column to tokenize in the initial table. This will allow inserting tokenized data during protection when tokenized data is longer than the input data.

8.1.3.7 - Short Data Tokenization

When using tokenizers, such as, SLT_1_3, SLT_2_3, and SLT_X_1, the minimum input limit for tokenizable characters or bytes is three. When using tokenizers, such as, SLT_1_6 and SLT_2_6, the minimum input limit for tokenizable characters or bytes is six.

The possible flag values for short data tokenization are described in the following table.

Table: Short tokens flag values

The following tokens support short data tokenization:

• Numeric (0-9)
• Alpha (a-z, A-Z)
• Upper-case Alpha (A-Z)
• Alpha-Numeric (0-9, a-z, A-Z)
• Upper-Case Alpha-Numeric (0-9, A-Z)
• Lower ASCII
• Email
• Unicode Gen2

The following deprecated tokens support short data tokenization:

• Printable
• Unicode
• Unicode Base64

Important: Short input data tokenization can be at risk as a user can easily guess the lookup table and the original data by tokenizing some input data. Consider carefully before using the short data tokenization. If possible, short data input must be avoided.

For more information about the maximum length setting for non-length-preserving token elements, refer to Minimum and Maximum Input Length by Token Types.

8.1.3.8 - Case-Preserving and Position-Preserving Tokenization

This section explains the Case-Preserving and Position-Preserving tokenization options.

• Case-Preserving and Position-Preserving tokenization was designed to support specific business requirements. However, this design comes with a trade-off, as it affects the cryptographic strength of the tokens.
• When preserving the case and position of Alpha-Numeric characters, some information may be leaked through the tokenized value.
• In addition, depending on the length of the Alpha and Numeric substrings, tokens may suffer the same weaknesses as Short Tokens, as described in the section Short Data Tokenization.
• It is recommended that this method should not be used for most use cases. Before using this method, contact Protegrity Support to ensure that the risks are fully understood.

8.1.3.8.1 - Case-Preserving Tokenization

When working with data that is received from multiple sources, the data can contain different casing properties. The data processing stage makes the casing consistent prior to distributing the data to additional systems.

If tokenization is performed prior to the data processing stage, then it results in tokens that differ in its casing properties as per the non-processed data.

To preserve the casing of the non-processed data while tokenizing, an additional tokenization option is provided for the Alpha-Numeric (0-9, a-z, A-Z) token type. The casing of the alphabets in the tokenized value matches the casing of the alphabets in the input value.

Note:
You can specify the case-preserving tokenization option when using the SLT_2_3 tokenizer and Alpha-Numeric (0-9, a-z, A-Z) token type only.
If you select the Preserve Case property on the ESA Web UI, then the Preserve Position property is also selected, by default. Hence, the position of the alphabets and numbers is preserved along with the casing of the alphabets in the output tokenized value.
If you are selecting the Preserve Case or Preserve Position property on the ESA Web UI, then the following additional properties are set:

• The Preserve Length property is enabled and Allow Short Data property is set to Yes, by default. These two properties are not modifiable.
• The retention of characters or digits from the left and the right are disabled, by default. The From Left and From Right properties are both set to zero.

For more information about specifying the case-preserving tokenization option for the Alpha-Numeric (0-9, a-z, A-Z) token type, refer to Create Token Data Elements.

The following table provides some examples for the case-preserving tokenization option.

Table: Case-Preserving Tokenization Examples

8.1.3.8.2 - Position-Preserving Tokenization

The alphabetic and numeric positions in the position-preserving tokenized value matches the alphabetic and numeric positions in the input value.

You can specify the position-preserving tokenization option when using the SLT_2_3 tokenizer and Alpha-Numeric (0-9, a-z, A-Z) token type only.
If you are selecting the Preserve Case or Preserve Position property, then the following additional properties are set:

• The Preserve Length property is enabled and Allow Short Data property is set to Yes, by default. These two properties are not modifiable.
• The retention of characters or digits from the left and the right are disabled, by default. The From Left and From Right properties are both set to zero.

For more information about specifying the position-preserving tokenization option for the Alpha-Numeric (0-9, a-z, A-Z) token type, refer to Create Token Data Elements.

The following table provides some examples for the position-preserving tokenization option.

Table: Position-Preserving Tokenization Examples

8.1.3.9 - External Initialization Vector (EIV)

8.1.3.9.1 - Tokenization Model with External IV

The External IV value is set as a new parameter when calling protect, unprotect or reprotect API from the client application.

The following example explains how the tokenization is performed with the External IV defined. As mentioned before, the main characteristic of the External IV feature is obtaining different outputs for the same input. To have different outputs, you need to specify different IVs.

Note: The External IV is used, prior to protection, as input to modify the data to protect. The External IV is ignored when using encryption.

8.1.3.9.2 - External IV Tokenization Properties

The tokenization with the External IV is done only if the IV is specified during the protect operation through the end user API. When performing unprotect and re-protect operations, the same IV value used for protection must be identified.

If External IV is not provided in either a protect or unprotect function call, then the input is tokenized as-is without any IV.

The External IV value has the following properties:

• Supports ASCII and Unicode characters.
• Minimum 1 byte for the input.
• Maximum 256 bytes for the input.
• Empty and NULL strings are not supported as External IV values. These strings will be ignored during tokenization. The process will continue as if External IV was not used.

Here is an example of the tokenized input value with the External IV for a Numeric token:

Table: Example-External IV for a Numeric token

8.1.3.10 - Truncating Whitespaces

With fixed length fields or columns, input data may be shorter than the length of the field. When this happens, data may be appended with either, or both, trailing and leading whitespace. In those situations, the whitespace is considered during Tokenization. It will affect the tokenization results.

For instance, consider a scenario where the name “Hultgren Caylor” is stored in a Hive Char(30) column.

As the length of the data is less than 30 characters, trailing whitespaces are appended to it. In this case, assume that we need to protect this column with a data element that preserves the first and last character (L=1, R=1). Now with this setting, the expectation is to preserve character H at the start and the character r at the end, in the protected value output. However, the actual data has trailing whitespaces. This results in the output containing the character “H” at the start and a whitespace character " " at the end. The unnecessary trailing whitespaces cause the final protected output to generate a different token.

It is recommended to truncate trailing and leading whitespaces from the data. This applies before sending the data to Protect, Unprotect, or Reprotect UDFs. Truncating unnecessary whitespaces ensures that only the actual data is considered during tokenization. Any trailing and leading whitespaces are not taken into account.

In addition, it is important to follow a consistent approach for truncating the whitespaces across all operations, such as, Protect, Unprotect, Reprotect. For instance, if we have truncated unnecessary trailing whitespaces from the input before the Protect operation, then the same logic of truncating whitespaces from the input, during Unprotect and Reprotect operations needs to be followed.

8.1.4 - Tokenization Types

8.1.4.1 - Numeric (0-9)

The Numeric token type tokenizes digits from 0 to 9.

Table: Numeric Tokenization Type properties

The following table lists the examples of numeric tokenization values.

Table: Examples of Numeric tokenization values

Numeric Tokenization Properties for different protectors

Application Protector

The following table shows supported input data types for Application protectors with the Numeric token.

Table: Supported input data types for Application protectors with Numeric token

^*1 - The API accepts and returns data in BYTE[] format. The customer application needs to convert the input into byte arrays before calling the API, and similarly, convert the output from byte arrays after receiving the response from the API.

^*2 - The Protegrity Application Protector only supports bytes converted from the string data type. If any other data type is directly converted to bytes and passed as input to the Application Protectors APIs that support byte as input and provide byte as output, then data corruption might occur.

For more information about Application protectors, refer to Application Protector.

Big Data Protector

Protegrity supports MapReduce, Hive, Pig, HBase, Spark, and Impala, which utilizes Hadoop Distributed File System (HDFS) or Ozone as the data storage layer. The data is protected from internal and external threats, and users and business processes can continue to utilize the secured data. Protegrity protects data inside the files using tokenization and strong encryption protection methods.

The following table shows supported input data types for Big Data protectors with the Numeric token.

Table: Supported input data types for Big Data protectors with Numeric token

^*1 – If the input and output types of the API are BYTE[], then the customer application should convert the input to and output from the byte array, before calling the API.

^*2 – The Protegrity MapReduce protector, HBase coprocessor, and Spark protector only support bytes converted from the string data type. Data types that are not bytes converted from the string data type might cause data corruption to occur when:

• Any other data type is directly converted to bytes and passed as input to the MapReduce or Spark API that supports byte as input and provides byte as output.
• Any other data type is directly converted to bytes and inserted in an HBase table. Where the HBase table is configured with the Protegrity HBase coprocessor.

^*3 – If you are using the Char tokenization UDFs in Hive, then ensure that the data elements have length preservation selected. In Char tokenization UDFs, using data elements without length preservation selected, is not supported.

For more information about Big Data protectors, refer to Big Data Protector.

Data Warehouse Protector

The Protegrity Data Warehouse Protector is an advanced security solution designed to protect sensitive data at the column level. This enables you to secure your data, while still permitting access to authorized users. Additionally, the Data Warehouse Protector integrates seamlessly with existing database systems using the User-Defined Functions for an enhanced security. Protegrity protects data inside the data warehouses using various tokenization and encryption methods.

The following table shows the supported input data types for the Teradata protector with the Numeric token.

Table: Supported input data types for Data Warehouse protectors with Numeric token

For more information about Data Warehouse protectors, refer to Data Warehouse Protector.

Database Protectors

The following table shows supported input data types for Database protectors with the Numeric token.

Table: Supported input data types for Database protectors with Numeric token

Note: For numeric data elements where length preservation is not enabled, the maximum supported length is 3,842 characters. Data up to this length can be tokenized and de-tokenized without errors.

For more information about Database protectors, refer to Database Protectors

8.1.4.2 - Integer (0-9)

The Integer token type tokenizes 2, 4, or 8 byte size integers.

Table: Integer Tokenization Type properties

The following table shows examples of the way in which a value will be tokenized with the Integer token.

Table: Examples of Integer tokenization values

The pty.ins_integer UDF in the Oracle, Teradata, and Impala Protectors, supports input data length of 4 bytes only. For 2 bytes, the following error is returned: Invalid input size.

Integer Tokenization Properties for different protectors

Application Protector

The following table shows supported input data types for Application protectors with the Integer token.

Table: Supported input data types for Application protectors with Integer token

If the user passes a 4-byte integer with values ranging from -2,147,483,648 to +2,147,483,647, the data element for the protect, unprotect, or reprotect APIs should be an 4-byte integer token type. However, if the user uses 2-byte integer token type, the data protection operation will not be successful. For a Bulk call using the protect, unprotect, and reprotect APIs, the error code, 44, appears. For a single call using the protect, unprotect, and reprotect APIs, an exception will be thrown and the error message, 44, Content of input data is not valid appears.

For more information about Application protectors, refer to Application Protector.

Big Data Protector

Protegrity supports MapReduce, Hive, Pig, HBase, Spark, and Impala, which utilizes Hadoop Distributed File System (HDFS) or Ozone as the data storage layer. The data is protected from internal and external threats, and users and business processes can continue to utilize the secured data. Protegrity protects data inside the files using tokenization and strong encryption protection methods.

The following table shows supported input data types for Big Data protectors with the Integer token.

Table: Supported input data types for Big Data protectors with Integer token

^*1 – If the input and output types of the API are BYTE[], then the customer application should convert the input to and output from the byte array, before calling the API.

^*2 – The Protegrity MapReduce protector, HBase coprocessor, and Spark protector only support bytes converted from the string data type. Bytes as input that are not generated from string data type might cause data corruption to occur when:

• Any other data type is directly converted to bytes should be passed as input to the MapReduce or Spark API that supports byte as input and provides byte as output.
• Any other data type is directly converted to bytes and inserted in an HBase table. Where the HBase table is configured with the Protegrity HBase coprocessor.

For more information about Big Data protectors, refer to Big Data Protector.

Data Warehouse Protector

The Protegrity Data Warehouse Protector is an advanced security solution designed to protect sensitive data at the column level. This enables you to secure your data, while still permitting access to authorized users. Additionally, the Data Warehouse Protector integrates seamlessly with existing database systems using the User-Defined Functions for an enhanced security. Protegrity protects data inside the data warehouses using various tokenization and encryption methods.

The following table shows the supported input data types for the Teradata protector with the Integer token.

Table: Supported input data types for Data Warehouse protectors with Integer token

For more information about Data Warehouse protectors, refer to Data Warehouse Protector.

Database Protectors

The following table shows supported input data types for Database protectors with the Integer token.

Table: Supported input data types for Database protectors with Integer token

For more information about Database protectors, refer to Database Protectors

8.1.4.3 - Credit Card

The Credit Card token type helps maintain transparency. It provides ways to clearly distinguish a token from the real value which is a recommendation of the PCI DSS. The Credit Card token type supports only numeric input (no separators are allowed as input).

Table: Credit Card Tokenization properties

The credit card number real value is distinguished from the tokenized value based on the token value validation properties.

Table: Specific Properties of the Credit Card Token Type

You can create a Credit Card token element and select no validation property for it. If the Credit Card token is involved, it will be handled similar to a Numeric token. However, additional checks will be applied to the input based on the properties detailed in the Credit Card token general properties column in the table above.

To enable the Credit Card token properties, such as, Invalid LUHN checksum and Invalid Card Type, with the SLT Tokenizers, refer to Credit Card Properties with SLT Tokenizers.

Invalid Luhn Checksum

The purpose of the Luhn checksum is to detect incorrectly entered card details. If you enable Invalid Luhn Checksum token validation, then you must use valid credit cards otherwise tokenization will be denied for an invalid credit card number.

A valid credit card has a valid Luhn checksum. Upon tokenization, the tokenized value will have an invalid Luhn checksum. Here is an example of the tokenized credit card with the invalid Luhn digit.

Table: Credit Card Number with Luhn Checksum Examples

Invalid Card Type

An invalid credit card indicates an issue with the credit card details. An invalid card type will result in token values not starting with the digits that real credit card numbers begin with. The first digit in a real credit card number is the Major Industry Identifier. Thus, digits 3,4,5,6, and 0 can be the first digits of the real credit card number, which are then substituted during tokenization.

Table: Real Credit Card Values with Tokenized Values

Here is an example of the tokenized credit card with the invalid card type.

Table: Credit Card Number with Invalid Card Type Examples

Alphabetic Indicator

The alphabetic indicator replaces the tokenized value with an alphabet. If you enable Alphabetic Indicator validation, then the resulting token value will have one alphabetic character.

You will need to choose the position of the alphabetic character before tokenizing a credit card number otherwise the resulting token will have no alphabetic indicator.

The alphabetic indicator will substitute the tokenized value according to the following rule:

Table: Alphabetic Indicator with Tokenized Digits

In the following table, the Visa Card Number “4067604564321454” is tokenized. A tokenized value, represented by “7594107411315001”, is substituted with an alphabetic character in a selected position.

Table: Examples of Credit Card Tokenization with Alphabetic Indicator

Credit Card Properties with SLT Tokenizers

The Credit Card Properties with SLT Tokenizers explains the minimum data length required for tokenization. This occurs when the Credit Card token properties is used in combination with the SLT Tokenizers.

If you enable Credit Card token properties for tokenization, such as Invalid LUHN checksum and Invalid Card Type, you need to select an appropriate SLT Tokenizer. This is required to ensure the minimum data length is available for successful tokenization.

The following table represents the minimum data length required for tokenization as per the usage of Credit Card token properties with the SLT Tokenizers.

Table: Minimum Data Length - Credit Card Token Properties with SLT Tokenizers

Credit Card Tokenization Properties for different protectors

Application Protector

The following table shows supported input data types for Application protectors with the Credit Card token.

Table: Supported input data types for Application protectors with Credit Card token

^*1 - The API accepts and returns data in BYTE[] format. The customer application needs to convert the input into byte arrays before calling the API, and similarly, convert the output from byte arrays after receiving the response from the API.

^*2 - The Protegrity Application Protector only supports bytes converted from the string data type. If any other data type is directly converted to bytes and passed as input to the Application Protectors APIs that support byte as input and provide byte as output, then data corruption might occur.

For more information about Application protectors, refer to Application Protector.

Big Data Protector

Protegrity supports MapReduce, Hive, Pig, HBase, Spark, and Impala, which utilizes Hadoop Distributed File System (HDFS) or Ozone as the data storage layer. The data is protected from internal and external threats, and users and business processes can continue to utilize the secured data. Protegrity protects data inside the files using tokenization and strong encryption protection methods.

The following table shows supported input data types for Big Data protectors with the Credit Card token.

Table: Supported input data types for Big Data protectors with Credit Card token

^*1 – If the input and output types of the API are BYTE[], then the customer application should convert the input to and output from the byte array, before calling the API.

^*2 – The Protegrity MapReduce protector, HBase coprocessor, and Spark protector only support bytes converted from the string data type. Bytes as input that are not generated from string data type might cause data corruption to occur when:

• Any other data type is directly converted to bytes should be passed as input to the MapReduce or Spark API that supports byte as input and provides byte as output.
• Any other data type is directly converted to bytes and inserted in an HBase table. Where the HBase table is configured with the Protegrity HBase coprocessor.

For more information about Big Data protectors, refer to Big Data Protector.

Data Warehouse Protector

The Protegrity Data Warehouse Protector is an advanced security solution designed to protect sensitive data at the column level. This enables you to secure your data, while still permitting access to authorized users. Additionally, the Data Warehouse Protector integrates seamlessly with existing database systems using the User-Defined Functions for an enhanced security. Protegrity protects data inside the data warehouses using various tokenization and encryption methods.

The following table shows the supported input data types for the Teradata protector with the Credit Card token.

Table: Supported input data types for Data Warehouse protectors with Credit Card token

For more information about Data Warehouse protectors, refer to Data Warehouse Protector.

Database Protectors

The following table shows supported input data types for Database protectors with the Credit card token.

Table: Supported input data types for Database protectors with Credit Card token

For more information about Database protectors, refer to Database Protectors

8.1.4.4 - Alpha (A-Z)

The Alpha token type tokenizes both uppercase and lowercase letters.

Table: Alpha Tokenization Type properties

The following table shows examples of the way in which a value will be tokenized with the Alpha token.

Table: Examples of Numeric tokenization values