This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Configuring the Teradata Data Warehouse Protector

1: Updating the Config.ini File
2: Updating the Output Buffer
3: Troubleshooting

This section outlines the configuration process for the Protegrity Teradata Data Warehouse Protector.

1 - Updating the Config.ini File

Log in to the server as the user with the required permissions.
Navigate to the directory where you have downloaded the installation package.
To view the contents within the directory, run the following command:
```
/opt/protegrity/databaseprotector/teradata/data #  ls -ltr
```

Press ENTER.
The list of available configurable files appears.

total 4
-rw-r----- 1 tdatuser tdtrusted 1058 Oct 14 01:27 config.ini

To open the config.ini file, run the following command:

/opt/protegrity/databaseprotector/teradata/data # vim config.ini

Press ENTER.
The contents of the config.ini file appears.

###############################################################################
# Log Provider Config
###############################################################################
[log]

# In case that connection to fluent-bit is lost, set how audits/logs are handled
# 
# drop  : (default) Protector throws logs away if connection to the fluentbit is lost
# error : Protector returns error without protecting/unprotecting 
#         data if connection to the fluentbit is lost
mode = drop

# Host/IP to fluent-bit where audits/logs will be forwarded from the protector
#
# Default localhost
host = localhost

###############################################################################
# Protector Config
###############################################################################
[protector]

# cadence is used to decide whether deployment is dynamic or immutable.
#
# '0' is used for immutable deployment.
# Non-negative values other than '0' is used as policy sync interval for dynamic deployment.
# default cadence value is '60'.
cadence = 60

Update the parameters as mentioned in the table.

The following table consists of the config.ini parameters along with the descriptions:

Configuration Component	Parameter	Description
Log	mode	Specifies how the protector logs are handled by the Log Forwarder. If the connection to the Log Forwarder host is lost, you can set the connection mode to one of the following types: - drop: Specifies the logs that the protector fails to record when the connection to the Log Forwarder is lost. By default, the Log Forwarder is configured to operate in the drop mode. - error: Stops all the data security operations and throws an error when the connection to the Log Forwarder is lost. Syntax: Parameter = Value Example: mode = error
	host	Specifies the Log Forwarder hostname or the IP address where the logs are forwarded from the protector. The default host for the Log Forwarder is localhost. Syntax: Parameter = Value Example: host = <Hostname or IP Address>
Protector	cadence	Specifies the time interval at which the protector synchronizes with the shared memory for fetching the policy package. The default value for the cadence parameter is 60 seconds. The minimum and maximum values that can be set for the cadence parameter are 0 seconds and 86400 seconds (24 hours) respectively. Important: If the cadence parameter value is set to 0 seconds, then the policy is fetched only once at the time of initialization. After initialization, the protector does not fetch for the new policy changes as a result of immutable deployment. Syntax: Parameter = Value Example: cadence = <time interval in seconds>

Save the changes to the config.ini file.
Important: Restart the Teradata Database to reflect any changes made to the config.ini file.

2 - Updating the Output Buffer

This page discusses the process to update the output buffer length for the Varchar Unicode UDFs.

By default, the value of the output buffer length is 500 characters. This value can be modified during the installation of the Teradata objects.

After completing the installation process, you may need to manually update the output buffer length values if necessary. For instance, if you need to protect strings longer than 500 bytes, adjust the buffer length to accommodate the largest string size. Be aware that a big buffer size slows the overall performance. Additionally, each protection method has a size limitations. For example, tokenization has a maximum size limit of 4096 bytes. The output buffer sizes for all the UDFs are stored in both, the dbpuserconf.ini and createvarcharunicode.sql files.

Updating the createvarcharunicode.sql file

Log in to the server as the user with the required permissions.
Navigate to the /opt/protegrity/databaseprotector/teradata/sqlscripts/ directory.
To update the output buffer length in the createvarcharunicode.sql file, run the following command:
```
vim createvarcharunicode.sql
```
Press ENTER.
The contents of the createvarcharunicode.sql file appears.
Ensure to update the value of the output buffer length for the PTY_VARCHARUNICODEINS, PTY_VARCHARUNICODESEL, and PTY_VARCHARUNICODESELEX UDFs as per requirements.
Save the changes to the createvarcharunicode.sql file:
Important: To reflect any changes made to the createvarcharunicode file, restart the Teradata Database.

Updating the dbpuserconf.ini file

Log in to the server as the user with the required permissions.
Navigate to the directory where you have downloaded the dbpuserconf.ini file.
For example, /etc/protegrity/
To view the contents within the directory, run the following command:
```
/etc/protegrity/ #  ls -ltr
```

Press ENTER.
The list of available configurable files appears.

total 4
-rw-r----- 1 tdatuser tdtrusted 1058 Jan 28 01:27 dbpuserconf.ini

Open the dbpuserconf.ini file in any text editor.

 ###############################################################################
 # Config ini
 ###############################################################################
 [config_ini]
 # path points to database protector installation directory
 path = /opt/protegrity/databaseprotector/teradata/data/config.ini

 ###############################################################################
 # Protector Varchar Sizes (set by user during installation)
 ###############################################################################
 [varchar_sizes]
 UDF_VARCHAR_MAX = 500
 UDF_VARCHAR_OVERHEADMAX = 500
 VARCHAR_MAX_IN_BUF_LEN_TOKEN_LATIN = 500
 VARCHAR_MAX_OUT_BUF_LEN_TOKEN_LATIN = 676
 VARCHAR_MAX_IN_BUF_LEN_ENC_LATIN = 500
 VARCHAR_MAX_OUT_BUF_LEN_ENC_BYTES = 538
 VARCHAR_MAX_IN_BUF_LEN_TOKEN_UNICODE = 500
 VARCHAR_MAX_OUT_BUF_LEN_TOKEN_UNICODE = 1356
 VARCHAR_MAX_IN_BUF_LEN_ENC_UNICODE = 500
 VARCHAR_UNICODE_MAX_OUT_BUF_LEN_ENC_BYTES = 1038
 TdvmDev2:/etc/protegrity/ #

Important: Update the VARCHAR_MAX_OUT_BUF_LEN_TOKEN_UNICODE parameter with the required output buffer length.

Update the parameters as per requirements.
Save the changes to the dbpuserconf.ini file.

3 - Troubleshooting

This section lists the general configuration steps and the common errors that occur during installation or upgrade.

Recovering a Failed Upgrade

There can be scenarios where an automatic rollback of the Teradata Data Warehouse Protector UDF solution may complete with errors. This results in the system being in a potentially inconsistent state. In such instances, the installer retains the backup directories of the previously working installation. The system can be manually restored to the previous working installation.

Important:
Execute the steps using the appropriate system user.
Ensure the availability of appropriate operating system level and Teradata database privileges.
Execute the steps in the specified order.
Commands assume a Linux/Unix environment.
Use extreme caution when running rsync commands with the --delete option.

When a rollback operation fails, the installer retains the following backup directories (example with timestamp):

<path_to_previous_installation_dir>/logforwarder_<timestamp>
<path_to_previous_installation_dir>/rpagent_<timestamp>
<path_to_previous_installation_dir>/databaseprotector_<timestamp>
/etc/protegrity_<timestamp>

When a component is installed by manually executing the script, it is installed under a component-specific subdirectory within the user-provided installation directory:

Logforwarder → <installation_dir>/logforwarder/
RPAgent → <installation_dir>/rpagent/
Database Protector → <installation_dir>/databaseprotector/

However, the automation script will install the components and the protector in version-specific directories under the installation directory. For example, the components will be installed in the following structure:

<installation_dir>/<DBP_version>/<logforwarder>
<installation_dir>/<DBP_version>/<rpagent>
<installation_dir>/<DBP_version>/<databaseprotector>

The backup directories will also follow the similar structure while upgrading from v10.1.x onwards. In this structure, the current timestamp will also be appended to the directory name. The backup directories include the installation files of these component directories and must be restored into their corresponding target directories.

Verifying the Log Forwarder Status

To verify the status of the Log Forwarder, run the following command:

<installation_dir>/<DBP_version>/logforwarder/bin/logforwarderctrl status

Press ENTER. The script returns the status of the Log Forwarder.

To stop the Log Forwarder, run the following command:

<installation_dir>/<DBP_version>/logforwarder/bin/logforwarderctrl stop

Press ENTER. The command stops the Log Forwarder.
To check for any running instances of the Log Forwarder, run the following command:
```
ps -ef | grep logforwarder
```
Note: This command is useful in scenarios where installation is corrupt and the control command fails to return a valid status.
Press ENTER. The command lists all the running instances of the Log Forwarder along with the respective process ID.
To stop the specific instance of the Log Forwarder, run the following command:
```
kill -9 <logforwarder_process_id>
```
Press ENTER. The command will stop the specific instance of the Log Forwarder.

Verifying the RPAgent Status

To verify the status of the RPAgent, run the following command:

<installation_dir>/<DBP_version>/rpagent/bin/rpagentctrl status

Press ENTER. The script returns the status of the RPAgent.

To stop the RPAgent, run the following command:

<installation_dir>/<DBP_version>/rpagent/bin/rpagentctrl stop

Press ENTER. The command stops the RPAgent.
To check for any running instances of the RPAgent, run the following command:
```
ps -ef | grep rpagent
```
Note: This command is useful in scenarios where installation is corrupt and the control command fails to return a valid status.
Press ENTER. The command lists all the running instances of the RPAgent along with the respective process ID.
To stop the specific instance of the RPAgent, run the following command:
```
kill -9 <rpagent_process_id>
```
Press ENTER. The command will stop the specific instance of the RPAgent.

Performing a Clean-up

Due to the failed rollback, the UDFs and types may be present in the database. Remove them before proceeding with the restoration.

To remove the UDFs and the types:

To navigate to the directory containing the scripts, run the following command:
```
cd <installation_dir>/<DBP_version>/databaseprotector/teradata/sqlscripts
```
Log in to bteq.
To drop the existing objects, run the following command, with a database user that owns the UDFs, or has sufficient privileges to drop them:
```
.run file dropobjects.sql
```
Important: Errors such as “object does not exist” or “Function does not exist” may occur and can be safely ignored depending on the database state.
To drop the varcharunicode UDFs, run the following command as a database user that owns the existing types and UDFs or has sufficient privileges:
```
.run file dropvarcharunicode.sql
```
Important: Errors such as “object does not exist” or “Function does not exist” may occur and can be safely ignored depending on the database state.

Restoring the Component Directories and User Configuration

Restore the contents of all the Protegrity components and configuration directories using the retained backups.

For each component:

Identify the installation directory.
Replace its contents with the corresponding backup directory using a suitable tool such as rsync, cp, or mv.

Note: The Logforwarder, RPAgent, and Database Protector components may be installed in the same directory or in separate directories, depending on the environment.

Important:
Ensure all services are stopped before performing this step.
Do not merge directories manually.
Always fully replace the target directory contents with the backup contents.

Restoring the Log Forwarder for v10.0.x

To navigate to the backup directory, run the following command:
```
<path_to_previous_installation_dir>/logforwarder_<timestamp>
```
To navigate to the installation directory, run the following command:
```
<installation_dir>/logforwarder
```
To restore the Log Forwarder, run the following command:
```
rsync -a --delete <path_to_previous_installation_dir>/logforwarder_<timestamp>/ <installation_dir>/logforwarder/
```
Warning rsync --delete option permanently removes files from the target directory that are not present in the backup. Always verify that the target directory is the correct component directory before executing the command.

Restoring the Log Forwarder for v10.1.x

To navigate to the backup directory, run the following command:

<path_to_previous_installation_dir>/<DBP_version>/logforwarder_<timestamp>

To navigate to the installation directory, run the following command:
```
<installation_dir>/<DBP_version>/logforwarder
```
To restore the Log Forwarder, run the following command:
```
rsync -a --delete <path_to_previous_installation_dir>/<DBP_version>/logforwarder_<timestamp>/ <installation_dir>/logforwarder/
```
Warning rsync --delete option permanently removes files from the target directory that are not present in the backup. Always verify that the target directory is the correct component directory before executing the command.

Restoring the RPAgent for v10.0.x

To navigate to the backup directory, run the following command:
```
<path_to_previous_installation_dir>/rpagent_<timestamp>
```
To navigate to the installation directory, run the following command:
```
<installation_dir>/rpagent
```
To restore the RPAgent, run the following command:
```
rsync -a --delete <path_to_previous_installation_dir>/rpagent_<timestamp>/ <installation_dir>/rpagent/
```
Warning rsync --delete option permanently removes files from the target directory that are not present in the backup. Always verify that the target directory is the correct component directory before executing the command.

Restoring the RPAgent for v10.1.x

To navigate to the backup directory, run the following command:

<path_to_previous_installation_dir>/<DBP_version>/rpagent_<timestamp>

To navigate to the installation directory, run the following command:
```
<installation_dir>/<DBP_version>/rpagent
```
To restore the RPAgent, run the following command:
```
rsync -a --delete <path_to_previous_installation_dir>/<DBP_version>/rpagent_<timestamp>/ <installation_dir>/rpagent/
```
Warning rsync --delete option permanently removes files from the target directory that are not present in the backup. Always verify that the target directory is the correct component directory before executing the command.

Restoring the Teradata Data Warehouse Protector for v10.0.x

To navigate to the backup directory, run the following command:

<path_to_previous_installation_dir>/databaseprotector_<timestamp>

To navigate to the installation directory, run the following command:
```
<installation_dir>/databaseprotector
```
To restore the Teradata Data Warehouse Protector, run the following command:
```
rsync -a --delete <path_to_previous_installation_dir>/databaseprotector_<timestamp>/ <installation_dir>/databaseprotector/
```
Warning rsync --delete option permanently removes files from the target directory that are not present in the backup. Verify that the target directory is the correct component directory before executing the command.

Restoring the Teradata Data Warehouse Protector for v10.1.x

To navigate to the backup directory, run the following command:

<path_to_previous_installation_dir>/<DBP_version>/databaseprotector_<timestamp>

To navigate to the installation directory, run the following command:
```
<installation_dir>/<DBP_version>/databaseprotector
```
To restore the Teradata Data Warehouse Protector, run the following command:
```
rsync -a --delete <path_to_previous_installation_dir>/<DBP_version>/databaseprotector_<timestamp>/ <installation_dir>/databaseprotector/
```
Warning rsync --delete option permanently removes files from the target directory that are not present in the backup. Verify that the target directory is the correct component directory before executing the command.

Restoring the User Configuration

To navigate to the backup directory, run the following command:
```
/etc/protegrity
```
To restore the user configuration, run the following command:
```
rsync -a --delete /etc/protegrity_<timestamp>/ /etc/protegrity/
```
Note: The /etc/protegrity/ directory location does not change across installations or upgrades. This step ensures that all previous configuration settings are fully restored.

Starting the Services for v10.0.x

To start the Log Forwarder, run the following command:

<installation_dir>/logforwarder/bin/logforwarderctrl start

To start the RPAgent, run the following command:

<installation_dir>/rpagent/bin/rpagentctrl start

Starting the Services for v10.1.x

To start the Log Forwarder, run the following command:

<installation_dir>/<DBP_version>/logforwarder/bin/logforwarderctrl start

To start the RPAgent, run the following command:

<installation_dir>/<DBP_version>/rpagent/bin/rpagentctrl start

Recreating the Database Objects for v10.0.x

Due to the failed rollback, the Teradata Data Warehouse Protector types and UDFs may be in an invalid or inconsistent state. If database functionality is not correct, re-create the database objects.

To navigate to the directory containing the scripts, run the following command:
```
cd <installation_dir>/databaseprotector/teradata/sqlscripts
```
Log in to bteq.
To drop the existing objects, run the following command, with a Teradata database user that owns the UDFs, or has sufficient privileges to drop them:
```
.run file dropobjects.sql
```
Important: Errors such as “object does not exist” or “Function does not exist” may occur and can be safely ignored depending on the database state.
To drop the varcharunicode UDFs, run the following command as a database user that owns the existing types and UDFs or has sufficient privileges:
```
.run file dropvarcharunicode.sql
```
Important: Errors such as “object does not exist” or “Function does not exist” may occur and can be safely ignored depending on the database state.
To create the new UDFs, run the following command as a database user having all the required permissions to create the UDFs:
```
.run file=createobjects.sql
```
To create the varcharunicode UDFs, run the following command as a database user having all the required permissions to create the UDFs:
```
.run file=createvarcharunicode.sql
```

These scripts recreate the Teradata Data Warehouse Protector types and UDFs. The database objects are restored to a clean and consistent state. The installation or rollback process is fully recovered from the SQL partial-failure scenario.

Note: If issues persist after manual recovery, contact Protegrity Support and provide the installer log and details of the recovery steps performed.

Recreating the Database Objects for v10.1.x

Due to the failed rollback, the Teradata Data Warehouse Protector types and UDFs may be in an invalid or inconsistent state. If database functionality is not correct, re-create the database objects.

To navigate to the directory containing the scripts, run the following command:
```
cd <installation_dir>/<DBP_version>/databaseprotector/teradata/sqlscripts
```
Log in to bteq.
To drop the existing objects, run the following command, with a Teradata database user that owns the UDFs, or has sufficient privileges to drop them:
```
.run file dropobjects.sql
```
Important: Errors such as “object does not exist” or “Function does not exist” may occur and can be safely ignored depending on the database state.
To drop the varcharunicode UDFs, run the following command as a database user that owns the existing types and UDFs or has sufficient privileges:
```
.run file dropvarcharunicode.sql
```
Important: Errors such as “object does not exist” or “Function does not exist” may occur and can be safely ignored depending on the database state.
To create the new UDFs, run the following command as a database user having all the required permissions to create the UDFs:
```
.run file=createobjects.sql
```
To create the varcharunicode UDFs, run the following command as a database user having all the required permissions to create the UDFs:
```
.run file=createvarcharunicode.sql
```

Note: If issues persist after manual recovery, contact Protegrity Support and provide the installer log and details of the recovery steps performed.

Recovering a Partially Failed Installation

The Teradata Data Warehouse Protector installation and upgrade processes involves creating and dropping a large number of Teradata database types and UDFs. In some scenarios, the SQL scripts can partially fail and may result in an inconsistent database state.

The common scenarios include:

Fresh installation scenario where creation of some types or UDF fails.
Upgrade process where some new objects are created before a failure occurs.
Rollback process where the dropobjects.sql or dropvarcharunicode.sql script encounters objects that were never created or were already dropped.

In such scenarios, the rollback process may log warnings similar to:

[WARN] **************************************************************************
[WARN] IMPORTANT: One or more errors occurred while dropping new or restoring existing types and UDFs during rollback.
[WARN] The database may be in an inconsistent state with respect to UDFs.
[WARN] Manual DBA intervention is required to verify and restore UDF state.
[WARN] 
[WARN] The new Database Protector directory has been PRESERVED for manual cleanup:
[WARN]     Location: <new_databaseprotector_dir> (preserved on master AND all nodes)
[WARN]     Backup of previous Database Protector: <backup_databaseprotector_dir>
[WARN] Refer to the product documentation for manual recovery steps.

[ERROR] IMPORTANT: Rollback completed with some errors. Please check the log <log_file_name> for details
[ERROR] Manual intervention may be required to complete the rollback.
[WARN] Previous installation may be in an inconsistent state. 
[INFO] Backup directories have been retained for manual recovery:
[INFO] Logforwarder backup: <BACKUP_LOGFORWARDER_DIR>
[INFO] RPAgent backup: <BACKUP_RPAGENT_DIR>
[INFO] DatabaseProtector backup: <BACKUP_DBP_DIR>
[INFO] User configuration backup: <BACKUP_USER_CONF_DIR>
[INFO] You can use these backup directories to manually restore the previous working installation if needed.
[INFO] Refer to the product documentation for manual recovery steps using the backup directories.

These warnings are informational and do not automatically require action. However, a manual intervention is only required if the following instances are true:

The installer or rollback logs report SQL-related warnings or errors and
The current state of Teradata Data Warehouse Protector types and UDFs is incorrect or inconsistent.

These errors occur because:

SQL scripts may fail partially because of permission issues, transient database errors, or environmental errors.
During rollback, the dropobjects.sql or dropvarcharunicode.sql script attempts to drop all known objects.
Objects that were never created or were already removed, will generate errors such as:
- object does not exist
- already exists

Such errors are expected in partial-failure scenarios and do not necessarily indicate a fatal problem.

Perform a restore operation ONLY in the following scenarios:

when the verification indicates that database objects are missing, invalid, or corrupted
when the verification indicates that components such as Log Forwarder and RPAgent are missing, invalid, or corrupted

To perform a restore operation, follow the instructions mentioned in Recovering a Failed Upgrade.