Creating the User Defined Functions

The following combinations will work for a successful execution of the configurator script:

Databricks Dedicated Compute + Application Protector REST approach
Databricks Dedicated Compute + Cloud Protector approach
Databricks Standard Compute + Application Protector REST approach
Databricks Standard Compute + Cloud Protector approach
Databricks SQL Warehouse + Cloud Protector approach

The Databricks SQL Warehouse + Application Protector REST approach combination will not work. This is because Protegrity executes a few Python commands on the Databricks Compute to retrieve a listening IP for the Application Protector REST’s Server. When the Databricks Compute is a SQL Warehouse, the Python commands fail to execute. This occurs because the SQL Warehouse supports only SQL commands.

For the Application Protector REST Approach

The configurator script is used to create the UDFs. These Unity Catalog Batch Python UDFs are used to perform data protection and unprotection operations. Select the required approach and the operation ID to create the UDFs using the Application Protector REST server. This section explains the process to create the UDFs using the interactive method of installation.

To create the UDFs:

Log in to the staging machine.
Navigate to the directory where the installation files are extracted.

To execute the configurator script, run the following command:

./BigDataProtector-Configurator_Linux-ALL-64_x86-64_AWS.Databricks-<DBR_version>-64_<BDP_version>.sh

Press ENTER. The prompt to enter the operation ID appears.

Creating installation files...
Created installation files.

Enter the ID of the operation:

To create the UDFs, type 2.
Press ENTER. The prompt to enter the Databricks Workspace URL appears.
```
Enter the URL of the Databricks Workspace:
```
Enter the Databricks Workspace URL.
Press ENTER. The prompt to enter the application ID of the Databricks Service Principal appears.
```
Enter the Application ID of the Databricks Service Principal:
```
Enter the Application ID of the Databricks Service Principal.
Press ENTER. The prompt to enter the OAuth secret for the Service Principal appears.
```
Enter the OAuth Secret of the Databricks Service Principal:
```
Enter the OAuth secret.
Press ENTER. The prompt to enter the cluster ID appears.
```
Enter the ID of the Databricks Compute:
```
Note: The Cluster ID can be either for Standard Compute or Dedicated Compute. For more information about identifying the Cluster ID, refer to https://docs.databricks.com/aws/en/workspace/workspace-details/.
Enter the Cluster ID.

Press ENTER. The prompt to enter the name of the schema appears.

Enter the name of the Databricks Unity Catalog Catalog-Schema:

Enter the name of the catalog and the schema in the <catalog_name.schema_name> format.
Press ENTER. The prompt to select the approach appears.
```
Enter the ID of the approach:
```
To create the UDFs using the Application Protector REST approach, type 1.
Press ENTER. The prompt to enter the path of the CA Certificate appears.
```
Enter the path of the CA Certificate:
```
Enter the path of the CA Certificate.
Press ENTER. The prompt to enter the path of the Server Certificate appears.
```
Enter the path of the Server Certificate:
```
Enter the path of the Server Certificate.
Press ENTER. The prompt to enter the path of the Server key appears.
```
Enter the path of the Server Key:
```
Enter the path of the Server Key.
Press ENTER. The prompt to enter the name of the AWS Secret appears.
```
Enter the name of the AWS Secret:
```
Enter the name of the AWS Secret.
Press ENTER. The prompt to enter the region of the Secret appears.
```
Enter the name of the AWS Secret's AWS Region:
```
Enter the region where the Secret is created.
Press ENTER. The prompt to enter the name of the Service Credential appears.
```
Enter the name of the Databricks Unity Catalog Service Credential:
```
Enter the name of the Databricks Unity Catalog Service Credential.
Press ENTER. The prompt to enter the path of the Unity Catalog Volume appears.
```
Enter the path of the Databricks Unity Catalog Volume:
```
Enter the path of the Databricks Unity Catalog Volume.

Press ENTER. The script creates the UDFs at the specified location.

Executing specified operation...

1. Create the following environment variables in the Spark section of the Advanced properties of the Databricks Compute:
PTY_ESA_IP=PTY_ESA_IP
PTY_ESA_PORT=PTY_ESA_PORT
Either PTY_ESA_TOKEN=PTY_ESA_TOKEN or PTY_ESA_ADMINISTRATOR_USERNAME=PTY_ESA_ADMINISTRATOR_USERNAME and PTY_ESA_ADMINISTRATOR_PASSWORD=PTY_ESA_ADMINISTRATOR_PASSWORD
PTY_AUDIT_STORE_IP_PORT=PTY_AUDIT_STORE_IP_PORT
PTY_PROTECTOR_CONFIGURATION=PTY_PROTECTOR_CONFIGURATION
2. Attach "DATABRICKS_UNITY_CATALOG_VOLUME_PATH/DATABRICKS_INIT_SCRIPT_NAME" as an Init Script to the Databricks Compute.
3. Restart the Databricks Compute.

Executed specified operation.

For the Cloud Protector Approach

The configurator script is used to create the UDFs. These Unity Catalog Batch Python UDFs are used to perform data protection and unprotection operations. Select the required approach and the operation ID to create the UDFs using the Cloud Protector. This section explains the process to create the UDFs using the interactive method of installation.

To create the UDFs:

Log in to the staging machine.
Navigate to the directory where the installation files are extracted.

To execute the configurator script, run the following command:

./BigDataProtector-Configurator_Linux-ALL-64_x86-64_AWS.Databricks-<DBR_version>-64_<BDP_version>.sh

Press ENTER. The prompt to enter the operation ID appears.

Creating installation files...
Created installation files.

Enter the ID of the operation:

To create the UDFs, type 2
Press ENTER. The prompt to enter the Databricks Workspace URL appears.
```
Enter the URL of the Databricks Workspace:
```
Enter the Databricks Workspace URL.
Press ENTER. The prompt to enter the application ID of the Databricks Service Principal appears.
```
Enter the Application ID of the Databricks Service Principal:
```
Enter the Application ID of the Databricks Service Principal.
Press ENTER. The prompt to enter the OAuth secret for the Service Principal appears.
```
Enter the OAuth Secret of the Databricks Service Principal:
```
Enter the OAuth secret.
Press ENTER. The prompt to enter the cluster ID appears.
```
Enter the ID of the Databricks Compute:
```
Note: The Cluster ID can be either for SQL Warehouse, Standard Compute or Dedicated Compute. For more information about identifying the Cluster ID, refer to https://docs.databricks.com/aws/en/workspace/workspace-details/.
Enter the Cluster ID.

Press ENTER. The prompt to enter the name of the schema appears.

Enter the name of the Databricks Unity Catalog Catalog-Schema:

Enter the name of the catalog and the schema in the <catalog_name.schema_name> format.
Press ENTER. The prompt to select the approach appears.
```
Enter the ID of the approach:
```
To create the UDFs using the Cloud Protector approach, type 2.
Press ENTER. The prompt to enter the name of the AWS Lambda Function appears.
```
Enter the name of the AWS Lambda Function:
```
Enter the name of the AWS Lambda Function.
Press ENTER. The prompt to enter the region of the AWS Lambda function appears.
```
Enter the name of the AWS Lambda Function's AWS Region:
```
Enter the region name.
Press ENTER. The prompt to enter the name of the Service Credential appears.
```
Enter the name of the Databricks Unity Catalog Service Credential:
```
Enter the name of the Databricks Unity Catalog Service Credential.
Press ENTER. The script creates the UDFs at the specified location.
```
Executing specified operation...
Executed specified operation.
```

Feedback

Was this page helpful?

Last modified : February 12, 2026