For the Cloud Protector Approach

The prerequisites required to install and run the Big Data Protector on a Databricks Compute are listed below.

• Python3 along with the requests module is installed on the machine to execute the configurator script.

• A compatible version of ESA is installed, configured, and running.

• Access to the Databricks workspace is available.

• A Databricks cluster, of any one of the following type, is created and is in the running state:

  • Dedicated Compute
  • Standard Compute
  • SQL Warehouse

• Create the Databricks Service Principal.

• The Databricks Service Principal must have the Can attach to permission on the cluster.

• Install and configure the Cloud API on AWS.

  Note: For more information about installing and configuring the Cloud API on AWS, refer Cloud API.

• To modify the core parameters for RPSync, refer https://docs.protegrity.com/cloud-protect/4.0.0/docs/aws/api/installation/agent/#policy-agent-lambda-configuration.

• Install and configure a compatible version of ESA.

  Note: For more information about compatible ESA versions, refer Cloud API.

• Create an AWS Databricks Unity Catalog Service Credential.

  Note: For more information about creating the credential, refer to https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-services/service-credentials.

• Assigned the ACCESS privilege to the principals that will be using the AWS Databricks Unity Catalog Service Credential.

• Create a service principal and OAuth secret to deploy the UDFs.

  Note: For more information, refer to https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m?language=Connect.

• (Optional) Configure private connectivity to the Protegrity Cloud API.

  Note: For more information, refer to https://docs.databricks.com/aws/en/security/network/serverless-network-security/pl-to-internal-network.

• A Databricks Unity Catalog Volume is available with a Catalog and a Schema and the following permissions:

  • The Databricks Service Principal must have the ATTACH or MANAGE permission on the compute.
  • The Databricks Service Principal must have the Read volume and Write volume permission on the Databricks Unity Catalog Volume.
  • The Databricks Service Principal must have the Use catalog permission at the Catalog level.
  • The Databricks Service Principal must have the Use schema permission at the Schema level.
  • The Databricks Service Principal must have the Create function permission at the Schema level.
  • The Databricks Service Principal must have the manage permission at the Schema level.

• To use a SQL Warehouse with the Cloud Protector approach, create a SQL Warehouse. For more information, refer https://docs.databricks.com/aws/en/compute/sql-warehouse/create.