GCP on

Mon, 01 Jan 0001 00:00:00 +0000

Certificates on ESA

By default, ESA is configured with self-signed certificates, which can only be validated using self-signed CA certificate supplied in Cloud Function Environment variables configuration.

In case ESA is configured with publicly signed certificates, this section can be skipped since the Cloud Function will use public CA to validate ESA certificates.

To obtain self-signed CA certificate from ESA:

Log in to ESA Web UI.
Select Settings > Network > Manage Certificates.

Mon, 01 Jan 0001 00:00:00 +0000

Creating ESA Credentials

Policy Agent Function requires ESA credentials to be provided as one of the two options:

Note

The username and password of the ESA user requires role with DPS Admin and Export Certificates permissions. Security Administrator is one of the predefined roles which contains the above permissions, however for separation of duties it is recommended to create custom role.

Secret Manager

Secret Manager is the recommended option for storing ESA credentials.

Mon, 01 Jan 0001 00:00:00 +0000

Custom Cloud Function

If you have the skills to write code, you may provide a custom Cloud Function that returns the ESA credentials to the Policy Agent. One use case is when reading the ESA credentials from a third-party password vault.

Create the Cloud Function:

Create a new 2nd gen Cloud Function using any runtime.
1. The Policy Agent does not provide an input payload.
2. The Cloud Function must return a response according to the following schema:

Mon, 01 Jan 0001 00:00:00 +0000

Secret Manager

Secret Manager is the recommended option for storing ESA credentials.

Create ESA credentials secrets:

Log in to Google Account and select project where Protegrity service will be installed.
Go to Security > Secret Manager.
Select CREATE SECRET.

Specify the Secret Value:

{
 "username": "{esa_username}", 
 "password": "{esa_password}"
}

Select Create Secret.
Once the secret is created, you should see the secret screen opened. If not click on the secret name to see a screen with secret versions.

Mon, 01 Jan 0001 00:00:00 +0000

ESA Server

Policy Agent function requires ESA server running and accessible from Agent Cloud Function on TCP port 8443. Make sure inbound connections on TCP:8443 are allowed for the network where ESA is hosted.

Note down ESA IP address:

ESA IP Address (EsaIpAddress): ___________________

Mon, 01 Jan 0001 00:00:00 +0000

Identify or Create a new VPC

Google Cloud VPC is used to route traffic from Policy Agent Cloud Function to ESA. If your ESA is in a Google Cloud VPC, it is recommended to create a serverless VPC access and record its name:

google_vpc_access_connector_name: ___________________

Note

For more information on serverless VPC connector, refer to the following link. https://cloud.google.com/vpc/docs/configure-serverless-vpc-access

If ESA is not on Google Cloud VPC, you can either create one or choose to let Terraform script to create one. The Terraform script will create the following elements:

Mon, 01 Jan 0001 00:00:00 +0000

Install Policy Agent Function through Terraform Scripts

Agent Terraform scripts provided by Protegrity create a Cloud Function in your Google account. If you don’t specify the deployment bucket Terraform parameter, a new storage bucket will also be created. You can also create the following optional resources by specifying the corresponding parameters:

Service account with IAM role
VPC with NAT external IP
VPC access connector

To install Policy Agent Function through Terraform:

Mon, 01 Jan 0001 00:00:00 +0000

Test Agent Function Installation

After configuration is complete, you can test the function.

To test and run the Policy Agent Function:

From the Google Cloud console, go to Cloud Run Functions or Cloud Run.
Click on the function you just deployed: pty_agent_{deployment_id}.
Click Test button at the top right section of the screen.
Scroll down to CLI test command.
Copy and run the curl command to trigger the agent. Alternatively, use the option Test in Cloud Shell.

Mon, 01 Jan 0001 00:00:00 +0000

Troubleshooting

Configure additional logging:

Set log_level Terraform variable on the Agent function to DEBUG.

In the GCP Logs Explorer, you can run the query below, replacing placeholders with your deployment id and project name.

resource.type="cloud_run_revision"
resource.labels.service_name=~"pty-agent-<deploymentd-id>"
severity=ERROR OR textPayload=~"\[error\]"
-logName="projects/<gcp-project-id>/logs/run.googleapis.com%2Frequests"

Expand each log entry for more details. Check for jsonPayload > exception to see more detailed error.

Error message

Details

iap_agent_gcp.cloud_functions_util.CloudFunctionsApiException: Resource 'projects/<account>/locations/<region>/functions/protegrity-protect-<deployment-id>' was not found

This error may indicate the following configuration issues:

The function name indicated in setting gcp_protect_function_resource_name has been provided incorrectly, and thus cannot be found.
disable_deploy has been set, and a dummy function has been entered to work around the gcp_protect_function_resource_name requirement. The Agent deployment requires a deployed Protect or Log Forwarder Cloud Run function to operate.

[ERROR] policy_agent:Invalid GCP_PROTECT_FUNCTION_RESOURCE_NAME parameter value. Must be a comma separated list of Lambda Function names or ARNs.

This error may indicate the following configuration issues:

The setting gcp_protect_function_resource_name is empty. The Agent deployment requires a deployed Protect or Log Forwarder Cloud Run function to operate, this setting may not be left empty.
The list of function names provided to gcp_protect_function_resource_name contains invalid function name or is not valid csv format.

[ERROR] iap_agent_gcp.cloud_functions_util:<HttpError 403 when requesting https://cloudfunctions.googleapis.com/v2/projects/<account>/locations/<region>/functions/pty-protect-<deployment-id>:generateDo
wnloadUrl?alt=json returned "Permission 'cloudfunctions.functions.sourceCodeGet' denied on 'projects/<account>/locations/<region>/functions/<deployment-id>'". Details: "Permission 'cloudfunctions.functions.sourceCodeGet' denied on 'projects/<account>/locations/<region>/functions/pty-protect-<deployment-id>'">
[ERROR] policy_agent:Permission 'cloudfunctions.functions.sourceCodeGet' denied on 'projects/<account>/locations/<region>/functions/pty-protect-<deployment-id>'
...
iap_agent_gcp.cloud_functions_util.CloudFunctionsApiException: Permission 'cloudfunctions.functions.sourceCodeGet' denied on 'projects/<account>/locations/<region>/functions/pty-protect-<deployment-id>'

Indicates the Agent Cloud Run function’s identity does not have permissions to sourceCodeGet for Protect/Log Forwarder function(s) provided to the gcp_protect_function_resource_name configuration.

Mon, 01 Jan 0001 00:00:00 +0000

Recovering Logs in Dead Letter Topic (Alternative)

When the recommended method of for recovery described in Recovering Logs in Dead Letter Topic (Recommended) is not an option, you may use the existing Log Forwarder to reprocess undelivered logs.

Mon, 01 Jan 0001 00:00:00 +0000

Protegrity Cloud Protect Log Forwarder installation provides a solution to recover undelivered audit logs. Reasons for undeliverable logs may include:

Changes to network configuration in ESA or cloud provider (VPC, firewall, certificate rotation, service user credentials)
Log Forwarder IAM Service Account permissions
Log Forwarder Cloud Run Function configuration
Disruption in cloud provider service

Mon, 01 Jan 0001 00:00:00 +0000

Log Forwarder Dead Letter Pub/Sub Architecture

Log Forwarder is triggered by pub/sub events generated by Protect Functions. If Log Forwarder is unable to reach ESA to deliver the logs, they are pushed to a dead letter pub/sub topic. Dead letter pub/sub topic is created when installing the Log Forwarder with the service installation script. See Install Log Forwarder Function via Terraform Scripts for dead letter topic configuration options and naming conventions.

Mon, 01 Jan 0001 00:00:00 +0000

Monitoring Undelivered Logs

Logs pushed to the dead letter pub/sub topic will be purged and no longer recoverable when specified dlq_topic_message_retention_duration has been reached. Monitoring the dead letter topic is recommended to ensure timely recovery of audit messages before they are permanently lost. Consult the GCP monitoring alerts documentation for setting up alerts based on pub/sub topic metrics.

Mon, 01 Jan 0001 00:00:00 +0000

Recovering Logs in Dead Letter Topic (Recommended)

Protegrity recommends creation of an additional Log Forwarder installation in the case where logs are not delivered to ESA, as described in Log Forwarder Dead Letter Pub/Sub Architecture.

Mon, 01 Jan 0001 00:00:00 +0000

Configuring Regular Expression to Extract Policy Username

Cloud Protect Cloud Function exposes USERNAME_REGEX configuration to allow extraction of policy username from user in the request.

USERNAME_REGEX Cloud Function Environment configuration

The USERNAME_REGEX environment variable can be set to contain regular expression with one capturing group. This group is used to extract the username. Examples below show different regular expression values and the resulting policy user.

USERNAME_REGEX

User in the request

Mon, 01 Jan 0001 00:00:00 +0000

Certificates on ESA

By default, ESA is configured with self-signed certificates, which can only be validated using self-signed CA certificate supplied in Log Forwarder configuration.

Note

Certificate Validation can be bypassed for testing purposes, see section: Install Log Forwarder via Terraform

In case ESA is configured with publicly signed certificates, this section can be skipped since the Log Forwarder will use public CA to validate ESA certificates.

To obtain self-signed CA certificate from ESA:

Mon, 01 Jan 0001 00:00:00 +0000

Create Log Forwarder Service Account

To create Log Forwarder Service Account:

Log in to Google Account and select the project where Protegrity service will be installed.
Navigate to IAM & Admin > Service Accounts.
Select CREATE SERVICE ACCOUNT.
Specify service account name and description.
Select Create and Continue.
In the next step, click Select Role. Then select the following roles:
- Cloud KMS CryptoKey Decrypter
- Pub/Sub Publisher
- Secret Manager Secret Accessor
Click Done.

Mon, 01 Jan 0001 00:00:00 +0000

Create Service Account For Forwarder Pub/Sub

Pub/Sub service requires Cloud Run Invoker permissions in order to be able to send messages to the Forwarder function.

Log in to Google Account and select the project where Protegrity forwarder will be installed.
Navigate to IAM & Admin > Service Accounts.
Select CREATE SERVICE ACCOUNT.
Specify service account name and description.
Select Create and Continue.
In the next step, click Select Role. Then select Cloud Run Invoker.

Mon, 01 Jan 0001 00:00:00 +0000

ESA Audit Store Configuration

ESA server is required as the recipient of audit logs. Verify the information below to ensure ESA is accessible and configured properly.

ESA server running and accessible on TCP port 9200.
Audit Store service is configured and running on ESA. For information related to ESA Audit Store configuration, refer to Audit Store Guide.

Mon, 01 Jan 0001 00:00:00 +0000

Certificate Authentication

Certificate authentication uses a public certificate and a private certificate key. Consult Audit Store Guide on how to set up certificate authentication on ESA and how to download certificate and certificate key. Certificate contains no sensitive information. Certificate key contains private information and for this reason is stored in GCP Secret Manager. Both certificate and certificate key must be converted to single-line values using code similar to the following Powershell code snippet:

Mon, 01 Jan 0001 00:00:00 +0000

ESA Authentication

Audit Log Forwarder must authenticate with ESA using certificate-based authentication with client certificate and certificate key. Download the following certificates from the /etc/ksa/certificates/plug directory of the ESA:

File Name	Description
client.key	Client certificate key
client.pem	Client certificate (PEM)

Both certificate and certificate key must be converted to single-line values using code similar to the following examples.

Client certificate (client.pem):

$folder = 'C:\Temp'
cd $folder
(Get-Content "$folder\client.pem") -join '\n' | Set-Content "$folder\one-liner-client.pem"
cat "$folder\one-liner-client.pem"

folder="/tmp"
cd "$folder"
awk 'NF {printf "%s\\n",$0}' "client.pem" > "one-liner-client.pem"
cat "one-liner-client.pem"

Client certificate key (client.key):

Mon, 01 Jan 0001 00:00:00 +0000

Install Log Forwarder Function via Terraform Scripts

Resources created with Terraform scripts include Audit Log Forwarder Cloud Functions Service and Pub/Sub topic. If you don’t specify the deployment bucket Terraform parameter, a new storage bucket will also be created. You can optionally choose to create a new service account with custom IAM role.

To install using Terraform:

From the command shell move to directory where you downloaded Protegrity installation bundle.

Mon, 01 Jan 0001 00:00:00 +0000

Google Cloud Project

Identify or create a new Google Cloud Project where the Protegrity solution will be installed. It is recommended to create a new project. This provides greater security controls and avoids conflicts with other applications that might impact regional account limits. An individual with the Owner role will be required for some of the subsequent installations.

Google Project ID: ___________________

Google Project Number: ___________________

Google Cloud Region: ___________________

Mon, 01 Jan 0001 00:00:00 +0000

Preparation

Ensure that all the steps in Google Cloud Project are performed.
Log in to the Google Cloud account where Protegrity will be installed.
Select the project.
Ensure that you have access to shell command on your computer or Cloud Shell with Terraform CLI v0.14 or higher installed.
Ensure that the Terraform scripts provided by Protegrity are available on your local computer.

Mon, 01 Jan 0001 00:00:00 +0000

Protect Function Pub/Sub Log Output

Protect function must be configured to output audit logs to Pub/Sub topic.

To configure Protect function audit log output:

Go to Protect function Terraform deployment.
Navigate to pty-protect-gcp/main.tf.
Set Terraform variable pty_log_output=“pub_sub”.
Set Terraform variable pty_pub_sub_topic to log forwarder Pub/Sub topic.

Note
You can obtain the topic resource name from Log Forwarder Terraform output: audit_log_topic.
Run terraform apply.

Mon, 01 Jan 0001 00:00:00 +0000

Configure ESA Secrets In GCP Secret Manager

Audit Log Forwarder Function uses GCP Secret Manager to store ESA Audit Store credentials used during authentication.

For information on how to configure basic and certificate authentication for Audit Store on ESA refer to Audit Store Guide.

Log in to Google Account and select project where Protegrity service will be installed.
Go to Security > Secret Manager.
Select CREATE SECRET.
Specify the Secret Value:

Mon, 01 Jan 0001 00:00:00 +0000

Google Cloud Storage

Cloud Storage buckets are required for the Gen 2 Cloud Function sources, the Terraform backend, and the deployment of the Protegrity installation artifacts. It is recommended that you create 3 separate buckets to separate files used for different purposes. If you cannot create 3 separate buckets, you may reuse a bucket for multiple purposes.

Create the buckets:

Run the cloud command below to enable the Google Storage Transfer API.

Mon, 01 Jan 0001 00:00:00 +0000

Test Log Forwarder Function Installation

Before continuing with next steps, you can verify whether Log Forwarder Function is installed correctly. This step is optional and can be skipped.

Below you can find example CURL command to test your function.
Before you can execute it, test if you can obtain temporary authentication token. Run the gcloud auth login and then gcloud auth print-identity-token commands. The logged in gcloud user must have the Cloud Run Invoker permissions. Continue to the next step if the command succeeds and prints the token.

Mon, 01 Jan 0001 00:00:00 +0000

Troubleshooting

Configure additional logging:

Set min_log_level Terraform variable on both Protect function and Log Forwarder function to config.

In the GCP Logs Explorer, you can run the query below, replacing placeholders with your deployment id and project name.

resource.type="cloud_run_revision"
resource.labels.service_name=~"pty-(protect|forwarder)-<deploymentd-id>"
severity=ERROR OR textPayload=~"\[error\]"
-logName="projects/<gcp-project-id>/logs/run.googleapis.com%2Frequests"

Expand each log entry for more details. Check for jsonPayload > exception to see more detailed error.

Error message

Details

Pub/Sub configuration error.

Indicates problems with Pub/Sub service configuration/availability.

Mon, 01 Jan 0001 00:00:00 +0000

Turn on Instance-based billing.

Both Protect and Log Forwarder functions must run for a short period of time after all requests are handled. In order for the GCP Cloud Run service to allow that, the Instance-based billing feature must be enabled for both function deployments.

To enable Instance-based billing:

Log in to Google Account and select the project where Protegrity Cloud Run Function was installed.
Navigate to Cloud Run.
Click on the Cloud Function name.

Mon, 01 Jan 0001 00:00:00 +0000

Grant Pub/Sub Publisher Permission to the Protect Function Service Account

Protect function requires permissions to publish audit log messages to Pub/Sub.

Log in to Google Account and select the project where Protegrity service will be installed.
Navigate to IAM & Admin.
Search for protector function service account email recorded in protect service installation step.
Select Edit principal pencil icon.
Select ADD ANOTHER ROLE.
Select Pub/Sub Publisher.
Click Save.

Mon, 01 Jan 0001 00:00:00 +0000

VPC configuration

Similar to Policy Agent Function, log forwarder function requires Google Cloud VPC to route traffic from the function to ESA. Review the VPC configuration steps for agent in section Identify or Create a new VPC. Same VPC connector as the policy agent can be used. Note down VPC connector name:

google_vpc_access_connector_name: ___________________

Mon, 01 Jan 0001 00:00:00 +0000

Log Forwarder Performance

Log forwarder architecture is optimized to minimize the amount of connections and reduce the overall network bandwidth required to send audit logs to ESA. This is achieved with batching and aggregation taking place on two levels. The first level is in protector function instances, where audit logs from consecutive requests to an instance are batched and aggregated. The second level of batching and aggregation takes place in the log forwarder function before audit logs are forwarded to ESA. This section shows how to configure the deployment to accommodate different patterns of anticipated audit log stream. It also shows how to monitor deployment resources to detect problems before audit records are lost.

Mon, 01 Jan 0001 00:00:00 +0000

Performance Considerations

The following factors may affect performance benchmarks:

Cold startup: Cloud Function spends additional time on the initial invocation to decrypt and load the policy into memory. This time can vary depending on the policy size. Once the Function is initialized, subsequent “warm executions” should process quickly.
Size of policy: The size of the policy impacts cold start performance. Larger policies take more time to initialize.
Cloud Function memory: GCP provides more virtual cores based on the memory configuration. The initial configuration of 2048 MB provides a good tradeoff between performance and cost with the benchmarked policy. Memory can be increased to optimize for your individual cases.
Number of security operations (protect or unprotect).
Cloud Function max instances and concurrency quota: The instance limit affects how functions are scaled. By default the limit is not set to allow handling any traffic pattern. The instance limit can be set to prevent abnormally high request levels. Cloud Functions are also subject to maximum quota for concurrent invocations and request rate.
Size of data element: Operations on larger text consume time.

Mon, 01 Jan 0001 00:00:00 +0000

This guide describes how to configure the Protegrity Policy Agent and Log Forwarder to connect to a Protegrity Provisioned Cluster (PPC), highlighting the differences from connecting to ESA.

Key Differences: PPC vs ESA

Feature	ESA 10.2	PPC (this guide)
Datastore Key Fingerprint	Optional/Recommended	Required
CA Certificate on Agent	Optional/Recommended	Optional/Recommended
CA Certificate on Log Forwarder	Optional/Recommended	Not supported
Client Certificate Authentication from Log Forwarder	Optional/Recommended	Not supported
IP Address	ESA IP address	PPC address

Prerequisites

Access to PPC and required credentials.
Tools: curl, kubectl installed.

Policy Agent Setup with PPC

Important

When connecting to PPC, the Policy Agent requires use of a datastore key fingerprint. For connecting to ESA 10.2 with Cloud Protect Policy Agent, the fingerprint is optional but recommended. See Policy Agent Installation for general setup steps.

Follow these instructions as a guide for understanding specific inputs for Policy Agent integrating with PPC:

Mon, 01 Jan 0001 00:00:00 +0000

Cloud Functions Service Accounts

Cloud Functions use the service accounts created in this deployment. You can create Service accounts manually or use the Protegrity Terraform installation script to create one. Each service account requires specific permissions, which must be granted through IAM roles. Run the following steps to create service accounts and configure the required IAM access. If you use Terraform scripts, skip these steps.

Agent Function IAM Role

To create Agent Function IAM Role:

Mon, 01 Jan 0001 00:00:00 +0000

Google Cloud Project

Google Project ID: ___________________

Google Project Number: ___________________

Google Cloud Region: ___________________

Mon, 01 Jan 0001 00:00:00 +0000

Key Management Service

The Google Cloud Key Management Service (KMS) provides Protegrity Serverless solution the ability to encrypt and decrypt the Protegrity Security Policy.

To create KMS Key Ring and Asymmetric Encryption Master Key:

Log in to Google Account and select project where Protegrity service will be installed.
Navigate to Security > Key Management.
Select Create key ring.
Specify key ring name. For example, protegrity-policy-keyring.
select Key ring location which corresponds to the region where Protegrity solution will be installed.

Mon, 01 Jan 0001 00:00:00 +0000

Agent Function IAM Role

To create Agent Function IAM Role:

Log in to Google Account and select project where Protegrity service will be installed.
Navigate to IAM & Admin > Roles, Select CREATE ROLE.
Specify role name and description.
Select ADD PERMISSIONS.
Select the following permissions:
- cloudkms.cryptoKeyVersions.useToEncrypt
- cloudkms.cryptoKeyVersions.viewPublicKey
- secretmanager.versions.access
- storage.objects.get
- storage.objects.create
- storage.objects.delete
- storage.objects.list
- storage.objects.update
- storage.buckets.get
- cloudfunctions.functions.get
- cloudfunctions.functions.update
- cloudfunctions.functions.sourceCodeGet
- cloudfunctions.functions.sourceCodeSet
- iam.serviceAccounts.actAs
Click Add and then Create.

Alternatively, you can run the following command from the Cloud Shell Terminal.

Mon, 01 Jan 0001 00:00:00 +0000

Agent Service Account

To create Agent Service Account:

Log in to Google Account and select project where Protegrity service will be installed.
Navigate to IAM & Admin > Service Accounts.
Select CREATE SERVICE ACCOUNT.
Specify service account name and description.
Select Create and Continue.
In the next step, click Select Role.
Select Custom and select the role created above .
Click Done.
Once the service account is created, the screen should open on the service account. If the screen does not appear, refresh the page with the service account list and select the service account created.

Mon, 01 Jan 0001 00:00:00 +0000

Protect Function IAM role

To create Protect Function IAM role:

Log in to Google Account and select project where Protegrity service will be installed.
Navigate to IAM & Admin > Roles, Select CREATE ROLE.
Specify role name and description.
Select ADD PERMISSIONS.
Select the cloudkms.cryptoKeyVersions.useToDecrypt permission.
Click Add and then Create.

Mon, 01 Jan 0001 00:00:00 +0000

Protect Service Account

To create Protect Service Account:

Log in to Google Account and select the project where Protegrity service will be installed.
Navigate to IAM & Admin > Service Accounts.
Select CREATE SERVICE ACCOUNT.
Specify service account name and description.
Select Create and Continue.
In the next step, click Select Role. Then select Custom and select the role created above .
Click Done.
Once the service account is created, the screen should open on the service account. If the screen does not appear, refresh the page with the service account list and select the service account created.

Mon, 01 Jan 0001 00:00:00 +0000

Google Cloud Storage

Create the buckets:

Run the cloud command below to enable the Google Storage Transfer API.

Mon, 01 Jan 0001 00:00:00 +0000

Google Cloud Services

The following table describes the Google Cloud services that may a part of your Protegrity installation.

Service	Description
Cloud Run Functions	Provides serverless compute for Protegrity protection operations and the ESA integration to fetch policy updates.
API Gateway	Provides the end-point and access control (Required for Snowflake only).
Key Management Service	Provides cryptographic keys for envelope encryption/decryption of the policy.
Secret Manager Service	Stores secrets required during deployment, e.g., ESA credentials.
Cloud Storage Service	Storage location for the encrypted ESA policy package.
Identity and Access Management	Enforces access policies for deployed resources.
Cloud Logging Service	Application and audit logs, performance monitoring, and alerts.
Cloud VPC	Required for securing network access to On-Prem or cloud-based ESA.
Pub/Sub	Provides a messaging service when forwarding audit logs to ESA is enabled.

Mon, 01 Jan 0001 00:00:00 +0000

Preparation

Ensure that all the steps in pre-configuration are performed.
Log in to the Google Cloud account where Protegrity will be installed.
Select the project.
Ensure that you have access to shell command on your computer or Cloud Shell with Terraform CLI v0.14 or higher installed.
Ensure that the Terraform scripts provided by Protegrity are available on your local computer.

Mon, 01 Jan 0001 00:00:00 +0000

Decommission Previous Log Forwarder Version

Verify audit log transmission from old version is complete and remove installation. Before you begin

Warning

Destroying previous Log Forwarder terraform environment may result in loss of audit records. Only proceed after carefully validating that both audit topic and audit dead letter topic are no longer receiving messages from protectors AND do not contain unacknowledged messages.

To remove the previous installation:

Navigate to the previous Log Forwarder installation

Mon, 01 Jan 0001 00:00:00 +0000

Deploy Protect Function Changes

After the upgrade is tested, it can be deployed to production Protect Cloud Function. Follow the instructions below to run terraform and deploy the upgrade.

Note
If you are upgrading protector to one of these versions: [3.2.2, 3.2.3], in the main.tf file set the variable pty_pub_sub_topic to log forwarder Pub/Sub topic. You can obtain the topic resource name from the Terraform output: audit_log_topic for the newly deployed Log Forwarder.
In the main.tf file set upgrade_step variable to “deploy_changes”. Apply Terraform changes.

Mon, 01 Jan 0001 00:00:00 +0000

Upgrading to the Latest Version

You can download the latest version of the deployment package from https://my.protegrity.com. Navigate to Data Protection > Cloud Protect to download the latest version.

Perform the following steps to upgrade the Policy Agent Function and Protect Function separately.

Important

If new versions are available for both Agent and Protect Lambdas, Agent Lambda must be upgraded first.

Mon, 01 Jan 0001 00:00:00 +0000

Reset Agent environment variable for protect function resource name

If the protect function was upgraded, function resource name environment variable must be updated to the original value.

Reinstate the production function resource name in policy agent configuration. See the example below (replace placeholder with values recorded above):
```
GCP_PROTECT_FUNCTION_RESOURCE_NAME=<protect_function_resource_name>
```
Run the Terraform apply command to apply the change.

Mon, 01 Jan 0001 00:00:00 +0000

Resume Protegrity Agent Function Scheduled Job

If the Agent Function Scheduled Job was paused at the beginning of the upgrade process, you must resume it. Follow the steps below to resume the Policy Agent Scheduled Job.

Navigate back to Cloud Scheduler.
Select checkbox next to Protegrity Agent Function, e.g. pty-agent-<deployment-id>
Click Resume button placed on the top action panel.

Mon, 01 Jan 0001 00:00:00 +0000

Install New Log Forwarder Function

Upgrade of log forwarder component requires new Log Forwarder installation.

In the newly downloaded Protegrity Log Forwarder function Terraform directory, open the main.tf configuration file.
Populate Terraform variables values, using your current Log Forwarder Terraform main.tf file as a guide.

Warning
Following values must be changed to avoid conflicts with the current Log Forwarder installation. Recreation of resources when deployment_id is not changed may result in loss of audit logs.

a. prefix in the backend configuration

Mon, 01 Jan 0001 00:00:00 +0000

Load Policy to Log Forwarder Function

Modify Agent Cloud Function policy target with the new log forwarder function resource name. Run the Agent Function to load policy.

Open the main.tf file for Policy Agent.
Replace the current log forwarder function name in gcp_protect_function_resource_name variable (if it exists, otherwise add). Use the new Log Forwarder forwarder_function_resource_name recorded in previous steps.
Warning
Failure to complete this step may result in a faulty upgrade, where the new function code is not used.

Cloud API on GCP v3.2.3 Upgrading to the Latest Version If the value for gcp_policy_version_object_key has not been updated as part of any other component upgrade for the new release, replace gcp_policy_version_object_key with a new object key.

Mon, 01 Jan 0001 00:00:00 +0000

Pause Protegrity Agent Function Scheduled Job

App Function Scheduled Job is used to periodically run Protegrity Agent Function to synchronize policy from ESA. The trigger must be paused temporarily for the time of the upgrade process.

Follow the steps below to pause the Agent Function Scheduled Job.

From GCP Console, go to Cloud Scheduler.
Select checkbox next to Protegrity Agent Function, e.g. pty-agent-<deployment-id>
Click Pause button placed on the top action panel.

Mon, 01 Jan 0001 00:00:00 +0000

Rollback Protect Function Changes

Follow the instructions below to rollback upgrade changes deployed in previous section.

Note
If you are upgrading protector to one of these versions: [3.2.2, 3.2.3], in the main.tf file set the variable pty_pub_sub_topic to log forwarder Pub/Sub topic. You can obtain the topic resource name from the Terraform output: audit_log_topic for the newly deployed Log Forwarder.
In the main.tf file set upgrade_step variable to “rollback_changes”. Apply Terraform changes.

Mon, 01 Jan 0001 00:00:00 +0000

Stage Protect Function Changes

The first step in upgrading Cloud Protect Function is staging the new changes in a separate Cloud Function (Upgrade Protect Function). This is done automatically with Terraform script. Follow the instructions below to update and run Terraform and then test production workloads against Upgrade Protect Function to ensure no unexpected behavior when changes are deployed to production Protect Function.

In the newly downloaded Protegrity Protect Function Terraform directory, open the main.tf configuration file.

Mon, 01 Jan 0001 00:00:00 +0000

Update Log Forwarder Topic References

Cloud Protectors transmit logs to Log Forwarder function by using pub/sub topics. Any protectors utilizing Log Forwarder

Function must be updated to reference the upgraded Log Forwarder pub/sub topic.

Open the main.tf file for Protector.
Replace the current pty_pub_sub_topic value with the new Log Forwarder audit_log_topic recorded in previous steps.
Save the main.tf file and run the Terraform apply command to apply the changes.

Allow a few minutes for the change to take effect. Test the upgrade by invoking Protector such that audit logs are generated. Verify logs are flowing through the new topic by navigating to the pub/sub metrics in GCP console. Additionally validate delivery of audit logs in the ESA Audit Store.

Mon, 01 Jan 0001 00:00:00 +0000

Upgrading Protegrity Policy Agent Function

Note

If the release version of the artifact zip file has not changed since the previous installation, you can skip the Agent Function upgrade.

Upgrade the Policy Agent:

In the newly downloaded Protegrity Agent Function Terraform directory, open the main.tf configuration file.
Populate Terraform variables using values from your current agent Terraform main.tf file.
- If upgrading from a Policy Agent before 3.0.1, you must deploy a new Policy Agent function by changing the deployment_id and terraform backend prefix to new values. Starting in version 3.0.1, the Policy Agent uses 2nd gen Cloud Functions. If using a credentials function, ensure that it is a 2nd gen Cloud Function.
- Otherwise, use the same backend configuration with the same gcs bucket and prefix.
If your current Agent function version is lower than 3.0.13, make sure you set the new Terraform variables below. For more information about the new variables, refer to the Install Policy Agent Function through Terraform Scripts section.

Mon, 01 Jan 0001 00:00:00 +0000

Upgrading Log Forwarder

If the release version of the artifact zip file has not changed since the previous installation, Log Forwarder function upgrade may be skipped.

Note

If you are upgrading protector to one of these versions: [3.2.2, 3.2.3], skip this section.

Mon, 01 Jan 0001 00:00:00 +0000

Log Forwarder Upgrade Prerequisites

Follow the steps below to prepare for upgrade.

Identify current Protegrity Log Forwarder Function Terraform module directory.
Record release_version Terraform variable from pty-log-forwarder-gcp/variables.tf file.

Current Log Forwarder Function Release Version: ___________________
Record all Terraform variables from the main.tf file.
Navigate to directory where new version of Protegrity installation bundle was downloaded.
Unzip the main package. Then unzip protegrity-gcp-{protector}-{version}.zip. Verify that the following files are present:
- pty-log-forwarder-gcp/
- main.tf
- outputs.tf
- protegrity-gcp-{protector}-{version}.zip
- README.md
Open pty-log-forwarder-gcp/variables.tf file and check release-version variable. Compare it with the Current Log Forwarder Function Release Version recorded in the previous step. If the two versions are the same, Log Forwarder Function upgrade can be skipped. If not, continue with the next steps.

Mon, 01 Jan 0001 00:00:00 +0000

Upgrading Protect Function

Protect Function Terraform module contains upgrade functionality with zero downtime release and rollback capabilities.

Diagram below illustrates upgrade process.

Steps in the upgrade process have the following purpose:

Stage - The new version is staged as a separate function in parallel with the production function. The new version can be tested before deploying to production.
Deploy - The new version is deployed to protector function with no interruptions to production traffic.
Rollback - New version can be optionally rolled back to version before upgrade.

Protect Function Upgrade Prerequisites
Follow the steps below to prepare for upgrade.

Mon, 01 Jan 0001 00:00:00 +0000

Protect Function Upgrade Prerequisites

Follow the steps below to prepare for upgrade.

Identify current Protegrity Protect Function Terraform module directory.
Record release-version Terraform variable from pty-protect-gcp/variables.tf file.

Current Protect Function Release Version: ___________________
Record all Terraform variables from the main.tf file.
Navigate to directory where new version of Protegrity installation bundle was downloaded.
Unzip the main package. Then unzip protegrity-gcp-{protector}-{version}.zip. Verify that the following files are present:
- pty-protect-gcp/
- main.tf
- outputs.tf
- protegrity-gcp-{protector}-{version}.zip
- README.md
Open pty-protect-gcp/variables.tf file and check release-version variable. Compare it with the Current Protect Function Release Version recorded in the previous step. If the two versions are the same, Protect Function upgrade can be skipped. If not, continue with the next steps.

Mon, 01 Jan 0001 00:00:00 +0000

2nd Generation Protect Function Upgrade Prerequisites

To run Cloud Protect in Gen 2 Cloud Functions, a new instance of Cloud Protect must be created using the included terraform installation scripts. Upgrading a Gen 1 Cloud Function to Gen 2 is not currently supported for Cloud Protect.

Run the cloud command below to enable Google Storage Transfer API.
```
gcloud services enable storagetransfer.googleapis.com
```
Create Cloud Functions storage bucket below. Replace <gcp-project-number> and <region> placeholders.