Architecture

Deployment Architecture

The Protegrity product should be deployed in the customer’s Cloud account within the same Google Cloud region as the Snowflake cluster. The product incorporates Protegrity’s vaultless tokenization engine within Google Cloud Functions. The encrypted data security policy from an ESA is deployed periodically as a static resource together with Cloud Function binaries. The policy is decrypted in memory at runtime within the Cloud Function. This architecture allows Protegrity to be highly available and scale very quickly without direct dependency on any other Protegrity services.

The product exposes a remote data protection service invoked from external User Defined Functions (UDFs), a native feature of specific Cloud databases. The UDFs can be invoked through direct SQL queries or database views. The external UDF makes parallel API calls to the serverless Protegrity Cloud function via the GCP API Gateway to perform protect and unprotect data operations. Each network REST request includes a micro-batch of data to process and a secure context header generated by the database with the username and a Protegrity context header with the data element type, and operation to perform. The product applies the ESA security policy including user authorization and returns a corresponding response. Security operations on sensitive data performed by protector can be audited. The product can be configured to send audit logs to ESA via optional component called Log Forwarder.

The security policy is synchronized through another serverless component called the Protegrity Policy Agent. The agent operates on a configurable schedule, fetches the policy from the ESA, performs envelope encryption using Google Key Management Service, and deploys new version of Cloud Function with updated policy. This solution can be configured to automatically provision the static policy or the final step can be performed on-demand by an administrator. There is no downtime for users during this process. Instances provisioned with the function’s previous policy version may continue running (and processing traffic) for several minutes after a deployment has finished.

The following diagram shows the high-level architecture described above.

The following diagram shows a reference architecture for synchronizing the security policy from the ESA to the product.

The Protegrity Policy Agent requires network access from the Cloud to your ESA. Most organizations install the ESA on-premise. Therefore, it is recommended that the Policy Agent is installed into a private subnet with a Cloud VPC using a NAT Gateway to enable this communication through a corporate firewall.

The ESA is a soft appliance that must be pre-installed on a separate server. It is used to create and manage security policies.

For more information about installing the ESA, and creating and managing policies, refer to the Policy Management Section.

Audit Log Forwarding Architecture

Audit logs are by default sent to Cloud Logging. The Protegrity Product can also be configured to send audit logs to ESA. Such configuration requires deploying Log Forwarder component which is available as part of Protegrity Product deployment bundle. The diagram below shows additional resources deployed with Log Forwarder component.

The log forwarder component includes Pub/Sub service topic and the audit log forwarder function. Pub/Sub service is used to asynchronously send audit records to forwarder function, where similar audit logs are aggregated before sending to ESA. Aggregation rules are described in the Protegrity Log Forwarding guide. When the protector function is configured to send audit logs to log forwarder, audit logs are aggregated on the protector function before sending to Pub/Sub topic. Protector function exposes configuration to control the time it spends aggregating audit logs which is described in the protector function installation section.

The security of audit logs is ensured by using HTTPS connection on each link of the communication between protector function and ESA. Integrity and authenticity of audit logs is additionally checked on log forwarder which verifies individual logs signature. The signature verification is done upon arrival from Pub/Sub topic before applying aggregation. If signature cannot be verified, the log is forwarded as is to ESA where additional signature verification can be configured. Log forwarder function uses basic auth and optional certificate verification to authenticate calls to ESA. Basic auth credentials are stored securely in Secret Manager Service.

To learn more about individual audit log entry structure and purpose of audit logs, refer to Audit Logging section in this document. Installation instruction can be found in the Audit Log Forwarder Installation.

The audit log forwarding requires network access from the cloud to the ESA. Most organizations install the ESA on-premise. Therefore, it is recommended that the Log Forwarder Function is installed into a private subnet with a Cloud VPC using a NAT Gateway to enable this communication through a corporate firewall.

Snowflake Connectivity

Snowflake communicates to the Snowflake Protector through the Google Cloud API Gateway. The API Gateway is configured to use an OAuth 2.0 implicit grant flow. Snowflake generates a JWT token used for remote authorization with the API Gateway. Snowflake requests are directed to the customer’s Google API Gateway which is configured to verify the JWT token. The following figure illustrates the integration architecture between Snowflake and the Protegrity software.

Snowflake’s API Integration Object

The Snowflake API Integration Object provides a connection between your Snowflake Google Cloud Account and your Google Cloud account where the Protegrity product is hosted. Creating this connection requires setting up the API Gateway and the appropriate access policies. These steps are provided in the installation.