This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Appendices

Additional references for the protector.

1 - Integrating Cloud Protect with PPC (Protegrity Provisioned Cluster)

Concepts for integrating with PPC (Protegrity Provisioned Cluster)

This guide describes how to configure the Protegrity Policy Agent and Log Forwarder to connect to a Protegrity Provisioned Cluster (PPC), highlighting the differences from connecting to ESA.

Key Differences: PPC vs ESA

FeatureESA 10.2PPC (this guide)
Datastore Key FingerprintOptional/RecommendedRequired
CA Certificate on AgentOptional/RecommendedOptional/Recommended
CA Certificate on Log ForwarderOptional/RecommendedNot supported
Client Certificate Authentication from Log ForwarderOptional/RecommendedNot supported
IP AddressESA IP addressPPC address

Prerequisites

  • Access to PPC and required credentials.
  • Tools: curl, kubectl installed.

Policy Agent Setup with PPC

Follow these instructions as a guide for understanding specific inputs for Policy Agent integrating with PPC:

  1. Obtain the Datastore Key Fingerprint

    To retrieve the fingerprint for your Policy Agent:

    curl -k -H "Authorization: Bearer ${TOKEN}" -X POST https://${HOST}/pty/v2/pim/datastores/1/export/keys  -H "Content-Type: application/json" --data '{
  "algorithm": "RSA-OAEP-256",
  "description": "example-key-from-kms",
  "pem": "-----BEGIN PUBLIC KEY-----\nABC123... ...890XYZ\n-----END PUBLIC KEY-----"
}'

    Sample Output:

    {"uid":"1","algorithm":"RSA-OAEP-256","fingerprint":"4c:46:d8:05:35:2e:eb:39:4d:39:8e:6f:28:c3:ab:d3:bc:9e:7a:cb:95:cb:b1:8e:b5:90:21:0f:d3:2c:0b:27","description":"example-key-from-kms"}

    Record the fingerprint and configure it as the value for PTY_DATASTORE_KEY in the Policy Agent function environment variable.

  2. Retrieve the PPC CA Certificate

    To obtain the CA certificate from PPC:

    kubectl -n api-gateway get secret ingress-certificate-secret -o jsonpath='{.data.ca\.crt}' | base64 -d > CA.pem

    Use the CA.pem that was returned as described in Policy Agent Installation.

  3. Configure the PPC Address

    Use the PPC fqdn in place of the ESA IP address wherever required in your configuration.

Log Forwarder Setup with PPC

  • The Log Forwarder will proceed without certificates and will print a warning if PTY_ESA_CA_SERVER_CERT is not provided.
  • No additional certificate or CA configuration is needed for PPC.

    2 - Configuring Regular Expression to Extract Policy Username

    Example configurations for user extraction with regular expressions

    Configuring Regular Expression to Extract Policy Username

    Cloud Protect Function exposes USERNAME_REGEX configuration to allow extraction of policy username from user in the request.

    • USERNAME_REGEX Function Environment configuration

      The USERNAME_REGEX environment variable can be set to contain regular expression with one capturing group. This group is used to extract the username. Examples below show different regular expression values and the resulting policy user.

    USERNAME_REGEX

    User in the request

    Effective Policy User

    Not Set

    juliet.snow@domain.com

    juliet.snow@domain.com

    juliet.snow/ad_postfix

    juliet.snow/ad_postfix

    ^(.*)[@/].*$

    juliet.snow@domain.com

    juliet.snow

    juliet.snow/ad_postfix

    juliet.snow

    3 - Sample Snowflake External Functions for Protegrity Integration

    Explore example implementations of secure Snowflake external functions for Protegrity integration

    Appendix A. Sample Snowflake External Function

    Method: Tokenization

    Type: ALPHA

     

    Snowflake Data Types

    Snowflake Max Size

    Protegrity Max Size

    VARCHAR

    16M (16,777,216 bytes)

    4K (4,096 bytes)

    CHAR

    STRING

    TEXT

     

    External Function Sample Definitions:

    CREATE SECURE EXTERNAL FUNCTION PTY_PROTECT_ALPHA ( val varchar ) 
  RETURNS varchar 
  NULL 
  IMMUTABLE 
  COMMENT = 'Protects using an ALPHA data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
  'X-Protegrity-HCoP-Rules'=
  '{"jsonpaths": [{"op_type":"PROTECT","data_element":"TOK_ALPHA"}]}'
  )
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    CREATE SECURE EXTERNAL FUNCTION PTY_UNPROTECT_ALPHA ( val varchar ) 
  RETURNS varchar 
  NULL 
  IMMUTABLE 
  COMMENT = 'Unprotects using an ALPHA data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"UNPROTECT","data_element":"TOK_ALPHA"}]}'
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

     

    Sample EF Calls:

    SELECT PTY_PROTECT_ALPHA ('Hello World')

    SELECT PTY_UNPROTECT_ALPHA('rfDtw sLMJK');

     

    Snowflake Masking Policy example:

    create or replace masking policy alpha_policy as (val string) returns string -> 
 case 
 when current_role() in ('ACCOUNTADMIN') then PTY_UNPROTECT_ALPHA(val) 
 else val 
end;

    alter table pii_data modify column field01 set masking policy alpha_policy; 
alter table pii_data modify column field01 unset masking policy;

    Method: Tokenization

    Type: NUMERIC

     

    Snowflake Data Types

    Snowflake Max Size

    Protegrity Max Size

    NUMBER

     

     

    DECIMAL

    INTEGER

    DOUBLE

     

    External Function Sample Definitions:

    CREATE SECURE EXTERNAL FUNCTION PTY_PROTECT_NUMERIC ( val number ) 
  RETURNS number 
  NULL 
  IMMUTABLE 
  COMMENT = 'Protects using a NUMERIC data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"PROTECT","data_element":"TOK_NUMERIC"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    CREATE SECURE EXTERNAL FUNCTION PTY_UNPROTECT_NUMERIC ( val number) 
  RETURNS number 
  NULL 
  IMMUTABLE 
  COMMENT = 'Unprotects using a NUMERIC data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"UNPROTECT","data_element":"TOK_NUMERIC"}]}'
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

     

    Sample EF Calls:

    SELECT PTY_PROTECT_NUMERIC ('123456789');

    SELECT PTY_UNPROTECT_NUMERIC ('752513497');

     

    Snowflake Masking Policy example:

    create or replace masking policy num_policy as (val number) returns number -> 
 case 
 when current_role() in ('ACCOUNTADMIN') then PTY_UNPROTECT_NUMERIC(val) 
 else val 
end;

    alter table pii_data modify column field02 set masking policy num_policy; 
alter table pii_data modify column field02 unset masking policy;

    Method: Tokenization

    Type: DATE YYYY-MM-DD

     

    Snowflake Data Types

    Snowflake Max Size

    Protegrity Max Size

    DATE (any supported format)

    10 bytes

    10 bytes

     

    External Function Sample Definitions:

    CREATE SECURE EXTERNAL FUNCTION PTY_PROTECT_DATEYYYYMMDD ( val date ) 
  RETURNS date 
  NULL 
  IMMUTABLE 
  COMMENT = 'Protects using a Date data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"PROTECT","data_element":"TOK_DATEYYYYMMDD"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';  

    CREATE SECURE EXTERNAL FUNCTION PTY_UNPROTECT_DATEYYYYMMDD ( val date ) 
  RETURNS date 
  NULL 
  IMMUTABLE 
  COMMENT = 'Unprotects using a Date data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"UNPROTECT","data_element":"TOK_DATEYYYYMMDD"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    Sample EF Calls:

    SELECT PTY_PROTECT_DATEYYYYMMDD ('2020-12-31');

    SELECT PTY_UNPROTECT_DATEYYYYMMDD('0653-06-01');

    SELECT PTY_PROTECT_DATEYYYYMMDD ('31-DEC-2020');*

    SELECT PTY_UNPROTECT_DATEYYYYMMDD('01-JUN-0653');*

    SELECT PTY_PROTECT_DATEYYYYMMDD('12/31/2020');*

    SELECT PTY_UNPROTECT_DATEYYYYMMDD('06/01/0653');*

    SELECT PTY_PROTECT_DATEYYYYMMDD (current_date);

     

    Snowflake Masking Policy example:

    create or replace masking policy date_policy as (val date) returns date -> 
 case 
 when current_role() in ('ACCOUNTADMIN') then PTY_UNPROTECT_DATEYYYYMMDD (val) 
 else val 
end;

    alter table pii_data modify column field11 set masking policy date_policy; 
alter table pii_data modify column field11 unset masking policy;
    **\***: Automatic cast to YYYY-MM-DD, no need to make any conversions. The output is always in the YYYY-MM-DD format

    Cutover Dates of the Proleptic Gregorian Calendar: no issues (no conversions performed by Snowflake)

    Method: Tokenization

    Type: DATETIME

     

    Snowflake Data Types

    Snowflake Max Size

    Protegrity Max Size

    DATE

    10 bytes

    29 bytes

    DATETIME

    29 bytes

    TIMESTAMPNTZ*

    TIMESTAMP_NTZ*

    TIMESTAMP WITHOUT TIME ZONE*

     

    External Function Sample Definitions:

    CREATE SECURE EXTERNAL FUNCTION PTY_PROTECT_DATETIME ( val timestamp ) 
  RETURNS timestamp 
  NULL 
  IMMUTABLE 
  COMMENT = 'Protects using a TIMESTAMP data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"PROTECT","data_element":"TOK_DATETIME"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    CREATE SECURE EXTERNAL FUNCTION PTY_UNPROTECT_DATETIME ( val timestamp ) 
  RETURNS timestamp 
  NOT NULL 
  IMMUTABLE 
  COMMENT = 'Unprotects using a TIMESTAMP data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"UNPROTECT","data_element":"TOK_DATETIME"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    Sample EF Calls:

    SELECT PTY_PROTECT_DATETIME('2010-10-25');

    SELECT PTY_UNPROTECT_DATETIME('0845-04-04');

    SELECT PTY_PROTECT_DATETIME('2010-10-25 10:45:33');

    SELECT PTY_UNPROTECT_DATETIME('0845-04-04 10:45:33');

    SELECT PTY_PROTECT_DATETIME('2010-10-25 10:45:33.123');

    SELECT PTY_UNPROTECT_DATETIME('0845-04-04 10:45:33.123');

    SELECT PTY_PROTECT_DATETIME(current_date);

    SELECT PTY_PROTECT_DATETIME(cast(current_timestamp as TIMESTAMPNTZ));

     

    Snowflake Masking Policy example:

    create or replace masking policy datetime_policy as (val timestampntz) returns timestampntz -> 
 case 
 when current_role() in ('ACCOUNTADMIN') then PTY_UNPROTECT_DATETIME (val) 
 else val 
end;

    alter table pii_data modify column field12 set masking policy datetime_policy; 
alter table pii_data modify column field12 unset masking policy;
    **\***: Default TIMESTAMP in Snowflake includes Time Zone – not supported by Protegrity’s DATETIME data element

    Method: Tokenization

    Type: DECIMAL

     

    Snowflake Data Types

    Snowflake Max Size

    Protegrity Max Size

    NUMBER(N,M)

    38 digits

    36 digits

    NUMERIC(N,M)*

    DECIMAL(N,M)*

     

    External Function Sample Definitions:

    CREATE SECURE EXTERNAL FUNCTION PTY_PROTECT_DECIMAL ( val NUMBER(38,6) ) 
  RETURNS NUMBER(38,6) 
  NULL 
  IMMUTABLE 
  COMMENT = 'Protects using a DECIMAL data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"PROTECT","data_element":"TOK_DECIMAL"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    CREATE SECURE EXTERNAL FUNCTION PTY_UNPROTECT_DECIMAL ( val NUMBER(38,6) ) 
  RETURNS NUMBER(38,6) 
  NULL 
  IMMUTABLE 
  COMMENT = 'Unprotects using a DECIMAL data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"UNPROTECT","data_element":"TOK_DECIMAL"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    Sample EF Calls:

    SELECT PTY_PROTECT_DECIMAL (12345678.99);

    SELECT PTY_UNPROTECT_DECIMAL (21872469.760000);

     

    Snowflake Masking Policy example:

    create or replace masking policy decimal_policy as (val NUMBER(38,6)) returns NUMBER(38,6)-> 
 case 
 when current_role() in ('ACCOUNTADMIN') then PTY_UNPROTECT_DECIMAL (val) 
 else val 
end;

    alter table pii_data modify column field13 set masking policy decimal_policy; 
alter table pii_data modify column field13 unset masking policy;
    **\***: Synonymous with NUMBER

    Method: Tokenization

    Type: INTEGER

     

    Snowflake Data Types

    Snowflake Max Size

    Protegrity Max Size

    NUMBER

    38 digits

    2 bytes

    4 bytes

    8 bytes

    NUMERIC*

    INT*

    INTEGER*

    BIGINT*

    SMALLINT*

    TINYINT*

    BYTEINT*

     

    External Function Sample Definitions:

    CREATE SECURE EXTERNAL FUNCTION PTY_PROTECT_INTEGER ( val NUMBER ) 
  RETURNS NUMBER
  NULL 
  IMMUTABLE 
  COMMENT = 'Protects using an INTEGER data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"PROTECT","data_element":"TOK_INTEGER"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    CREATE SECURE EXTERNAL FUNCTION PTY_UNPROTECT_INTEGER ( val NUMBER ) 
  RETURNS NUMBER 
  NOT NULL 
  IMMUTABLE 
  COMMENT = 'Unprotects using an INTEGER data element'  
  API_INTEGRATION = REPLACE_WITH_YOUR_API_INTEGRATION_ID 
  HEADERS =(  
    'X-Protegrity-HCoP-Rules'=
    '{"jsonpaths":[{"op_type":"UNPROTECT","data_element":"TOK_INTEGER"}]}'    
  ) 
  CONTEXT_HEADERS = ( current_user, current_timestamp, current_account ) 
  AS '<API Gateway URL>/api/Protect';

    Sample EF Calls:

    SELECT PTY_PROTECT_INTEGER (123456789);

    SELECT PTY_UNPROTECT_INTEGER (1104108887);

     

    Snowflake Masking Policy example:

    create or replace masking policy int_policy as (val NUMBER ) returns NUMBER -> 
 case 
 when current_role() in ('ACCOUNTADMIN') then PTY_UNPROTECT_INTEGER (val) 
 else val 
end;

    alter table pii_data modify column field14 set masking policy int_policy; 
alter table pii_data modify column field14 unset masking policy;
    **\***: Synonymous with NUMBER, except that precision and scale cannot be specified \(i.e. always defaults to NUMBER\(38, 0\)\)

    **Recommended approach for protecting whole numbers fields in Snowflake

    When values are…then use the following Data Element:
    Between -32768 and 32767INTEGER (2 bytes)
    Between -2147483648 and 2147483647INTEGER (4 bytes)
    Between -9223372036854775808 and 9223372036854775807INTEGER (8 bytes)
    < -9223372036854775808 or > 9223372036854775807DECIMAL

    When in doubt, use DECIMAL for any numeric range.

    4 - ARM Template Installation - Required Permissions

    Outlines the required permissions to deploy Cloud Protect with provided ARM templates

    ARM Template Installation - Required Permissions

    Permissions below are required to install Protegrity service using ARM template.

    All permissions in the table must be granted with the Resource group scope.

    Permissions

    Description

    Built-In Azure Role

    Microsoft.Insights/components/read

    Microsoft.OperationalInsights/workspaces/read

    Read access to monitoring data and settings

    Monitoring Reader

    Microsoft.Insights/components/write

    Microsoft.OperationalInsights/workspaces/write

    Write and manage access to monitoring data and settings

    Monitoring Contributor

    Microsoft.Web/serverFarms/write

    Microsoft.Web/sites/write

    Microsoft.Web/sites/host/listkeys/action

    Microsoft.Web/serverFarms/join/action

    Microsoft.Web/register/action

    Write and manage access to web apps

    Website Contributor

    Microsoft.ManagedIdentity/userAssignedIdentities/assign/action

    Microsoft.ManagedIdentity/userAssignedIdentities/read

    Manage and assign managed identities

    Managed Identity Operator

    Microsoft.Resources/deployments/validate/action

    Microsoft.Resources/deployments/write

    Microsoft.Resources/deployments/operationStatuses/read

    Microsoft.Resources/deployments/read

    Manage and validate deployments

    Deployment Contributor

             

    Log Forwarder service ARM deployment requires additional permissions below:

    Permissions

    Description

    Built-In Azure Role

    Microsoft.EventHub/namespaces/write

    Microsoft.EventHub/namespaces/eventhubs/write

    Microsoft.EventHub/namespaces/networkrulesets/write

    Allow for the creation, update, and deletion of Event Hub namespaces, event hubs within those namespaces, and their network rule sets, enabling full management of Event Hub resources. Note: These permissions are only required when deploying new event Hub.

    Event Hubs Contributor

    Microsoft.EventHub/namespaces/read

    Read monitoring data and metrics, including Event Hub namespace data.

    Monitoring Reader

      

          

    The additional permissions listed below are required when API management is part of the deployment.

    Permissions

    Description

    Built-In Azure Role

    Microsoft.ApiManagement/service/write

    Microsoft.ApiManagement/service/apis/write

    Microsoft.ApiManagement/service/diagnostics/write

    Microsoft.ApiManagement/service/apis/operations/write

    Microsoft.ApiManagement/service/apis/operations/policies/write

    Microsoft.ApiManagement/service/backends/write

    Microsoft.ApiManagement/service/loggers/write

    Microsoft.ApiManagement/service/policies/write

    Microsoft.ApiManagement/service/apis/diagnostics/write

    Create or update API Management service instances, APIs, diagnostics, API operations, operation policies, backends, loggers, tenant policies, and API diagnostics.

    API Management Service Contributor

    Microsoft.ApiManagement/service/read

    Microsoft.ApiManagement/service/operationResults/read

    Read metadata for API Management service instances and get the status of long-running operations.

    API Management Service Reader

    5 - Associating ESA Data Store With Cloud Protect Agent

    Configure ESA data store for Policy Agent

    Associating ESA Data Store With Cloud Protect Agent

    ESA controls which policy is deployed to protector using concept of data store. A data store may contain a list of IP addresses identifying servers allowed to pull the policy associated with that specific data store. Data store may also be defined as default data store, which allows any server to pull the policy, provided it does not belong to any other data stores. Node registration occurs when the policy server (in this case the policy agent) makes a policy request to ESA, where the agent’s IP address is identified by ESA.

    Policy agent function source IP address used for node registration on ESA depends on ESA hubcontroller configuration ASSIGN_DATASTORE_USING_NODE_IP and the PTY_ADDIPADDRESSHEADER configuration exposed by the agent function.

    The function service uses multiple network interfaces, internal network interface with ephemeral IP range of 169.254.x.x and external network interface with IP range described in Function app outbound IP addresses section under function configuration. By default, when agent function is contacting ESA to register node for policy download, ESA uses agent function outbound IP address. This default behavior is caused by the default ESA hubcontroller configuration ASSIGN_DATASTORE_USING_NODE_IP=false and agent default configuration PTY_ADDIPADDRESSHEADER=yes.

    In some cases, when there is a proxy server between the ESA and agent function, the desirable ESA configuration is ASSIGN_DATASTORE_USING_NODE_IP=true. and PTY_ADDIPADDRESSHEADER=no which will cause the ESA to use proxy server IP address.

    The table below shows how the hubcontroller and agent settings will affect node IP registration on ESA.

    Agent source IPAgent Function Outbound IPProxy IPESA config - ASSIGN_DATASTORE_USING_NODE_IPAgent function config - PTY_ADDIPADDRESSHEADERAgent node registration IP
    169.254.144.8120.75.43.207No Proxytrueyes169.254.144.81
    trueno20.75.43.207
    falseyes20.75.43.207
    falseno20.75.43.207
    169.254.144.8120.75.43.20734.230.42.110trueyes169.254.144.81
    trueno34.230.42.110
    falseyes34.230.42.110
    falseno34.230.42.110