Appendices

• 1: ADFS Federation using AWS Cognito User Pools
• 2: Installing the Policy Agent and Protector in Different AWS Accounts
• 3: Integrating Cloud Protect with PPC (Protegrity Provisioned Cluster)
• 4: Invoke Lambda Directly
• 5: Policy Agent - Custom VPC Endpoint Hostname Configuration
• 6: Protection Methods
• 7: Configuring Regular Expression to Extract Policy Username
• 8: Associating ESA Data Store With Cloud Protect Agent

1 - ADFS Federation using AWS Cognito User Pools

ADFS Federation using AWS Cognito User Pools

Common use case for Protegrity Policy Member source integrates with customer’s Active Directory. The AWS API Gateway can integrate with ADFS using AWS Cognito. The following article describes the procedure.

https://aws.amazon.com/blogs/mobile/building-adfs-federation-for-your-web-app-using-amazon-cognito-user-pools/

2 - Installing the Policy Agent and Protector in Different AWS Accounts

The Policy Agent Lambda function and Protect Lambda functions can be installed in separate AWS accounts. However, additional configuration is required to authorize the Policy Agent to provision the security policy to a remote Protect Lambda function.

Create Agent Lambda IAM policy

1. Login to the AWS account that hosts the Protect Lambda function.

2. From the AWS IAM console, select Policies > Create Policy.

3. Select the JSON tab and copy the following snippet.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "LambdaUpdateFunction",
         "Effect": "Allow",
         "Action": [
           "lambda:UpdateFunctionConfiguration"
         ],
         "Resource": [
           "arn:aws:lambda:*:*:function:*"
         ]
       },
       {
         "Sid": "LambdaReadLayerVersion",
         "Effect": "Allow",
         "Action": [
           "lambda:GetLayerVersion",
           "lambda:ListLayerVersions"
         ],
         "Resource": "*"
       },
       {
         "Sid": "LambdaDeleteLayerVersion",
         "Effect": "Allow",
         "Action": "lambda:DeleteLayerVersion",
         "Resource": "arn:aws:lambda:*:*:layer:*:*"
       },
       {
         "Sid": "LambdaPublishLayerVersion",
         "Effect": "Allow",
         "Action": "lambda:PublishLayerVersion",
         "Resource": "arn:aws:lambda:*:*:layer:*"
       },
       {
         "Sid": "S3GetObject",
         "Effect": "Allow",
         "Action": [
           "s3:GetObject"
         ],
         "Resource": "arn:aws:s3:::*/*"
       },
       {
         "Sid": "S3PutObject",
         "Effect": "Allow",
         "Action": [
           "s3:PutObject"
         ],
         "Resource": "arn:aws:s3:::*/*"
       },
       {
         "Sid": "LambdaGetConfiguration",
         "Effect": "Allow",
         "Action": [
             "lambda:GetFunctionConfiguration"
         ],
         "Resource": [
             "arn:aws:lambda:*:*:function:*"
         ]
       }
     ]
   }
   

4. Replace the wildcards (*) with the region, account, and resource name information where required.

5. Select Review policy, type in the policy name, and confirm. Record policy name:

   Agent Lambda Cross Account Policy Name: ___________________

Create Policy Agent cross-account IAM Role

1. Login to the AWS account that hosts the Protect Lambda function.

2. From the AWS IAM console, select Roles > Create Role

3. Select AWS Service > Lambda . Proceed to Permissions.

4. Select Policy created in the step above. Proceed to Tags.

5. Specify Tag, proceed to the final screen. Type in policy name and confirm. Record the name.

   Policy Agent Cross Account IAM Role Name: ___________________

Allow the Policy Agent Cross-Account Role to be Assumed by the Policy Agent IAM Role

1. Login to the AWS account that hosts the Protect Lambda function.

2. Navigate to the previously created IAM Role (Agent Lambda Cross-Account IAM Role Name).

3. Navigate to Trust Relationships > Edit Trust Relationships.

4. Modify the Policy Document, replacing the placeholder value indicated in the following snippet as <Agent Lambda IAM Execution Role ARN> with ARN of Agent Lambda IAM Role that was created in Agent Installation.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
   
      "Principal": {
   
               "AWS": "<Agent Lambda IAM Execution Role Name>"
   
         },
         "Action": "sts:AssumeRole"
       }
     ]
   }
   

5. Click Update Trust Policy.

Add Assume Role to the Policy Agent Execution IAM Role

1. Login to the AWS account that hosts the Policy Agent.

2. Navigate to the Agent Lambda IAM Execution Role that was created in Agent Installation.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
   
      "Principal": {
   
               "AWS": "<Agent Lambda IAM Execution Role Name>"
   
         },
         "Action": "sts:AssumeRole"
       }
     ]
   }
   

3. Add Inline Policy.

4. Modify the Policy Document, replacing the placeholder value indicated in the following snippet as <Agent Lambda Cross-Account IAM ARN> with the value recorded in Create Policy Agent cross-account IAM Role.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "sts:AssumeRole"
         ],
         "Resource": "<Agent Lambda Cross-Account IAM  ARN>."
       }
     ]
   }
   

5. When you are finished, choose Review Policy.

6. On the Review policy page, type a Name, then choose Create Policy.

Update the Policy Agent Lambda Configuration

1. From the AWS console, navigate to Lambda, and select the Policy Agent Lambda function.

2. Select Configuration tab | Environment variables.

3. Select Edit and add the following environment variables with the value from Agent Lambda Cross-Account IAM ARN:

   Parameter	Value
   AWS_ASSUME_ROLE	Agent Lambda Cross-Account IAM ARN

4. Ensure the values in the Parameters AWS_POLICY_S3_BUCKET, AWS_PROTECT_FN_NAME and AWS_POLICY_LAYER_NAME are all in the Protect Lambda Function AWS Account.

5. In case custom VPC hostname configuration is used, you will need to set the ENDPOINT_URL. Refer to Policy Agent - Custom VPC Endpoint Hostname Configuration.

   AWS_VPC_ENDPOINT_URL

   <AWS_VPC_ENDPOINT>

6. Click Save and Run the Lambda. The Lambda will now assume the Role in Protect Lambda Function AWS Account and update the policy cross accounts.

3 - Integrating Cloud Protect with PPC (Protegrity Provisioned Cluster)

This guide describes how to configure the Protegrity Policy Agent and Log Forwarder to connect to a Protegrity Provisioned Cluster (PPC), highlighting the differences from connecting to ESA.

Key Differences: PPC vs ESA

Prerequisites

• Access to PPC and required credentials.
• Tools: curl, kubectl installed.

Policy Agent Setup with PPC

Follow these instructions as a guide for understanding specific inputs for Policy Agent integrating with PPC:

1. Obtain the Datastore Key Fingerprint

   To retrieve the fingerprint for your Policy Agent:

   curl -k -H "Authorization: Bearer ${TOKEN}" -X POST https://${HOST}/pty/v2/pim/datastores/1/export/keys  -H "Content-Type: application/json" --data '{
     "algorithm": "RSA-OAEP-256",
     "description": "example-key-from-kms",
     "pem": "-----BEGIN PUBLIC KEY-----\nABC123... ...890XYZ\n-----END PUBLIC KEY-----"
   }'
   

   Sample Output:

   {"uid":"1","algorithm":"RSA-OAEP-256","fingerprint":"4c:46:d8:05:35:2e:eb:39:4d:39:8e:6f:28:c3:ab:d3:bc:9e:7a:cb:95:cb:b1:8e:b5:90:21:0f:d3:2c:0b:27","description":"example-key-from-kms"}
   

   Record the fingerprint value and configure it as the PTY_DATASTORE_KEY for the Policy Agent.

2. Retrieve the PPC CA Certificate

   To obtain the CA certificate from PPC:

   kubectl -n api-gateway get secret ingress-certificate-secret -o jsonpath='{.data.ca\.crt}' | base64 -d > CA.pem
   

   Use the CA.pem that was returned as described in Policy Agent Installation.

3. Configure the PPC Address

   Use the PPC address in place of the ESA IP address wherever required in your configuration.

Log Forwarder Setup with PPC

• The Log Forwarder will proceed without certificates and will print a warning if PtyEsaCaServerCert and PtyEsaClientCertificatesSecretId are not provided.
• No additional certificate or CA configuration is needed for PPC.

Troubleshooting

Protector Lambda fails with “AWS KMS Decrypt failed”

Symptom:

After a successful Policy Agent run and layer update, the Protector Lambda returns:

{
  "body": "{\"error_msg\":\"Failed to open decoder: rpdecode decrypt failure: dek callback failed: AWS KMS Decrypt failed: \",\"success\":false}",
  "isBase64Encoded": false,
  "statusCode": 400
}

The Protector Lambda logs show:

[SEVERE] [utils.cpp:185] AWS KMS Decrypt failed:

Cause:

The public key configured in the PPC/ESA datastore does not match the KMS key pair used by the Policy Agent. The policy package is encrypted with the public key stored in the datastore. If that key does not correspond to the KMS key pair whose private key is used for decryption, the Protector Lambda will fail to decrypt the policy.

Resolution:

1. Identify the KMS key pair used by the Policy Agent (the key ARN configured during pre-configuration).
2. Export the public key from that KMS key pair.
3. In PPC/ESA, ensure the datastore’s export key is configured with the public key from that same KMS key pair. See Obtain the Datastore Key Fingerprint above.
4. Re-run the Policy Agent to generate a new policy package encrypted with the correct key.
5. Test the Protector Lambda again.

Additional Notes

• For troubleshooting, consult the Protegrity Documentation.

4 - Invoke Lambda Directly

AWS Lambda can be invoked directly, such as from AWS SDK. This section contains information about request and response payloads with examples demonstrating direct invocation using AWS CLI and Python SDK (Boto3).

Request Payload

Lambda request payload for the direct invocation is defined as following

{
  "body": "<rest-api-request-payload>",
  "path": "/v1/<operation>",
  "headers": {}
}

• body - JSON string. Request schemas defined in Rest API Request.
• path - can be either ‘/v1/protect’ or ‘/v1/unprotect’.
• headers - can be used to pass authorization headers. See example below.

Example request:

{
  "body": "{\"query_id\": \"3\",\"user\": \"user1\",\"data_element\": \"deAlpha\",\"data\": [\"data1\", \"data2\"]}",
  "path": "/v1/protect",
  "headers": {}
}

Example Request with JWT authorization:

authorization="jwt"
jwt_verify=1
allow_assume_user=1
jwt_user_claim="username"
jwt_secret_base64="Must be set to public certificate"

{
  "body": "{\"query_id\": \"3\",\"user\": \"user1\",\"data_element\": \"deAlpha\",\"data\": [\"data1\", \"data2\"]}",
  "path": "/v1/protect",
  "headers": {
    "authorization": "bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2MTM4NjIzNzEsImlkIjoiYS1iLWMtZC1lLWYtMS0yLTMiLCJ1c2VybmFtZSI6IlBhdWwgQXRyZWlkZXMifQ.R1NcJ43540HKdhEBOK9WaMMpjBOYSJetckQKrcPQdz0z6sx1EDwHXYngBP9DtHgUM-6Vf1VNjtFh_Nqfeepp1BavmigIXoe3ZbrxRI3DFKi2UuLmgn--EYrSGlWsQjnmjaz5qUkID9iY2MtsRunKSuolSvG9UsD1G32kv0KZYX0"
  }
}

Response Payload

Lambda response payload has the following structure

{
  "body": "<rest-api-response-payload>"
  "isBase64Encoded": false,
  "statusCode": <http-status-code>,
}

• body - JSON string. Response schemas defined in Rest API Response.
• isBase64Encoded - always set to false.
• statusCode - HTTP status code integer.

Success Response Payload Example:

{
  "body": "{\"encoding\":\"utf8\",\"results\":[\"xcgd\", \"migs\"],\"success\":true}", 
  "isBase64Encoded": false, 
  "statusCode": 200
}

Error Response

Cloud API Lambda returns following error responses depending on the error type

Cloud API Protection Operation Error
      Returned when invalid data element is used or user has insufficient permissions to execute security operation.

{
  "body": "{\"error_msg\":\"Unprotect failed. Data element not found. Refer to audit log for details.\",\"success\":false}", 
  "isBase64Encoded": false, 
  "statusCode": 400
}

Cloud API Invalid Request Error
      Missing fields in the incoming request or malformed request JSON.

{
  "body": "Request format is not supported", 
  "isBase64Encoded": false, 
  "statusCode": 400
}

Cloud API Unexpected Lambda Exception Error
      Caused by Lambda runtime exception, for instance due to too short timeout or not enough memory.

{
  "errorMessage": "2023-01-18T16:42:19.593Z d0cf62d0-9eaf-427b-8ca5-1bdd8bd0b082 Task timed out after 10.25 seconds"
}

Examples

Prerequisites:

• AWS SDK or Command Line
• AWS Access Key ID and AWS Access Key

See Request Payload for request payload examples.

AWS CLI command to invoke Cloud API Lambda function:

aws lambda invoke --function-name Protegrity_Protect_RESTAPI_{stackname} --payload
    fileb://request_payload.json --log-type Tail output

Sample Python code demonstrating Cloud API Direct Lambda Calls

import json
import logging
import boto3
lambda_client = boto3.client("lambda")
logging.basicConfig(format="%(message)s")
logger = logging.getLogger('pty_cloud_api_sample')
logger.setLevel(logging.DEBUG)


class ProtectClient(object):
  """ 
    Sample client demonstrating how to invoke Protegrity Cloud API Lambda

      protect_fn: str - Name of the Cloud API Lambda (for example, Protegrity_Protect_RESTAPI_my_deployment)
  """

  def __init__(self, protect_fn):
    self.protect_fn = protect_fn

  def invoke_protect(self, values, data_element, operation, user, query_id,
              column_info=""):
    """ 
    Invokes Protegrity Cloud API Lambda to execute protect or unprotect operation

      values: list[str] - List of values to be protected/unprotected
      data_element: str - Name of the policy data element to use with protect/unprotect operation
      operation: str - Either 'protect' or 'unprotect'
      user: str - Policy user
      query_id: str - Query id will be present in the audit log
      column_info: - Used for troubleshooting, for instance, when protecting values/rows from multiple database columns
    """

    # Set authorization header here if JWT authorization is 
    # enabled in Cloud API Function configuration
    headers = {"Authorization": ""}
    request_body = {
      "user": user,
      "data_element": data_element,
      "data": values,
      "query_id": query_id
    }
    payload = json.dumps({"body": json.dumps(request_body), "path": f"/v1/{operation}",
                  "headers": headers})
    logger.debug(f"Request payload: {payload}")
    response = lambda_client.invoke(FunctionName=self.protect_fn, Payload=payload)
    lambda_response_payload = json.loads(response["Payload"].read().decode())
    logger.debug(f"Response payload: {lambda_response_payload}")
    response_status_code = lambda_response_payload.get("statusCode")
    response_body_string = lambda_response_payload.get("body")
    if response_status_code == None or response_body_string == None:
      raise Exception(f"Unexpected Cloud API Lambda error: [{lambda_response_payload}]")
    try:
      body_json = json.loads(response_body_string)
      if response_status_code == 200:
        return body_json.get("results", [])
      elif body_json.get("error_msg"):
        raise Exception(f"Cloud API Lambda error: [{response_status_code} - {body_json.get('error_msg')}]")
      raise Exception(f"Unexpected Cloud API Lambda error: [{lambda_response_payload}]")
    except json.decoder.JSONDecodeError:
      # Cloud API may return error in the response body
      # For example, {"statusCode": 400, "body":"Error message"}
      raise Exception(f"Cloud API Lambda error: [{response_status_code} - {response_body_string}]")

# Replace cloud-api-lambda-name with the name of the Cloud API Lambda
# For example, Protegrity_Protect_RESTAPI_my_deployment
protect_client = ProtectClient('cloud-api-lambda-name')
protected_data = ["UtfVk UHgcD!"]
logger.info(f"Protected data: {protected_data}")
unprotected_data = protect_client.invoke_protect(
  values=protected_data, 
  data_element='alpha', 
  operation='unprrotect', 
  user='test-user', 
  query_id='1234')
logger.info(f"Unprotected data: {unprotected_data}")

5 - Policy Agent - Custom VPC Endpoint Hostname Configuration

The Policy Agent uses default endpoint hostnames to communicate with other AWS services (for example, secretsmanager.amazonaws.com). This configuration will only work in VPCs where Amazon-provided DNS is available (default VPC configuration with private DNS option enabled for the endpoint). If your VPC uses custom DNS, follow the instructions below to configure the Policy Agent Lambda to use custom endpoint hostnames.

Identify DNS Hostnames

To identify DNS hostnames:

1. From AWS console, select VPC > Endpoints.

2. Select Secrets Manager endpoint from the list of endpoints.

3. Under Details > DNS Names, note the private endpoint DNS names adding https:// at the beginning of the endpoint name.

   For example, https://vpce-1234-4pzomrye.kms.us-west-1.vpce.amazonaws.com

4. Note down DNS names for the KMS and Lambda endpoints:

   AWS_SECRETSMANAGER_ENDPOINT: https://_________________

   AWS_KMS_ENDPOINT: https://_________________

   AWS_LAMBDA_ENDPOINT: https://_________________

Update the Policy Agent Lambda configuration

To update policy agent lambda configuration:

1. From the AWS console, navigate to Lambda, and select the Policy Agent Lambda function.

2. Select the Configuration section and choose Environment variables.

3. Select Edit and add the following environment variables with the corresponding endpoint URLs recorded in steps 3-4:

   Parameters	Value
   AWS_SECRETSMANAGER_ENDPOINT_URL	<AWS_SECRETS_ENDPOINT>
   AWS_KMS_ENDPOINT_URL	<AWS KMS ENDPOINT>
   AWS_LAMBDA_ENDPOINT_URL	<AWS LAMBDA ENDPOINT>

4. Click Save and Run the Lambda. The Lambda will now use endpoints you have just configured.

6 - Protection Methods

Protection Methods

For more information about the protection methods supported by Protegrity, refer to the Protection Methods Reference.

7 - Configuring Regular Expression to Extract Policy Username

Configuring Regular Expression to Extract Policy Username

Cloud Protect Lambda Function exposes USERNAME_REGEX configuration to allow extraction of policy username from user in the request.

• USERNAME_REGEX Lambda Environment configuration

  The USERNAME_REGEX configuration can be used to extract policy username from user in the request. The following are allowed values for USERNAME_REGEX:

  • 1 - Default build-in regular expression is used:

    ^arn:aws:(?:iam|sts)::[0-9]{12}:(?:role|user|group|assumed\-role|federated\-user)\/([\w\/+=,.\-]{1,1024}|[\w\/+=,.\-@]{1,1024})(?:@[a-zA-Z0-9\-]{1,320}(?:\.\w+)+)?$
    

  • ^User regex$ - Custom regex with one capturing group. This group is used to extract the username. Examples below show different regular expression values and the resulting policy user.

^arn:aws:(?:iam|sts)::[0-9]{12}:((?:role|user|group|assumed-role|federated-user).*)$

8 - Associating ESA Data Store With Cloud Protect Agent

Associating ESA Data Store With Cloud Protect Agent

ESA controls which policy is deployed to protector using concept of data store. A data store may contain a list of IP addresses identifying servers allowed to pull the policy associated with that specific data store. Data store may also be defined as default data store, which allows any server to pull the policy, provided it does not belong to any other data stores. Node registration occurs when the policy server (in this case the policy agent) makes a policy request to ESA, where the agent’s IP address is identified by ESA.

Policy agent lambda source IP address used for node registration on ESA depends on ESA hubcontroller configuration ASSIGN_DATASTORE_USING_NODE_IP and the PTY_ADDIPADDRESSHEADER configuration exposed by the agent lambda.

The Lambda service uses multiple network interfaces, internal network interface with ephemeral IP range of 169.254.x.x and external network interface with IP range of the VPC subnet the Lambda is associated with. By default, when agent lambda is contacting ESA to register node for policy download, ESA uses agent Lambda VPC IP address. This default behavior is caused by the default ESA hubcontroller configuration ASSIGN_DATASTORE_USING_NODE_IP=false and agent default configuration PTY_ADDIPADDRESSHEADER=yes.

In some cases, when there is a proxy server between the ESA and agent lambda, the desirable ESA configuration is ASSIGN_DATASTORE_USING_NODE_IP=true. and PTY_ADDIPADDRESSHEADER=no which will cause the ESA to use proxy server IP address.

The table below shows how the hubcontroller and agent settings will affect node IP registration on ESA.