REST Container

• 1: Understanding the Architecture
• 1.1: Architecture and Components using Dynamic-based Deployment
• 1.2: Architecture and Components using Static Deployment
• 2: System Requirements
• 2.1: Software Requirements
• 2.2: Hardware Requirements
• 3: Preparing the Environment
• 3.1: Initializing the Jump Box
• 3.2: Extracting the Installation Package
• 3.3: Creating Certificates
• 3.4: Uploading the Images to the Container Repository
• 3.5: Creating the AWS Environment
• 3.5.1: Creating the AWS Setup for Static Mode
• 3.5.1.1: Creating an Data Encryption Key (DEK)
• 3.5.1.2: Creating an AWS S3 Bucket
• 3.5.1.3: Creating an AWS EFS
• 3.5.2: Creating a Kubernetes Cluster
• 4: Installing the Protector
• 4.1: Deploying REST Container for Dynamic Method
• 4.1.1: Deploying Log Forwarder
• 4.1.2: Deploying Resilient Package Proxy (RPP)
• 4.1.3: Deploying the REST Container with Dynamic Method
• 4.1.4: Uninstalling the Protector in Dynamic Method
• 4.2: Deploying REST Product in Static Mode
• 4.2.1: Retrieving the Policy Package from the ESA
• 4.2.2: Deploying Log Forwarder
• 4.2.3: Deploying KMSProxy Container
• 4.2.4: Deploying REST Container Using Static Method
• 4.2.5: Updating the Policy Package
• 4.2.6: Uninstalling the Protector in Static Method
• 5: Application Protector API on REST
• 5.1: Version 4 (V4) Application Protector API on REST
• 5.1.1: List of REST APIs
• 5.1.1.1: HTTP GET version
• 5.1.1.2: HTTP POST protect
• 5.1.1.3: HTTP POST unprotect
• 5.1.1.4: HTTP POST reprotect
• 5.1.1.5: HTTP GET doc
• 5.1.1.6: HTTP Headers
• 5.1.2: V4 AP REST HTTP Response Codes
• 5.2: Version 1 (V1) Application Protector API on REST
• 5.2.1: List of REST APIs
• 5.2.1.1: HTTP GET version
• 5.2.1.2: HTTP POST protect
• 5.2.1.3: HTTP POST unprotect
• 5.2.1.4: HTTP POST reprotect
• 5.2.1.5: HTTP Headers
• 5.2.2: Error Handling for v1 API
• 5.2.3: V1 AP REST HTTP Response Codes
• 6: Using Samples
• 7: Running the Autoscaling Script
• 8: Upgrading the Protector from Version 9.x to 10.x
• 9: Upgrading the Protector from Version 10.x to 10.y
• 10: Using Dockerfiles to Build Custom Images
• 11: Appendix - Deploying the Helm Charts by Using the Set Argument

1 - Understanding the Architecture

The Protegrity REST Container can be deployed using one of the following deployment methods:

• Using dynamic-based deployment
• Using static-based deployment

1.1 - Architecture and Components using Dynamic-based Deployment

Key features of an dynamic-based deployment include:

• The deployments can be used in use cases where policy updates need to be available on the cluster continuously.
• The RPP component is synchronized with the ESA for policy updates at a predefined rate.
• The dynamic deployment requires the ESA to be always connected to support the policy updates.

For more information about package deployment approaches, refer to Resilient Package Deployment.

The following figure represents the architecture for deploying the REST Container with RPP on a Kubernetes cluster.

Deployment Steps:

1. Create the ESA with the policy and datastore.

2. Deploy the Resilient Package Proxy (RPP) instances with mTLS certificates to communicate with the ESA and to host the proxy endpoint for protectors.

3. Deploy the REST protector with mTLS certificates to communicate with the RPP. The communication between the RPP and the protector is secured using mTLS.

4. After the protector instance starts as part of the application POD, the protector sends a request to the RPP instance to retrieve the policy package.

5. At periodic intervals, the protector tries to pull the new policy package from RPP instance. If the package present on the RPP instance has expired due to cache invalidation policy, the RPP pulls the new package from an upstream RPP or the ESA.

1.2 - Architecture and Components using Static Deployment

Key features of a Static-based deployment include:

• The deployments can be used in use cases where a fixed policy package is required.
• The policy updates need to be triggered through automation using ConfigMap updates.

For more information about package deployment approaches, refer to Resilient Package Deployment.

The following figure represents the architecture for deploying the REST Container with static deployment on a Kubernetes cluster.

Deployment Steps:

1. The ESA administrator user pulls the policy package from the ESA and stores it to an Object Store or a Volume Mount.

2. The Policy Loader sidecar container reads the internal configmap for policy updates.

3. The sidecar container retrieves the policy package from the Object Store or Volume Mount.

4. The sidecar container then stores the policy package in the tmpfs directory.

5. The REST protector reads the policy package from the tmpfs directory.

6. Based on the values specified in the internal config.ini file, the protector initates the RP Callback REST.

7. The RP Callback decrypts the Data Encryption Key (DEK) using the KMS Proxy container.

8. The KMS Proxy container reads the decrypted DEK from the cache, if present.

9. If the DEK is not present in the cache, the KMS Proxy container uses the KMS Backend to retrieve the DEK from the Cloud KMS and store the decrypted DEK in the cache.

10. The Protector decrypts the policy package using the DEK and initializes its internal library.

2 - System Requirements

This section provides an overview of the software and hardware requirements required for deploying the REST Container.

2.1 - Software Requirements

Ensure that the following prerequisites are met for deploying the REST Protector package REST_RHUBI-9-64_x86-64_K8S_<Version>.tgz.

ESA prerequisites

• Policy – Ensure that you have defined the security policy in the ESA. For more information about defining a security policy, refer to the section Policy Management.

• Datastore - Attach the policy to the default datastore in the ESA or to a range of allowed servers that are added to a datastore.

  The IP address range of the allowed servers must be the same as that of the nodes in the Kubernetes cluster where the AP-REST containers have been deployed.

For more information about datastores, refer to the section Data Stores.

• ESA user - Create an ESA user that will be used to invoke the RPS REST API for retrieving the security policy and the certificates from the ESA. Ensure that the user is assigned the Export Resilient Package role. This user is used to export the policy in a static-based deployment.

  For more information about assigning roles, refer to the section Managing Roles.

Jump Box Configuration

The Linux instance or the Jump Box can be used to communicate with the Kubernetes cluster. This instance can be on-premise or on AWS. The Jump Box instance is used to execute all the deployment related commands.

Ensure that the following prerequisites are installed on the Jump Box:

• Helm, which is used as the package manager for all the applications.
• Docker to communicate with the Container Registry, where you want to upload the Docker images.
• eksctl, which is a CLI utility to communicate with Amazon EKS.

Cloud or AWS prerequisites

You need access to an AWS account. You also need access to the following AWS resources.

• AWS Elastic File System (EFS), if you want to upload the policy package to AWS EFS instead of AWS S3. You require both read and write permissions. This is required for static-based deployment.
  • Install the latest version of the EFS-CSI driver, which is required if you are using AWS EFS as the persistent volume. This is required for statci-based deployment.

For more information about installing the EFS-CSI driver, refer to the Amazon EFS CSI driver documentation.

• AWS S3, if you want to use AWS S3 for storing the policy snapshot, instead of AWS EFS. You require both read and write permissions. This is required for static-based deployment.

  For more information about the AWS S3-specific permissions, refer to the API Reference document for AWS S3.

• IAM User - Required to create the Kubernetes cluster. This user requires the following permissions:

  • AmazonEC2FullAccess - This is a managed policy by AWS

  • AmazonEKSClusterPolicy - This is a managed policy by AWS

  • AmazonEKSServicePolicy - This is a managed policy by AWS

  • AWSCloudFormationFullAccess - This is a managed policy by AWS

  • Custom policy that allows the user to perform the following actions:

    • Create a new role and an instance profile.
    • Retrieve information about a role and an instance profile.
    • Attach a policy to the specified IAM role.
    
    

    The following actions must be permitted on the IAM service:

    • GetInstanceProfile
    • GetRole
    • AddRoleToInstanceProfile
    • CreateInstanceProfile
    • CreateRole
    • PassRole
    • AttachRolePolicy

  • Custom policy that allows the user to perform the following actions:

    • Delete a role and an instance profile.
    • Detach a policy from a specified role.
    • Delete a policy from the specified role.
    • Remove an IAM role from the specified EC2 instance profile.
    
    

    The following actions must be permitted on the IAM service:

    • GetOpenIDConnectProvider
    • CreateOpenIDConnectProvider
    • DeleteInstanceProfile
    • DeleteRole
    • RemoveRoleFromInstanceProfile
    • DeleteRolePolicy
    • DetachRolePolicy
    • PutRolePolicy

  • Custom policy that allows the user to manage EKS clusters. The following actions must be permitted on the EKS service:

    • ListClusters
    • ListNodegroups
    • ListTagsForResource
    • ListUpdates
    • DescribeCluster
    • DescribeNodegroup
    • DescribeUpdate
    • CreateCluster
    • CreateNodegroup
    • DeleteCluster
    • DeleteNodegroup
    • UpdateClusterConfig
    • UpdateClusterVersion
    • UpdateNodegroupConfig
    • UpdateNodegroupVersion

  For more information about creating an IAM user, refer to the section Creating an IAM User in Your AWS Account in the AWS documentation. Contact your system administrator for creating the IAM users.

  For more information about the EKS-specific permissions, refer to the API Reference document for Amazon EKS.

• Access to AWS Elastic Container Registry (ECR) to upload the Container images.

• Access to Route53 for mapping the hostname of the Elastic Load Balancer to a DNS entry in the Amazon Route53 service. This is required if you are terminating the TLS connection from the client application on the Load Balancer.

• Access to AWS KMS. This is required for static-based deployment.

2.2 - Hardware Requirements

The following table lists the minimum hardware configuration for each pod where the REST Container is deployed.

The instance type used for the cluster node is t3.2xlarge. The minimum CPU requirement for the node is 8 vCPU and the minimum memory capacity is 32 GiB.

Note: The package size of a policy with 70 thousand users and 26 data elements is 257447563 bytes.

3 - Preparing the Environment

This section provides an overview of the steps required to prepare the environment for deploying the REST Container product.

3.1 - Initializing the Jump Box

The Linux instance should be connected to the Kubernetes cluster. The following is the minimum system requirements to be configured for a Linux instance.

3.2 - Extracting the Installation Package

This section describes the steps to download and extract the installation package for the REST protector.

To download the installation package:

1. Download the REST_RHUBI-9-64_x86-64_K8S_<Version>.tgz file on the Linux instance.

2. Run the following command to extract the files from the REST_RHUBI-9-64_x86-64_K8S_<Version>.tgz file.

   tar -xvf REST_RHUBI-9-64_x86-64_K8S_<Version>.tgz

   The signatures directory and the REST_RHUBI-9-64_x86-64_K8S_<Version>.tgz fileare extracted.

3. Run the following command to extract the files from the REST_RHUBI-9-64_x86-64_K8S_<Version>.tgz file.

   tar -xvf REST_RHUBI-9-64_x86-64_K8S_<Version>.tgz

   The following directories and files are extracted:

   • devops - Helm charts, Dockerfiles, and container images to deploy the REST Container using the Static policy.
   • protector - Dockerfiles and container images to create the REST Container.
   • dynamic - Helm charts, Dockerfiles, and container images to deploy the REST Container using the Dynamic method.
   • common - Helm charts, Dockerfiles, and container images to deploy the Log Forwarder.
   • certs - Create certificates required for secure communication.
   • HOW-TO-BUILD-DOCKER-IMAGES - Text file specifying how to build the Docker images.
   • manifest.json - Metadata file specifying the product version and component names.

The following shows a list of the Helm charts and container images.

3.3 - Creating Certificates

This section describes the steps to create certificates required for secure communication. These certificates are for secure communication between:

• ESA and the RPP.
• RPP and the protector.
• KMSProxy and the protector.
• REST protector and the curl client.

To download the installation package:

1. Navigate to the directory where you have extracted the installation package.

2. Navigate to the certs directory. The following files are available:

   • CertificatesSetup_Linux_x64_<Version>tgz - Download the certificates from the ESA. You can use them as the common certificates in the dynamic deployment between the RPProxy and the ESA, and between the RPProxy and the protector. You can also use these certificates separately as the upstream certificate between the ESA and RPProxy in the dynamic deployment.
   • CreateCertificate_Linux_x64_<Version>.tgz - Generate self-signed client and server certificates. In the Dynamic method, these certificates are used for communication between RPProxy and the protector, and the REST protector and the curl client. In the Static policy method, these certificates are used for communication between KMSProxy and the protector, and the REST protector and the curl client. Customers can choose to use their own certificates.

3. Extract both the packages using the following command.

   tar -xvf CertificatesSetup_Linux_x64_<Version>.tgz
   tar -xvf CreateCertificate_Linux_x64_<Version>.tgz
   

   The following files are extracted:

   • CertificatesSetup_Linux_x64_<Version>.sh
   • CreateCertificate_Linux_x64_<Version>.sh

Certificates for communication between the ESA and the RPP

1. Run the following command to create ESA certificates for establishing a secure communication between the ESA and the RPP.

./CertificatesSetup_Linux_x64_<Version>.sh (-u <username> -p <password>) [-h <hostname>] [--port <port>] [-d <directory>]

Options:
  -u      User with the Export Certificates role
  -p      Password for user with the Export Certificates role
  -h      Host or IP address of the ESA
  --port  Port number of the ESA
  -d      local directory where certificates are stored

For more information about the command, use the –help parameter as shown in the following command.

./CertificatesSetup_Linux_x64_<Version>.sh --help

The output displays all the options that can be used with the command. It also provides usage examples.

Certificates for client and server communication between RPP and Protector, and KMS-Proxy and Protector

5. Run the following command to create server-side certificates.

./CreateCertificate_Linux_x64_<Version>.sh (client | server ) --name <common name> [--dir <directory> ] [--dns <dnsname>] [--ip <ip address>] 

Options:
  client        Generate client certificate
  server        Generate server certificate
  --name        Certificate common name.
  --dns         Specify domain names. To specify multiple DNS names, repeat the --dns flag.
  --ip          Specify IP addresses. To specify multiple IP address, repeat the --ip flag.
  --noenc       The certificate key file is not encrypted. No secret.txt file created.
  --dir         Output base directory for certificates.
  --print       Prints OpenSSL configuration files used to generate certificates.
  --help        Print help message.

This command is used to create the certificates for both the Dynamic and Static-based deployments.

For more information about the command, use the –help parameter as shown in the following command.

./CreateCertificate_Linux_x64_<Version>.sh --help

The output displays all the options that can be used with the command. It also provides usage examples.

3.4 - Uploading the Images to the Container Repository

Before you begin, ensure that you have set up your Container Registry.

To upload the images to the Container Repository:

1. Install Docker on the Linux instance.

   For more information about installing Docker on a Linux machine, refer to the Docker documentation.

2. Run the following command to authenticate your Docker client to Amazon ECR.

   aws ecr get-login-password --region <Name of ECR region where you want to upload the container image> | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.<Name of ECR region where you want to upload the container image>.amazonaws.com

   For more information about authenticating your Docker client to Amazon ECR, refer to the AWS CLI Command Reference documentation.

3. Extract the installation package.

   The AP-REST, RPProxy, Policy Loader, and KMSProxy container images are extracted.

   For more information about extracting the installation package, refer to the section Extracting the Installation Package.

4. Perform the following steps to upload the AP-REST container image to Amazon ECR.

   a. Run the following command to load the AP-REST container image into Docker.

   docker load -i REST_RHUBI-9-64_x86-64_K8S_<Version>.tar.gz

   b. Run the following command to list the AP-REST container image.

   docker images

   c. Tag the image to the Amazon ECR by running the following command.

   docker tag <Container image>:<Tag> <Container registry path>/<Container image>:<Tag>

   For example:

   docker tag ap-rest:AWS <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/ap-rest:AWS

   For more information regarding tagging an image, refer to the section Pushing an image in the AWS documentation.

   d. Push the tagged image to the Amazon ECR by running the following command.

   docker push <Container registry path>/<Container image>:<Tag>

   For example:

   docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/ap-rest:AWS

   For more information about creating custom images, refer to the section Using Dockerfiles to Build Custom Images.

5. Navigate to the directory where you have extracted the Helm charts packages for the AP-REST containers.

6. In the values.yaml file, update the appropriate path for the iaprestImage setting, along with the tag.

7. Repeat steps 1 to 6 for uploading the respective images for RPProxy, Policy Loader, and KMSProxy.

3.5 - Creating the AWS Environment

This section describes how to create the AWS runtime environment.

Prerequisites

Before creating the runtime environment on AWS, ensure that you have a valid AWS account and the following information:

• Login URL for the AWS account
• Authentication credentials for the AWS account

Audience

It is recommended that you have working knowledge of AWS and knowledge of the following concepts:

• Introduction to AWS S3
• Introduction to AWS Cloud Security
• Introduction to AWS EKS

3.5.1 - Creating the AWS Setup for Static Mode

This section describes how to create the following AWS resources for static mode:

• Data Encryption Key
• AWS S3 bucket
• AWS EFS

3.5.1.1 - Creating an Data Encryption Key (DEK)

To create a Data Encryption Key:

1. Log in to the AWS environment.

2. Navigate to Services.

   A list of AWS services appears.

3. In Security, Identity, & Compliance, click Key Management Service.

   The AWS Key Management Service (KMS) console opens. By default, the Customer managed keys screen appears.

4. Click Create key.

   The Configure key screen appears.

5. In the Key type section, select the Asymmetric option to create a single customer master key that will be used to perform the encrypt and decrypt operations.

6. In the Key usage section, select the Encrypt and decrypt option.

7. In the Key spec section, select one option.

   For example, select RSA_4096.

8. In the Advanced options section, select the Single-Region Key option.

9. Click Next.

   The Add labels screen appears.

10. In the Alias field, specify the display name for the key, and then click Next.

    The Review and edit key policy screen appears.

11. Click Finish.

    The Customer managed keys screen appears, displaying the newly created customer master key.

12. Click the key alias.

    A screen specifying the configuration for the selected key appears.

13. In the General Configuration section, copy the value specified in the ARN field, and save it on your local machine.

    You need to attach the key to the KMSDecryptAccess policy. You also need to specify this ARN value in the command for creating a Kubernetes secret for the key.

14. Navigate to Services > IAM.

15. Click Policies.

    The Policies screen appears.

16. Select the KMSDecryptAccess policy.

    The Permissions tab appears.

17. Click Edit policy to edit the policy in JSON format.

18. Modify the policy to add the ARN of the key that you have copied in step 13 to the Resource parameter.

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": "kms:Decrypt",
                "Resource": [
                    "<ARN of the AWS Customer Master Key>"
                ]
            }
        ]
    }
    

19. Click Review policy, and then click Save changes to save the changes to the policy.

3.5.1.2 - Creating an AWS S3 Bucket

Important: This procedure is optional, and is required only if you want to use AWS S3 for storing the policy snapshot during static deployment, instead of the persistent volume.

To create an AWS S3 bucket:

1. Login to the AWS environment.

2. Navigate to Services.

   A list of AWS services appears.

3. In Storage, click S3.

   The S3 buckets screen appears.

4. Click Create bucket.

   The Create bucket screen appears.

5. In the General configuration screen, specify the following details.

   1. In the Bucket name field, enter a unique name for the bucket.

   2. In the AWS Region field, choose the same region in which you want to create your EC2 instance.

   If you want to configure your bucket or set any specific permissions, then you can specify the required values in the remaining sections of the screen. Otherwise, you can directly go to the next step to create a bucket.

6. Click Create bucket.

   The bucket is created.

3.5.1.3 - Creating an AWS EFS

Important: This procedure is optional, and is required only if you want to use AWS EFS for storing the policy package during static deployment, instead of AWS S3.

To create an AWS EFS:

1. Login to the AWS environment.

2. Navigate to Services.

   A list of AWS services appears.

3. In Storage, click EFS.

   The File Systems screen appears.

4. Click Create file system.

   The Configure network access screen appears.

5. In the VPC list, select the VPC where you will be creating the Kubernetes cluster.

6. Click Next Step.

   The Configure file system settings screen appears.

7. Click Next Step.

   The Configure client access screen appears.

8. Click Next Step.

   The Review and create screen appears.

9. Click Create File System.

   The file system is created.

   Note the value in the File System ID column. You need to specify this value as the value of the volumeHandle parameter in the pv.yaml file in step 10c.

10. Perform the following steps if you want to use a persistent volume for storing the policy package instead of the AWS S3 bucket.

    a. Create a file named storage_class.yaml for creating an AWS EFS storage class.

    The following snippet shows the contents of the storage_class.yaml file.

      kind: StorageClass
      apiVersion: storage.k8s.io/v1
      metadata:
        name: efs-sc
      provisioner: efs.csi.aws.com
    

    Important: If you want to copy the contents of the storage_class.yaml file, then ensure that you indent the file as per YAML requirements.

    b. Run the following command to provision the AWS EFS using the storage_class.yaml file.

    kubectl apply -f storage_class.yaml

    An AWS EFS storage class is provisioned.

    c. Create a file named pv.yaml for creating a persistent volume resource.

    The following snippet shows the contents of the pv.yaml file.

      apiVersion: v1
      kind: PersistentVolume
      metadata:
        name: efs-pv1
        labels:
          purpose: policy-store
      spec:
        capacity:
          storage: 1Gi
        volumeMode: Filesystem
        accessModes:
          - ReadWriteMany
        persistentVolumeReclaimPolicy: Retain
        storageClassName: **efs-sc**
        csi:
          driver: efs.csi.aws.com
          volumeHandle: **fs-618248e2:**/
    

    Important: If you want to copy the contents of the pv.yaml file, then ensure that you indent the file as per YAML requirements.

    This persistent volume resource is associated with the AWS EFS storage class that you have created in step 10b.

    In the storageClassName parameter, ensure that you specify the same name for the storage class that you specified in the storage_class.yaml file in step 10a.

    For example, specify efs-sc as the value of the storageClassName parameter.

    d. Run the following command to create the persistent volume resource.

    kubectl apply -f pv.yaml

    A persistent volume resource is created.

    e. Create a file named pvc.yaml for creating a claim on the persistent volume that you have created in step 10d.

    The following snippet shows the contents of the pvc.yaml file.

      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
        name: efs-claim1
      spec:
        selector:
          matchLabels:
            purpose: "policy-store"
        accessModes:
          - ReadWriteMany
        storageClassName: **efs-sc**
        resources:
          requests:
            storage: 1Gi
    

    Important: If you want to copy the contents of the pvc.yaml file, then ensure that you indent the file as per YAML requirements.

    This persistent volume claim is associated with the AWS EFS storage class that you have created in step 10b. The value of the storage parameter in the pvc.yaml defines the storage that is available for saving the policy dump.

    In the storageClassName parameter, ensure that you specify the same name for the storage class that you specified in the storage_class.yaml file in step 10a.

    For example, specify efs-sc as the value of the storageClassName parameter.

    f. Run the following command to create the persistent volume claim.

    kubectl apply -f pvc.yaml -n <Namespace>

    For example:

    kubectl apply -f pvc.yaml -n iap-rest

    A persistent volume claim is created. In this example, iap-rest is the namespace where the REST protector will be deployed.

    g. On the Linux instance, create a mount point for the AWS EFS by running the following command.

    mkdir /efs

    This command creates a mount point efs on the file system.

    h. Install the Amazon EFS client using the following command.

    sudo yum install -y amazon-efs-utils

    For more information about installing the EFS client, refer to the section Manually installing the Amazon EFS client in the Amazon Elastic File System User Guide.

    i. Run the following mount command to mount the AWS EFS on the directory created in step 10g.

    sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport <file-system-id>.efs.<aws-region>.amazonaws.com:/ /efs

    For example:

    sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-618248e2.efs.<aws-region>.amazonaws.com:/ /efs

    Ensure that you set the value of the <file-system-id> parameter to the value of the volumeHandle parameter, as specified in the pv.yaml file in step 10c.

    For more information about the permissions required for mounting an AWS EFS, refer to the section Working with Users, Groups, and Permissions at the Network File System (NFS) Level in the AWS documentation.

3.5.2 - Creating a Kubernetes Cluster

Note: The steps listed in this section for creating a Kubernetes cluster are for reference use. If you have an existing Kubernetes cluster or want to create a Kubernetes cluster based on your own requirements, then you can directly navigate to step 4 to connect your Kubernetes cluster and the Linux instance. However, you must ensure that your ingress port is enabled on the Network Security group of your VPC.

Important: If you have an existing Kubernetes cluster or want to create a Kubernetes cluster using a different method, then you must install the Kubernetes Metrics Server and Cluster Autoscaler before deploying the Release.

To create a Kubernetes cluster:

1. Create a key pair for the EC2 instance on which you want to create the Kubernetes cluster.

   For more information on creating the key pair, refer to the section Create a key pair for your Amazon EC2 instance in the Amazon EC2 documentation.

   After the key pair is created, you need to specify the key pair name in the publicKeyName field of the createCluster.yaml file, for creating a Kubernetes cluster.

2. Login to the Linux instance, and create a file named createCluster.yaml to specify the configurations for creating the Kubernetes cluster.

   The following snippet displays the contents of the createCluster.yaml file.

   apiVersion: eksctl.io/v1alpha5
   kind: ClusterConfig
   metadata:
     name: <Name of your Kubernetes cluster>
     region: <Region where you want to deploy your Kubernetes cluster>
     version: "<Kubernetes version>"
   vpc:
     id: "<ID of the VPC where you want to deploy the Kubernetes cluster>"
     subnets: #In this section specify the subnet region and subnet id accordingly
       private:
         <Availability zone for the region where you want to deploy your Kubernetes cluster>:
             id: "<Subnet ID>"
         <Availability zone for the region where you want to deploy your Kubernetes cluster>
             id: "<Subnet ID>"
   nodeGroups:
     - name: <Name of your Node Group>
       instanceType: m5.large
       minSize: 1
       maxSize: 3
       tags:
         k8s.io/cluster-autoscaler/enabled: "true"
         k8s.io/cluster-autoscaler/<Name of your Kubernetes cluster>: "owned"
       privateNetworking: true
       securityGroups:
         withShared: true
         withLocal: true
         attachIDs: ['<Security group linked to your VPC>']
       ssh:
         publicKeyName: '<EC2 keypair>'
       iam:
         attachPolicyARNs:
           - "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
         withAddonPolicies:
           autoScaler: true
   

   Important: If you want to copy the contents of the createCluster.yaml file, then ensure that you indent the file as per YAML requirements.

   For more information about the sample configuration file used to create a Kubernetes cluster, refer to the section Create cluster using config file in the eksctl documentation.

   In the ssh/publicKeyName parameter, you must specify the value of the key pair that you have created in step 1.

   In the iam/attachPolicyARNs parameter, you must specify the following policy ARNs:

   • ARN of the AmazonEKS_CNI_Policy policy - This is a default AWS policy that enables the Amazon VPC CNI Plugin to modify the IP address configuration on your EKS nodes.

     For more information about this policy, refer to the AWS documentation.

     You need to sign in to your AWS account to access the AWS documentation for this policy.

   The content snippet displays the reference configuration required to create a Kubernetes cluster using a private VPC. If you want to use a different configuration for creating your Kubernetes cluster, then you need to refer to the section Creating and managing clusters in the eksctl documentation.

   For more information about creating a configuration file to create a Kubernetes cluster, refer to the section Creating and managing clusters in the eksctl documentation.

3. Run the following command to create a Kubernetes cluster.

   eksctl create cluster -f ./createCluster.yaml

   Important: IAM User 1, who creates the Kubernetes cluster, is automatically assigned the cluster-admin role in Kubernetes.

4. Run the following command to connect your Linux instance to the Kubernetes cluster.

   aws eks update-kubeconfig --name <Name of Kubernetes cluster>

5. Validate whether the cluster is up by running the following command.

   kubectl get nodes

   The command lists the Kubernetes nodes available in your cluster.

6. Deploy the Cluster Autoscaler component to enable the autoscaling of nodes in the EKS cluster.

   This step is required only if the Cluster Autoscaler component is not installed.

   For more information about deploying the Cluster Autoscaler, refer to the section Deploy the Cluster Autoscaler in the Amazon EKS documentation.

7. Install the Metrics Server to enable the horizontal autoscaling of pods in the Kubernetes cluster.

   This step is required only if the Metric Server is not installed.

   For more information about installing the Metrics Server, refer to the section Horizontal Pod Autoscaler in the Amazon EKS documentation.

   After you have created the Kubernetes cluster, you can deploy the AP-REST container using dynamic or static mode of deployment.

8. Run following commands to tag the cluster subnets to ensure that the Elastic load balancer can discover them.

   • aws ec2 create-tags --tags Key=kubernetes.io/cluster/<Cluster Name>,Value=shared --resources <Subnet ID>
   • aws ec2 create-tags --tags Key=kubernetes.io/role/internal-elb,Value=1 --resources <Subnet ID>
   • aws ec2 create-tags --tags Key=kubernetes.io/role/elb,Value=1 --resources <Subnet ID>

   
   Repeat this step for all the cluster subnets.

4 - Installing the Protector

This section provides an overview of the steps required to install the REST Container using either the Static or the Dynamic method.

4.1 - Deploying REST Container for Dynamic Method

This section describes how to deploy the REST Container integrated with RPP. Deploy in the following order:

1. Log Forwarder
2. RPP
3. REST Container

4.1.1 - Deploying Log Forwarder

The Log Forwarder is deployed as a DaemonSet. The following steps describe how to deploy Log Forwarder.

1. On the Linux instance, run the following command to create the namespace required for Helm deployment.

   kubectl create namespace <Namespace name>
   

   For example:

   kubectl create namespace iap-rest
   

2. On the Linux instance, navigate to the location where you have extracted the Helm charts to deploy the Log Forwarder.

   For more information about the extracted Helm charts, refer to the section Extracting the Installation Package.

   The logforwarder > values.yaml file contains the default configuration values for deploying the Log Forwarder container on the Kubernetes cluster. The following content shows an extract of the values.yaml file.

    ...
   
     # - Protegrity PSU(Protegrity Storage Unit)/ESA configuration.
     # Logforwarder will send audit records to below specified hosts/ip.
     # User can specify multiple PSU/ESA distribute the audit records and avoid downtime.
     opensearch:
      # -- specify a given name to uniquely identify PSU/ESA in the deployment.
      - name:
         # -- hostname/ip address of PSU/ESA
         host:
         # -- port address of ESA/PSU
         port: 9200
      # - name: node-2
      #   host: test-insight
      #   port: 9200
   
     # -- Kubernetes service configuration, represents a TCP endpoint to receive audit records
     # from the protectors.
     service:
       # -- Configure service type: ClusterIP for Logforwarder endpoint.
       type: ClusterIP
       # -- port to accept incoming audit records from the protector
       port: 15780
   
    ...
   

3. Modify the default values in the values.yaml file as required.

4. Run the following command to deploy the Log Forwarder on the Kubernetes cluster.

helm install <Release_Name> --namespace <Namespace where you want to deploy the RPP container> <Location of the directory that contains the Helm charts>

For example:

helm install log1 --namespace iap-rest <Custom_path>/commonlogforwarder/

<Custom_path> is the directory where you have extracted the installation package.

5. Run the following command to check the status of the pods.

kubectl get pods -n <Namespace>

For example:

kubectl get pods -n iap-rest

NAME                                         READY   STATUS    RESTARTS        AGE

log1-logforwarder-f6gvj                      1/1     Running   0               11h

log1-logforwarder-ls4hn                      1/1     Running   0               11h

log1-logforwarder-phk4t                      1/1     Running   0               11h

log1-logforwarder-z2mz7                      1/1     Running   0               11h

As the Log Forwarder is deployed as a DaemonSet, one instance of Log Forwarder is deployed on each node. In this example, one Log Forwarder pod is deployed per node.

For information about configuring the Log Forwarder, refer to the section Configuration Parameters for Forwarding Audits and Logs.

4.1.2 - Deploying Resilient Package Proxy (RPP)

The following steps describe how to deploy RPP.

Note: Ensure that you have deployed the Log Forwarder before deploying the RPP. For more information about deploying the Log Forwarder, refer to the section Deploying the Log Forwarder.

1. Run the following command on the Jump box to generate the common certificate from the ESA certificates.

CertificatesSetup_Linux_x64_<Version>.sh -u <User> -p <Password> -h <Hostname or IP address of ESA> --port <Port number of ESA> -d <Directory>

For example:

CertificatesSetup_Linux_x64_<Version>.sh -u admin -p admin12345 -h 10.10.10.10 --port 8443 -d rpproxy

For more information about generating the ESA certificates, refer to the section Creating Certificates.

The following files are created:

• CA.pem
• cert.key
• cert.pem
• secret.txt

2. Run the following command to create a Kubernetes secret using the common certificate generated in step 1.

kubectl -n <Namespace> create secret generic common-cert --from-file=CA.pem=./CA.pem  --from-file=cert.key=./cert.key --from-file=cert.pem=./cert.pem --from-file=secret.txt=./secret.txt

Specify this secret as the value of the commonCertSecrets parameter in the values.yaml file. In this case, this secret is used in the following ways:

• RPP uses the certificate as an upstream server certificate to download the policy packages from the ESA.
• The protector uses the certificate as a client certificate to download the policy packages from the RPP.

If you do not specify any value for the commonCertSecrets parameter, then you need to specify separate values for the rpp/upstream/certificateSecret and service/certificateSecret parameters.

3. Run the following command on the Jump box to generate the upstream certificate between the ESA and the RPP.

CertificatesSetup_Linux_x64_<Version>.sh -u <User> -p <Password> -h <Hostname or IP address of ESA> --port <Port number of ESA> -d <Directory>

For example:

CertificatesSetup_Linux_x64_<Version>.sh -u admin -p admin12345 -h 10.10.10.10 --port 8443 -d rpproxy

For more information about generating the ESA certificates, refer to the section Creating Certificates.

The following files are created:

• CA.pem
• cert.key
• cert.pem
• secret.txt

Note: This certificate is created only if you are not using the common certificate.

4. Run the following command to create a Kubernetes secret using the upstream certificate generated in step 3.

kubectl -n <Namespace> create secret generic upstream-cert --from-file=CA.pem=./CA.pem  --from-file=cert.key=./cert.key --from-file=cert.pem=./cert.pem --from-file=secret.txt=./secret.txt

Note: This secret is created only if you are not using the common certificate.

Specify this secret as the value of the rpp/upstream/certificateSecret parameter in the values.yaml file.

5. Run the following command to generate the service TLS certificate.

CreateCertificate_Linux_x64_<Version>.sh server --name <Directory> --dns <Release_Name>.<namespace>.svc

For example:

CreateCertificate_Linux_x64_<Version>.sh server --name rpproxy --dns rpp.iap-rest.svc

For more information about generating the server certificates, refer to the section Creating Certificates.

The following client certificates files are created in the rpproxy folder:

• cert.pem
• cert.key
• CA.pem
• secret.txt

These certificate is used by the protector as a server certificate to authenticate the RPP to download policy packages.

Ensure that the namespace and release name that you specify in this command are the same names that you specify in step 7 while deploying the RPP Helm chart.

Note: This certificate is created only if you are not using the common certificate.

6. Run the following command to generate the secret for the service TLS certificate.

kubectl -n <Namespace> create secret generic service-certs --from-file=CA.pem=<path-to-CA.pem> --from-file=cert.key=<path-to-cert.key> --from-file=cert.pem=<path-to-cert.pem> --from-file=secret.txt=<path-to-secret.txt>

For more information about generating the client certificates, refer to the section Creating Certificates.

Note: This secret is created only if you are not using the common certificate.

Specify this secret as the value of the service/certificateSecret parameter in the values.yaml file.

7. On the Linux instance, navigate to the location where you have extracted the Helm charts to deploy the RPP.

For more information about the extracted Helm charts, refer to the section Initializing the Linux instance.

The rp-proxy > values.yaml file contains the default configuration values for deploying the RPP container on the Kubernetes cluster.

...

podSecurityContext:
  fsGroup: 1000

...

#-- k8s secret for storing common certificates
# eg. kubectl command: 
#     kubectl -n $RPP_NAMESPACE create secret generic common-certs \
#     --from-literal=CA.pem=<path-to-CA.pem> --from-literal=cert.key=<path-to-cert.key> \
#     --from-literal=cert.pem=<path-to-cert.pem> --from-literal=secret.txt=<path-to-secret.txt>
commonCertSecrets:

rpp:
  #-- upstream configuration
  # host: Upstream host to connect
  # port: Upstream port to connect
  upstream:
    host:
    port: 25400
    #-- certificateSecret : k8s secret for storing upstream tls certificates 
    # NOTE : Only to be set when not using common certificate secret
    # eg. kubectl command: 
    #     kubectl -n $RPP_NAMESPACE create secret generic upstream-certs \
    #     --from-literal=CA.pem=<path-to-CA.pem> --from-literal=cert.key=<path-to-cert.key> \
    #     --from-literal=cert.pem=<path-to-cert.pem> --from-literal=secret.txt=<path-to-secret.txt>
    certificateSecret:

  #-- logging configuration
  # logLevel: Specifies the logging level for rpproxy
  # INFO (default)
  # ERROR
  # WARN
  # DEBUG
  # TRACE
  # logHost: Host to forward the logs (Default : 127.0.0.1)
  # logPort: Port to forward the logs (Default : 15780)
  logging:
    logLevel: "INFO"
    logHost: "127.0.0.1"
    logPort: 15780

  #-- service configuration
  # certificateSecret : k8s secret for storing service tls certificates
  # NOTE : Only to be set when not using common certificate secret
  # eg. kubectl command: 
  #     kubectl -n $RPP_NAMESPACE create secret generic service-certs \
  #     --from-literal=CA.pem=<path-to-CA.pem> --from-literal=cert.key=<path-to-cert.key> \
  #     --from-literal=cert.pem=<path-to-cert.pem> --from-literal=secret.txt=<path-to-secret.txt>
  # cacheTTL: 
  # TTL sets the duration (in seconds) of which a cached item is considered fresh.
  # When a cached item's TTL expires, the item will be revalidated.
  service:
    certificateSecret:
    cacheTTL: 60

...

8. Modify the default values in the values.yaml file as required.

9. Run the following command to deploy the RPP on the Kubernetes cluster.

helm install <Release_Name> --namespace <Namespace where you want to deploy the RPP container> <Location of the directory that contains the Helm charts>

For example:

helm install rpp --namespace iap-rest rpproxy/

Ensure that you specify the same release name and namespace that you have used while creating the service TLS certificate in step 5.

10. Run the following command to check the status of the pods.

kubectl get pods -n <Namespace>

For example:

kubectl get pods -n iap-rest

NAME                                         READY   STATUS    RESTARTS        AGE

rpp-rpproxy-5fd7d859b6-p9544                 1/1     Running   0               11h

4.1.3 - Deploying the REST Container with Dynamic Method

The following steps describe how to deploy the REST Container.

1. Run the following command to generate the client certificate for connecting to the RPP.

CreateCertificate_Linux_x64_<Version>.sh client --name <Directory> --dns <Release_Name>.<namespace>.svc

For example:

CreateCertificate_Linux_x64_<Version>.sh client --name rpproxy-client --dns rpp.iap-rest.svc

For more information about generating the client certificates, refer to the section Creating Certificates.

The following client certificates files are created in the rpproxy-client folder:

• cert.pem
• cert.key
• CA.pem
• secret.txt

This certificate is used by the protector as a client certificate to authenticate the RPP to download policy packages.

Ensure that the namespace and release name that you specify in this command are the same names that you specify in step 7 while deploying the RPP Helm chart.

2. Run the following command to generate the secret for the RPP client certificate created in step 1.

kubectl -n <RPP_Namespace> create secret generic rpp-client-certs --from-file=CA.pem=<path-to-CA.pem> --from-file=cert.key=<path-to-cert.key> --from-file=cert.pem=<path-to-cert.pem> --from-file=secret.txt=<path-to-secret.txt>

For more information about generating the client certificates, refer to the section Creating Certificates.

Specify this secret as the value of the protector/policy/certificates parameter in the values.yaml file.

3. Run the following command to generate the TLS certificate for the server that hosts the REST Container endpoint.

CreateCertificate_Linux_x64_<Version>.sh server --name <Directory> --dns <DNS_Name> --noenc

CreateCertificate_Linux_x64_<Version>.sh server --name rest-server --dns test-sampleapp-10-v1.example.com --noenc

The following server certificates files are created in the rest-server folder:

• cert.pem
• cert.key
• CA.pem

For more information about generating the certificates, refer to step 6 in section Creating Certificates.

4. Run the following command to generate a secret using the server certificate for the REST Container endpoint.

kubectl -n <Namespace> create secret generic pty-rest-server-secret --from-file=CA.pem=<path-to-CA.pem> --from-file=cert.key=<path-to-cert.key> --from-file=cert.pem=<path-to-cert.pem>

For more information about generating the server certificates, refer to the section Creating Certificates.

Specify this secret as the value of the service/certificates parameter in the values.yaml file.

5. Run the following command to generate the client certificate for accessing the REST Container endpoint.

CreateCertificate_Linux_x64_<Version>.sh client --name <Directory> --dns <Namespace_name> --noenc

CreateCertificate_Linux_x64_<Version>.sh client --name rest-client --dns test-sampleapp-10-v1.example.com --noenc

The following client certificates files are created in the rest-client folder:

• cert.pem
• cert.key
• CA.pem

These certificates are used in the curl command for invoking the REST APIs.

For more information about generating the certificates, refer to step 6 in section Creating Certificates.

6. On the Linux instance, navigate to the location where you have extracted the Helm charts to deploy the REST Container.

   The dynamic > values.yaml file contains the default configuration values for deploying the RPP container on the Kubernetes cluster.

# -- create image pull secrets and specify the name here.
# remove the [] after 'imagePullSecrets:' once you specify the secrets
imagePullSecrets: []
# - name: regcred

nameOverride: ""
fullnameOverride: ""

# REST protector image configuration
iaprestImage:
  # -- rest protector image registry address
  repository:
  # -- rest protector image tag name
  tag:
  # -- The pullPolicy for a container and the tag of the image affect 
  # when the kubelet attempts to pull (download) the specified image.
  pullPolicy: IfNotPresent

# Docker Hub Image (Root User): docker.io/nginx:stable 
# To use nginx image that runs with non-root permissions
# Ref. https://hub.docker.com/r/nginxinc/nginx-unprivileged
nginxImage:
  # -- nginx image registry address
  repository:
  # -- nginx image tag name
  tag: 
  # -- The pullPolicy for a container and the tag of the image affect 
  # when the kubelet attempts to pull (download) the specified image.
  pullPolicy: IfNotPresent

# specify CPU and memory requirement of REST protector container
iaprestResources:
  limits:
    cpu: 1000m 
    memory: 3000Mi
  requests:
    cpu: 500m
    memory: 800Mi

# specify CPU and memory requirement of nginx proxy container
nginxResources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 200m
    memory: 200Mi

...
   
## -- pod service account to be used
## leave the field empty if not applicable
serviceAccount:
  # The name of the service account to use.
  name:

# Specify any additional annotation to be associated with pod
podAnnotations:
  checksum/nginx-config: '{{ include (print $.Template.BasePath "/nginx-configmap.yaml") . | sha256sum }}'
  checksum/rest-config: '{{ include (print $.Template.BasePath "/rest-configmap.yaml") . | sha256sum }}'

## set the Pod's security context object
## leave the field empty if not applicable
podSecurityContext:
  fsGroup: 1000

## set the iapRest Container's security context object
## leave the field empty if not applicable
iaprestContainerSecurityContext:
  capabilities:
    drop:
    - ALL
  allowPrivilegeEscalation: false
  privileged : false
  runAsNonRoot : true
  readOnlyRootFilesystem: true
  seccompProfile:
    type: RuntimeDefault

## set the nginx Container's security context object
## leave the field empty if not applicable
nginxContainerSecurityContext:
  capabilities:
    drop:
    - ALL
  allowPrivilegeEscalation: false
  privileged : false
  runAsNonRoot : true
  readOnlyRootFilesystem: true
  seccompProfile:
    type: RuntimeDefault

# protector configuration
protector:
  # Policy information for the protector initialization
  policy:
    # Cadence determines how often the protector connects with ESA / proxy to 
    # fetch the policy updates in background. Default is 60 seconds. 
    # So by default, every 60 seconds protector tries to fetch the policy updates.
    # If the cadence is set to "0", then the protector will get the policy only 
    # once, which is not recommended.
    #
    # Default 60.
    cadence: 60

    # -- Host/IP to the service providing Resilient Packages either rpproxy 
    # service or ESA.
    host:

    # -- certificates used to communicate with service providing Resilient packages.
    # specify certificate secret name.
    # -- TLS certificate rp-proxy service.
    # kubectl -n $NAMESPACE create secret generic pty-rpp-tls \
    #   --from-file=cert.pem=./certs/cert.pem \
    #   --from-file=cert.key=./certs/cert.key \
    #   --from-file=CA.pem=./ca/CA.pem \
    #   --from-file=secret.txt=./certs/secret.txt
    certificates: 
  
  # Logforwarder configuration
  logs:
    # -- In case that connection to fluent-bit is lost, set how audits/logs are handled
    # 
    # drop  : Protector throws logs away if connection to the fluentbit is lost.
    # error : (default) Protector returns error without protecting/unprotecting 
    #         data if connection to the fluentbit is lost.
    mode: error

    # -- Host/IP to fluent-bit where audits/logs will be forwarded from the protector
    #
    # Default localhost
    host:

# nginx configuration
nginx:
  # configure audit records generate by nginx service.
  # The generated records are sent to stdout.
  # Error logs are enabled by default.
  logs:
    # -- configure http client request access logs, by default the records
    # are sent to stdout
    request_logs: false
    # -- configure kubelet health check probe access logs, by default the records
    # are sent to stdout.
    probe_logs: false

# -- specify the initial no. of rest Pod replicas
replicaCount: 1

# HPA configuration
autoScaling:
  # -- lower limit on the number of replicas to which the autoscaler
  # can scale down to.
  minReplicas: 1
  # -- upper limit on the number of replicas to which 
  # the autoscaler can scale up. It cannot be less that minReplicas.
  maxReplicas: 10
  # -- CPU utilization threshold which triggers the autoscaler
  targetCPU: 70

# Kubernetes service configuration, represents a HTTP service to host
# REST protector endpoint.
service:
  # -- Configure service type: LoadBalancer or ClusterIP for rest protector
  # endpoint
  type: ClusterIP
  port: 443

  # -- secret name containing server TLS certificates to host 
  # rest protector endpoint.
  # kubectl -n $NAMESPACE create secret generic pty-rest-tls \
  #   --from-file=cert.pem=./certs/cert.pem \
  #   --from-file=cert.key=./certs/cert.key \
  #   --from-file=CA.pem=./ca/CA.pem
  certificates:

  # -- Specify k8s service related annotations
  # annotation can configure internal load balancer
  # AWS internal load balancer
  #service.beta.kubernetes.io/aws-load-balancer-internal: "true"
  # AZURE internal load balancer
  #service.beta.kubernetes.io/azure-load-balancer-internal: "true"
  # GCP internal load balancer
  #networking.gke.io/load-balancer-type: "Internal" 
  annotations:
    #service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    #service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    #networking.gke.io/load-balancer-type: "Internal"

7. Modify the default values in the values.yaml file as required.

8. Run the following command to deploy the REST Container on the Kubernetes cluster.

helm install <Release_Name> --namespace <Namespace where you want to deploy the REST container> <Location of the directory that contains the Helm charts>

For example:

helm install iap-rest-dynamic --namespace iap-rest dynamic/

9. Run the following command to check the status of the pods.

kubectl get pods -n <Namespace>

For example:

kubectl get pods -n iap-rest

NAME                                         READY   STATUS    RESTARTS        AGE

iap-rest-iap-rest-dynamic-7b97d5dff7-grqph   2/2     Running   0               11h

log1-logforwarder-f6gvj                      1/1     Running   0               11h

log1-logforwarder-ls4hn                      1/1     Running   0               11h

log1-logforwarder-phk4t                      1/1     Running   0               11h

log1-logforwarder-z2mz7                      1/1     Running   0               11h

rpp-rpproxy-5fd7d859b6-p9544                 1/1     Running   0               11h

4.1.4 - Uninstalling the Protector in Dynamic Method

To uninstall the Protector:

1. Run the following command to uninstall the Log Forwarder from the Kubernetes cluster.

helm uninstall <Release_Name> --namespace <Namespace where the Log Forwarder is deployed>

For example:

helm uninstall log1 --namespace iap-rest

2. Run the following command to uninstall the RPP from the Kubernetes cluster.

helm uninstall <Release_Name> --namespace <Namespace where RPP is deployed>

For example:

helm uninstall rpp --namespace iap-rest

3. Run the following command to uninstall the REST Container from the Kubernetes cluster.

helm uninstall <Release_Name> --namespace <Namespace where the REST Container is deployed>

For example:

helm uninstall iap-rest-dynamic --namespace iap-rest

4. Run the following command to delete the Kubernetes secrets.

kubectl delete secret <Secret_Name> --namespace <Namespace where the REST Container is deployed>

For example:

kubectl delete secret common-cert --namespace iap-rest

Repeat this step to delete all the secrets that you have created while deploying the RPP and the REST Container:

• common-cert
• upstream-cert
• service-certs
• rpp-client-certs
• pty-rest-server-secret
• regcred

5. Run the following command to delete the Kubernetes namespace.

helm delete namespace <Namespace where the REST Container is deployed>

For example:

helm delete namespace iap-rest

4.2 - Deploying REST Product in Static Mode

This section describes how to deploy the REST Container in static mode.

4.2.1 - Retrieving the Policy Package from the ESA

This section describes how to invoke the RPS APIs to retrieve the policy package using the ESA.

Note: Ensure that the Export Resilient Package permission is granted to the role that is assigned to the user exporting the package from the ESA.

Warning: Do not modify the package that has been exported using the RPS Service API.

To retrieve the policy package from the ESA:

1. Download the policy package from the ESA and encrypt the policy package using a KMS, then run the following command.

   If you are using 10.1 ESA, then refer to the section Using the Encrypted Resilient Package REST APIs for more information about the RPS API.

   If you are using 10.2 ESA, then refer to the section Using the Encrypted Resilient Package REST APIs for more information about the RPS API.

   If you are using Protegrity Provisioned Cluster, then navigate to Protegrity Product Documentation. Then, navigate to Edition > AI Team Edition > Infrastructure > Protegrity REST APIs > Using the Encrypted Resilient Package REST APIs for more information about the RPS API.

   The policy package is downloaded to your machine.

2. Copy the policy package file to an AWS S3 bucket or AWS EFS, as required.

4.2.2 - Deploying Log Forwarder

The Log Forwarder is deployed as a DaemonSet. The following steps describe how to deploy Log Forwarder.

1. On the Linux instance, run the following command to create the namespace required for Helm deployment.

   kubectl create namespace <Namespace name>
   

   For example:

   kubectl create namespace iap-rest
   

2. On the Linux instance, navigate to the location where you have extracted the Helm charts to deploy the Log Forwarder.

   For more information about the extracted Helm charts, refer to the section Extracting the Installation Package.

   The logforwarder > values.yaml file contains the default configuration values for deploying the Log Forwarder container on the Kubernetes cluster. The following content shows an extract of the values.yaml file.

    ...
   
     # - Protegrity PSU(Protegrity Storage Unit)/ESA configuration.
     # Logforwarder will send audit records to below specified hosts/ip.
     # User can specify multiple PSU/ESA distribute the audit records and avoid downtime.
     opensearch:
      # -- specify a given name to uniquely identify PSU/ESA in the deployment.
      - name:
         # -- hostname/ip address of PSU/ESA
         host:
         # -- port address of ESA/PSU
         port: 9200
      # - name: node-2
      #   host: test-insight
      #   port: 9200
   
     # -- Kubernetes service configuration, represents a TCP endpoint to receive audit records
     # from the protectors.
     service:
       # -- Configure service type: ClusterIP for Logforwarder endpoint.
       type: ClusterIP
       # -- port to accept incoming audit records from the protector
       port: 15780
   
    ...
   

3. Modify the default values in the values.yaml file as required.

4. Run the following command to deploy the Log Forwarder on the Kubernetes cluster.

helm install <Release_Name> --namespace <Namespace where you want to deploy the RPP container> <Location of the directory that contains the Helm charts>

For example:

helm install log1 --namespace iap-rest <Custom_path>/commonlogforwarder/

<Custom_path> is the directory where you have extracted the installation package.

5. Run the following command to check the status of the pods.

kubectl get pods -n <Namespace>

For example:

kubectl get pods -n iap-rest

NAME                                         READY   STATUS    RESTARTS        AGE

log1-logforwarder-f6gvj                      1/1     Running   0               11h

log1-logforwarder-ls4hn                      1/1     Running   0               11h

log1-logforwarder-phk4t                      1/1     Running   0               11h

log1-logforwarder-z2mz7                      1/1     Running   0               11h

As the Log Forwarder is deployed as a DaemonSet, one instance of Log Forwarder is deployed on each node. In this example, one Log Forwarder pod is deployed per node.

For information about configuring the Log Forwarder, refer to the section Configuration Parameters for Forwarding Audits and Logs.

4.2.3 - Deploying KMSProxy Container

The following steps describe how to deploy the KMSProxy container.

1. Run the following command to generate the TLS server certificate for the KMS-Proxy service.

CreateCertificate_Linux_x64_<Version>.sh server --name <Directory> --dns <Release_Name>.<namespace>.svc

For example:

CreateCertificate_Linux_x64_<Version>.sh server --name kms-proxy-server --dns kms-proxy.<namespace>.svc

For more information about generating the client certificates, refer to the section Creating Certificates.

The following server certificates files are created in the kms-proxy-server folder:

• cert.pem
• cert.key
• CA.pem
• secret.txt

These certificates are used by the protector as a server certificate to authenticate the KMS-Proxy service.

Ensure that the namespace and release name that you specify in this command are the same names that you specify in step 5 while deploying the KMS-Proxy Helm chart.

For more information about the data encryption key used in the AWS KMS, refer to the section Creating an Data Encryption Key (DEK).

2. Run the following command to generate the secret for the KMS-Proxy server certificate.

kubectl -n <KMS-Proxy_Namespace> create secret generic service-certs --from-file=CA.pem=<path-to-CA.pem> --from-file=cert.key=<path-to-cert.key> --from-file=cert.pem=<path-to-cert.pem> --from-file=secret.txt=<path-to-secret.txt>

For more information about generating the client certificates, refer to the section Creating Certificates.

Specify this secret as the value of the service/certificateSecret parameter in the values.yaml file.

3. On the Linux instance, navigate to the location where you have extracted the Helm charts to deploy the KMSProxy container.
   For more information about the extracted Helm charts, refer to the section Extracting the Installation Package.

   The kms-proxy > values.yaml file contains the default configuration values for deploying the RPP container on the Kubernetes cluster.

...

    # -- service account must be linked to a cloud role to access appropriate KMS keyid.
    # the cloud role must have decrypt permission on keyid 
    serviceAccount:
    # The name of the service account to use.
      name: 

    # Specify any additional annotation to be associated with pod
    podAnnotations:
      checksum/kmsproxy-config: '{{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}'

    ## set the Pod's security context object
    podSecurityContext:
      fsGroup: 2000

    ## set the Container's security context object
    securityContext:
      capabilities:
       drop:
       - ALL
      readOnlyRootFilesystem: true
      runAsNonRoot: true
      runAsUser: 1000

    #-- cloud kms related configuration
    kms:
    # -- Specify Cloud KMS vendor
    # expected values are: AWS, GCP, AZURE
    vendor: ""

    #--- specify identifier for RSA key hosted by the cloud KMS.
    # In case of AWS identifier is the key ARN (Amazon resource identifier)
    # In GCP, identifier is key resourceid
    # and for Azure identifier is keyid
    keyid: ""

    # kms-proxy service configuration
    application:
    # -- The cache will keep the content(decrypted KEK) for the specified TTL(time to live) 
    # duration in seconds. Once the TTL expires the value from the cache is cleared.
    # Based on amount of time require to update/install the protector deployment, update
    # the ttl. Default is 1200 seconds(20 minutes)
    ttl: 1200

    # -- By default, log level for the application is set to INFO.
    # available logging levels ares INFO, DEBUG, TRACE
    # to enable http access log set the logLevel to TRACE
    logLevel: INFO

    # Kubernetes service configuration, represents a HTTP service to host
    # kms proxy endpoint.
    service:
      # -- Configure service type: ClusterIP for kms-proxy endpoint
      type: ClusterIP
      port: 443
      # -- TLS certificate of kms-proxy service.
      # kubectl -n $NAMESPACE create secret generic pty-kms-proxy-tls \
      #   --from-file=cert.pem=./certs/cert.pem \
      #   --from-file=cert.key=./certs/cert.key \
      #   --from-file=CA.pem=./ca/CA.pem \
     #   --from-file=secret.txt=./certs/secret.txt
      certificates:

4. Modify the default values in the values.yaml file as required.

5. Run the following command to deploy the KMSProxy container on the Kubernetes cluster.

helm install <Release_Name> --namespace <Namespace to deploy KMSProxy container> <Location of the directory containing Helm charts>

For example:

helm install kmsproxy --namespace iap-rest kms-proxy/

6. Run the following command to check the status of the pods.

kubectl get pods -n <Namespace>

For example:

kubectl get pods -n iap-rest

NAME                                         READY   STATUS    RESTARTS        AGE

kms-10-v1-kms-proxy-7b97d5dff7-grqph         2/2     Running   0               11h

log1-logforwarder-f6gvj                      1/1     Running   0               11h

log1-logforwarder-ls4hn                      1/1     Running   0               11h

log1-logforwarder-phk4t                      1/1     Running   0               11h

log1-logforwarder-z2mz7                      1/1     Running   0               11h

4.2.4 - Deploying REST Container Using Static Method

The following steps describe how to deploy the REST Container.

1. Run the following command to generate the client certificate to authenticate with the KMS-Proxy service.

CreateCertificate_Linux_x64_<Version>.sh client --name <Directory> --dns <Release_Name>.<namespace>.svc

For example:

CreateCertificate_Linux_x64_<Version>.sh client --name kms-client --dns kms-proxy.<namespace>.svc

For more information about generating the client certificates, refer to the section Creating Certificates.

The following client certificates files are created in the kms-client folder:

• cert.pem
• cert.key
• CA.pem
• secret.txt

This certificate is used by the protector as a client certificate to authenticate the protector with the KMS-Proxy service.

Ensure that the namespace and release name that you specify in this command are the same names that you specify in step 5 while deploying the KMS-Proxy Helm chart.

2. Run the following command to generate the secret for the KMS-Proxy client certificate created in step 1.

kubectl -n <KMS-Proxy_Namespace> create secret generic service-certs --from-file=CA.pem=<path-to-CA.pem> --from-file=cert.key=<path-to-cert.key> --from-file=cert.pem=<path-to-cert.pem> --from-file=secret.txt=<path-to-secret.txt>

For more information about generating the client certificates, refer to the section Creating Certificates.

Specify this secret as the value of the kms/certificates parameter in the values.yaml file.

3. Run the following command to generate the TLS certificate for the server that hosts the REST Container endpoint.

CreateCertificate_Linux_x64_<Version>.sh server --name <Directory> --dns <DNS_Name> --noenc

CreateCertificate_Linux_x64_<Version>.sh server --name rest-server --dns test-sampleapp-10-v1.example.com --noenc

The following server certificates files are created in the rest-server folder:

• cert.pem
• cert.key
• CA.pem

For more information about generating the certificates, refer to the section Creating Certificates.

4. Run the following command to generate a secret using the server certificate for the REST Container endpoint.

kubectl -n <Namespace> create secret generic pty-rest-server-secret --from-file=CA.pem=<path-to-CA.pem> --from-file=cert.key=<path-to-cert.key> --from-file=cert.pem=<path-to-cert.pem>

For more information about generating the server certificates, refer to the section Creating Certificates.

Specify this secret as the value of the service/certificates parameter in the values.yaml file.

5. Run the following command to generate the client secret for accessing the REST Container endpoint.

CreateCertificate_Linux_x64_<Version>.sh client --name <Directory> --dns <Namespace_name> --noenc

CreateCertificate_Linux_x64_<Version>.sh client --name rest-client --dns test-sampleapp-10-v1.example.com --noenc

The following client certificates files are created in the rest-client folder:

• cert.pem
• cert.key
• CA.pem

These certificates are used in the curl command for invoking the REST APIs.

For more information about generating the certificates, refer to the section Creating Certificates.

6. On the Linux instance, navigate to the location where you have extracted the Helm charts to deploy the REST Container.

   The devops > values.yaml file contains the default configuration values for deploying the RPP container on the Kubernetes cluster.

## -- create image pull secrets and specify the name here.
## remove the [] after 'imagePullSecrets:' once you specify the secrets
imagePullSecrets: []
# - name: regcred

nameOverride: ""
fullnameOverride: ""

# REST protector image configuration
iaprestImage:
  # -- rest protector image registry address
  repository:
  # -- rest protector image tag name
  tag:
  # -- The pullPolicy for a container and the tag of the image affect 
  # when the kubelet attempts to pull (download) the specified image.
  pullPolicy: IfNotPresent

# policy loader sidecar image configuration
policyLoaderImage: 
  # -- policy loader sidecar container image registry address
  repository:
  # -- policy loader sidecar container image tag name
  tag:
  # -- The pullPolicy for a container and the tag of the image affect 
  # when the kubelet attempts to pull (download) the specified image.
  pullPolicy: IfNotPresent

# Docker Hub Image (Root User): docker.io/nginx:stable 
# To use nginx image that runs with non-root permissions
# Ref. https://hub.docker.com/r/nginxinc/nginx-unprivileged
nginxImage:
  # -- nginx image registry address
  repository:
  # -- nginx image tag name
  tag:
  # -- The pullPolicy for a container and the tag of the image affect 
  # when the kubelet attempts to pull (download) the specified image.
  pullPolicy: IfNotPresent

# specify CPU and memory requirement of REST protector container
iaprestResources:
  limits:
    cpu: 1000m 
    memory: 3000Mi
  requests:
    cpu: 500m
    memory: 800Mi

# specify CPU and memory requirement of policy loader container
policyLoaderResources:
  limits:
    cpu: 200m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 200Mi

# specify CPU and memory requirement of nginx proxy container
nginxResources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 200m
    memory: 200Mi

...
   
# -- pod service account to be used.
# A k8s service account can be linked to cloud identity to allow pod to access
# cloud services like Object storage solutions.
serviceAccount: 
  # The name of the service account to use.
  name:

# Specify any additional annotation to be associated with pod
podAnnotations:
  checksum/nginx-config: '{{ include (print $.Template.BasePath "/nginx-configmap.yaml") . | sha256sum }}'
  checksum/rest-config: '{{ include (print $.Template.BasePath "/rest-configmap.yaml") . | sha256sum }}'

# set the Pod's security context object.
podSecurityContext:
  runAsUser: 1000
  runAsGroup: 1000
  fsGroup: 1000

# set the iapRest Container's security context object
iaprestContainerSecurityContext:
  capabilities:
    drop:
    - ALL
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  privileged : false
  seccompProfile:
    type: RuntimeDefault

# -- set the policy loader sidecar Container's security context object
# leave the field empty if not applicable
policyLoaderContainerSecurityContext:
  capabilities:
    drop:
    - ALL
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  privileged : false
  seccompProfile:
    type: RuntimeDefault

# -- set the nginx Container's security context object.
# leave the field empty if not applicable
nginxContainerSecurityContext:
  capabilities:
    drop:
    - ALL
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  privileged : false
  seccompProfile:
    type: RuntimeDefault

# protector configuration
protector:
  # Policy information for the protector initialization
  # Note: Policy update is control by policy puller sidecar, Below configuration
  # are for protector to refresh policy once it is updated by policy puller sidecar.
  policy:
    # -- Cadence determines how often the protector connects local filesystem 
    # to fetch the policy updates in background. Default is 60 seconds. 
    # So by default, every 60 seconds protector tries to fetch the policy updates.
    # If the cadence is set to "0", then the protector will get the policy only 
    # once, which is not recommended.
    cadence: 60

  # KMS proxy service configuration
  kms:
    # -- kms proxy service hostname.
    # kms proxy service helps protector to decrypt resilient policy package.
    host:

    # -- certificates to authenticate with kms proxy service.
    # Specify certificate secret name.
    # kubectl -n $NAMESPACE create secret generic pty-kms-proxy-tls \
    #   --from-file=cert.pem=./certs/cert.pem \
    #   --from-file=cert.key=./certs/cert.key \
    #   --from-file=CA.pem=./ca/CA.pem \
    #   --from-file=secret.txt=./certs/secret.txt
    certificates:

  # Logforwarder configuration
  logs:
    # -- specify log levels.
    # In case that connection to fluent-bit is lost, set how audits/logs are handled
    # 
    # drop  : Protector throws logs away if connection to the fluentbit is lost
    # error : (default) Protector returns error without protecting/unprotecting 
    #         data if connection to the fluentbit is lost
    mode: error

    # -- Host/IP of Logforwarder service where audits/logs are forwarded by the 
    # REST protector
    host:

# nginx configuration
nginx:
  # control audit records generate by nginx proxy.
  # the generated records are sent to stdout.
  # error logs are enabled by default.
  logs:
    # -- configure http client request access logs, by default the records
    # are sent to stdout
    request_logs: false
    # -- configure kubelet health check probe access logs, by default the records
    # are sent to stdout.
    probe_logs: false

# policy puller sidecar configuration
policyPuller:
  policy:
    # -- Control how often the sidecar application will read the configuration 
    # file for policy update information.
    # Interval is reset when previous pull operation is completed.
    # IMPORTANT: do not set interval to 0. 
    interval: 30

    # -- If using VolumeMount as storage destination for policy package
    # specify the persistent volume claim name to be used to mount the volume.
    pvcName:

    # -- Path to KMS encrypted Resilient policy package. Specify an URL encoded
    # path to package file. Here are few examples,
    # If stored in S3 then, s3://[s3 bucket name]/[to]/<[policy]>/<[package]>
    # If stored in Azure Blob storage then, https://[storage account].blob.core.windows.net/[to]/<[policy]>/<[package]> 
    # If stored in GCS then, gs://[bucket name]/[to]/<[policy]>/<[package]>
    # If stored in local filesystem (VolumeMount) then, [to]/<[policy]>/<[package]>
    # Important: updating it will not trigger pod restart.
    path:
  
  logs:
    # -- control policy puller log level
    # logs are forwarded to stdout
    # Supported Values
    # INFO - default
    # DEBUG
    level: INFO


# -- specify the initial no. of rest Pod replicas
replicaCount: 1

# HPA configuration
autoScaling:
  # -- lower limit on the number of replicas to which the autoscaler
  # can scale down to.
  minReplicas: 1
  # -- upper limit on the number of replicas to which 
  # the autoscaler can scale up. It cannot be less that minReplicas.
  maxReplicas: 10
  # -- CPU utilization threshold which triggers the autoscaler
  targetCPU: 70

# specify service type for rest container.
service:
  # -- Configure service type: LoadBalancer or ClusterIP for rest protector
  # endpoint
  type: ClusterIP
  port: 443

  # -- secret name containing server TLS certificates to host 
  # rest protector endpoint.
  # kubectl -n $NAMESPACE create secret generic pty-rest-tls \
  #   --from-file=cert.pem=./certs/cert.pem \
  #   --from-file=cert.key=./certs/cert.key \
  #   --from-file=CA.pem=./ca/CA.pem
  certificates:

  # -- Specify k8s service related annotations
  # annotation can configure internal load balancer
  # AWS internal load balancer
  #service.beta.kubernetes.io/aws-load-balancer-internal: "true"
  # AZURE internal load balancer
  #service.beta.kubernetes.io/azure-load-balancer-internal: "true"
  # GCP internal load balancer
  #networking.gke.io/load-balancer-type: "Internal" 
  annotations:
    #service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    #service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    #networking.gke.io/load-balancer-type: "Internal"

7. Modify the default values in the values.yaml file as required.

8. Run the following command to deploy the REST Container on the Kubernetes cluster.

helm install <Release_Name> --namespace <Namespace where you want to deploy the REST container> <Location of the directory that contains the Helm charts>

For example:

helm install iap-rest-devops --namespace iap-rest devops/

9. Run the following command to check the status of the pods.

kubectl get pods -n <Namespace>

For example:

kubectl get pods -n iap-rest

NAME                                         READY   STATUS    RESTARTS        AGE

kms-10-v1-kms-proxy-7b97d5dff7-grqph         2/2     Running   0               11h

log1-logforwarder-f6gvj                      1/1     Running   0               11h

log1-logforwarder-ls4hn                      1/1     Running   0               11h

log1-logforwarder-phk4t                      1/1     Running   0               11h

log1-logforwarder-z2mz7                      1/1     Running   0               11h

iap-rest-iap-rest-devops-5fd7d859b6-p9544    1/1     Running   0               11h

Alternatively, if you do not want to modify the values.yaml file, you can use set arguments to update the values during runtime.
For more information about deploying containers using set arguments, refer to the section Appendix - Deploying the Helm Charts by Using the Set Argument.

The test user can run the REST version API to verify the version of the REST protector.

4.2.5 - Updating the Policy Package

The following steps describe how to update the policy or the policy path.

1. Modify the policy or the location where the policy has been uploaded.

2. Run the helm upgrade command to update the policy package or the policy package path.

For example, the line --set policyPuller.policy.path="s3://restcontainer/static-iap-rest-rel-a/try/Sample_App_Policy.tgz" in the following code block indicates that the path where the policy package is stored has changed.

   helm -n devops-10-v2 upgrade test-sampleapp-10-v1 iap-rest-devops/ \

  --set imagePullSecrets[0].name="regcred" \

  --set iaprestImage.repository="<Account_ID>.dkr.ecr.<region_name>.amazonaws.com/container" \

  --set iaprestImage.tag="REST_RHUBI-9-64_x86-64_K8S_10.0.0.16.6a3a67.tgz" \

  --set policyLoaderImage.repository="<Account_ID>.dkr.ecr.<region_name>.amazonaws.com/container" \

  --set policyLoaderImage.tag="POLICY-LOADER_RHUBI-9-64_x86-64_K8S_1.0.0.11.bc1967.tgz" \

  --set nginxImage.repository="nginxinc/nginx-unprivileged" \

  --set nginxImage.tag="1.25.2" \

  --set serviceAccount.name="s3-v1-sa" \

  --set protector.kms.host="test-kms-10-v1-kms-proxy.devops-10-v2.svc" \

  --set protector.kms.certificates="pty-certs-cli-secret" \

  --set protector.logs.mode="error" \

  --set protector.logs.host="test-devops-logforwarder10-v1.devops-10-v2.svc" \

  --set nginx.logs.request_logs="false" \

  --set nginx.logs.probe_logs="false" \

  --set policyPuller.policy.interval="30" \

  --set policyPuller.logs.level="DEBUG" \

  --set protector.policy.cadence="60"\

  --set policyPuller.policy.path="s3://restcontainer/static-iap-rest-rel-a/try/Sample_App_Policy.tgz" \

  --set service.certificates="pty-rest-devops-secret"

For more information about using set arguments to deploy the Protector, refer to the section Appendix - Deploying the Helm Charts by Using the Set Argument.

3. Run the following command to check the status of the pods.

kubectl get pods -n <Namespace>

For example:

kubectl get pods -n iap-rest

NAME                                                   READY   STATUS    RESTARTS        AGE

test-devops-logforwarder10-v1-2m49b                     1/1     Running   0          163m
test-devops-logforwarder10-v1-wwjzh                     1/1     Running   0          165m
test-kms-10-v1-kms-proxy-687657cff9-dlzdz               1/1     Running   0          161m
test-sampleapp-10-v1-iap-rest-devops-54668997cf-kw628   3/3     Running   0          5m11s

4. Run the following command to check the logs.

kubectl logs <Pod_name> -n <Namespace> -f

For example:

kubectl logs test-sampleapp-10-v1-iap-rest-devops-54668997cf-kw628 -n iap-rest -f

The following logs appear on the console output. The line [INFO ] 2025/10/29 11:47:19.335550 runner.go:226: New Policy source path s3://restcontainers/new-10-49-7-212/new/policy-sample-app-10-49-7-212-v1.json indicates that the policy package path has been updated.

Defaulted container "policy-loader" out of: policy-loader, iap-rest-devops, nginx

[INFO ] 2025/10/29 11:45:16.090634 runner.go:104: starting policy loader with version: 1.0.0+13.e0beab

Starting Health Server.

[INFO ] 2025/10/29 11:45:16.090811 runner.go:187: fetching policy from storage media, AWS_S3

[INFO ] 2025/10/29 11:45:16.313683 runner.go:196: Loading policy from source path s3://restcontainers/new-10-49-7-212/policy-v1-10-49-7-212.json

[root@ip-10-49-5-222 ~]# kubectl logs test-sampleapp-10-v1-iap-rest-devops-7f4f9b9cc4-zbbkg -n devops-10-v6 -f

Defaulted container "policy-loader" out of: policy-loader, iap-rest-devops, nginx

[INFO ] 2025/10/29 11:45:16.090634 runner.go:104: starting policy loader with version: 1.0.0+13.e0beab

Starting Health Server.

[INFO ] 2025/10/29 11:45:16.090811 runner.go:187: fetching policy from storage media, AWS_S3

[INFO ] 2025/10/29 11:45:16.313683 runner.go:196: Loading policy from source path s3://restcontainers/new-10-49-7-212/policy-v1-10-49-7-212.json

[INFO ] 2025/10/29 11:45:48.914901 runner.go:220: fetching policy from storage media, AWS_S3

[INFO ] 2025/10/29 11:45:48.914935 runner.go:242: Policy source path is same. Checking based on timestamp.

[INFO ] 2025/10/29 11:45:49.057011 runner.go:250: Policy source is not modified since last fetch. Skipping policy load operation.

[INFO ] 2025/10/29 11:46:19.057887 runner.go:220: fetching policy from storage media, AWS_S3

[INFO ] 2025/10/29 11:46:19.057916 runner.go:242: Policy source path is same. Checking based on timestamp.

[INFO ] 2025/10/29 11:46:19.201224 runner.go:250: Policy source is not modified since last fetch. Skipping policy load operation.

[INFO ] 2025/10/29 11:46:49.201456 runner.go:220: fetching policy from storage media, AWS_S3

[INFO ] 2025/10/29 11:46:49.201485 runner.go:242: Policy source path is same. Checking based on timestamp.

[INFO ] 2025/10/29 11:46:49.335206 runner.go:250: Policy source is not modified since last fetch. Skipping policy load operation.

[INFO ] 2025/10/29 11:47:19.335501 runner.go:220: fetching policy from storage media, AWS_S3

[INFO ] 2025/10/29 11:47:19.335536 runner.go:224: Policy source path is modified. Triggering policy load operation.

[INFO ] 2025/10/29 11:47:19.335545 runner.go:225: Old Policy source path s3://restcontainers/new-10-49-7-212/policy-v1-10-49-7-212.json.

[INFO ] 2025/10/29 11:47:19.335550 runner.go:226: New Policy source path s3://restcontainers/new-10-49-7-212/new/policy-sample-app-10-49-7-212-v1.json

4.2.6 - Uninstalling the Protector in Static Method

To uninstall the Protector:

1. Run the following command to uninstall the Log Forwarder from the Kubernetes cluster.

helm uninstall <Release_Name> --namespace <Namespace where the Log Forwarder is deployed>

For example:

helm uninstall log1 --namespace iap-rest

2. Run the following command to uninstall the KMSProxy container from the Kubernetes cluster.

helm uninstall <Release_Name> --namespace <Namespace where KMSProxy container is deployed>

For example:

helm uninstall kmsproxy --namespace iap-rest

3. Run the following command to uninstall the REST Container from the Kubernetes cluster.

helm uninstall <Release_Name> --namespace <Namespace where the REST Container is deployed>

For example:

helm uninstall iap-rest-devops --namespace iap-rest

4. Run the following command to delete the Kubernetes secrets.

kubectl delete secret <Secret_Name> --namespace <Namespace where the REST Container is deployed>

For example:

kubectl delete secret service-certs --namespace iap-rest

Repeat this step to delete all the secrets that you have created while deploying the KMSProxy container and the REST Container:

• service-certs
• regcred

5. Run the following command to delete the Kubernetes namespace.

helm delete namespace <Namespace where the REST Container is deployed>

For example:

helm delete namespace iap-rest

5 - Application Protector API on REST

This section describes the AP REST APIs available for protection and unprotection of data:

• Version 4 API specification
• Version 1 API specification

5.1 - Version 4 (V4) Application Protector API on REST

5.1.1 - List of REST APIs

This section describes the AP REST APIs available for protection and unprotection of data.

5.1.1.1 - HTTP GET version

URI

https://hostname/v4/version

Method

GET

Parameters

Hostname: Host name of the endpoint, as defined in the AP-REST deployment

Resource: The resource to be used, which is /v4/version

Result

This function returns the current version of the AP REST protector API.

Response

Example

$ curl 'https://<HostName>/v4/version' --cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key

5.1.1.2 - HTTP POST protect

URI

https://hostname/v4/protect

Method

POST

Parameters

Hostname: Host name of the endpoint, as defined in the AP-REST deployment.

Resource: The resource to be used, which is /v4/protect.

Request Body

• User: Name of the user executing the API. The user must be present in the policy.
• Payload:
  • dataElement: Name of the data element used to protect the data. This field is mandatory.
  • data: Data to be protected. This field is mandatory.
  • externalIv: External Initialization Vector (IV) used for protecting the data.
  • externaltweak: External tweak used for protecting the data.

Result

This API returns protected data.

Example 1

Without external IV and external tweak

$ curl --location --request POST 'https://<hostname>/v4/protect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"TE_A_N_S13_L0R0_Y_ST","data":["bG9jaGFu"],"encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 1

Without external IV and external tweak

The following response appears for the status code 200, if the API is invoked successfully.

    {
  "errorCount": 0,
  "results": [
    {
      "id": 1,
      "encoding": "base64",
      "data": [
        "cEJPM2pF"
      ],
      "returnCode": 6
    }
  ]
}

Example 2

With external IV

$ curl --location --request POST 'https://<hostname>/v4/protect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"TE_A_N_S13_L0R0_Y_ST","data":["bG9jaGFu"],"externalIv":"cHJvdGVncml0eQ==","encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 2

With external IV

The following response appears for the status code 200, if the API is invoked successfully.

    {
  "errorCount": 0,
  "results": [
    {
      "id": 1,
      "encoding": "base64",
      "data": [
        "b2Rnb1ky"
      ],
      "returnCode": 6
    }
  ]
}

Example 3

With external tweak

$ curl --location --request POST 'https://<hostname>/v4/protect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"FPE_FF1_LA_APIP_L0R0_ASTNI_M2.UTF8","data":["bG9jaGFu"],"external_tweak_":"eIvJdGKncnl8eS==","encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 3

With external tweak

The following response appears for the status code 200, if the API is invoked successfully.

    {
  "errorCount": 0,
  "results": [
    {
      "id": 1,
      "encoding": "base64",
      "data": [
        "b2Rnb1ky"
      ],
      "returnCode": 6
    }
  ]
}

5.1.1.3 - HTTP POST unprotect

URI

https://hostname/v4/unprotect

Method

POST

Parameters

Hostname: Host name of the endpoint, as defined in the AP-REST deployment.

Resource: The resource to be used, which is /v4/unprotect.

Request Body

• User: Name of the user executing the API.
• Payload:
  • dataElement: Name of the data element used to unprotect the data. This field is mandatory.
  • data: Data to be unprotected. This field is mandatory.
  • externalIv: External Initialization Vector (IV) used for unprotecting the data.
  • externaltweak: External tweak used for unprotecting the data.

Result

This API returns unprotected data.

Example 1

Without external IV and external tweak

$ curl --location --request POST 'https://<hostname>/v4/unprotect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"TE_A_N_S13_L0R0_Y_ST","data":["cEJPM2pF"],"encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 1

Without external IV and external tweak

The following response appears for the status code 200, if the API is invoked successfully.

    {
  "errorCount": 0,
  "results": [
    {
      "id": 1,
      "encoding": "base64",
      "data": [
        "bG9jaGFu"
      ],
      "returnCode": 8
    }
  ]
}

Example 2

With external IV

$ curl --location --request POST 'https://<hostname>/v4/unprotect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"TE_A_N_S13_L0R0_Y_ST","data":["b2Rnb1ky"],"externalIv":"cHJvdGVncml0eQ==","encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 2

With external IV

The following response appears for the status code 200, if the API is invoked successfully.

    {
  "errorCount": 0,
  "results": [
    {
      "id": 1,
      "encoding": "base64",
      "data": [
        "bG9jaGFu"
      ],
      "returnCode": 8
    }
  ]
}

Example 3

With external tweak

$ curl --location --request POST 'https://<hostname>/v4/unprotect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"FPE_FF1_LA_APIP_L0R0_ASTNI_M2.UTF8","data":["b2Rnb1ky"],"external_tweak_":"eIvJdGKncnl8eS==","encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 3

With external tweak

The following response appears for the status code 200, if the API is invoked successfully.

    {
  "errorCount": 0,
  "results": [
    {
      "id": 1,
      "encoding": "base64",
      "data": [
        "bG9jaGFu"
      ],
      "returnCode": 8
    }
  ]
}

5.1.1.4 - HTTP POST reprotect

URI

https://hostname/v4/reprotect

Method

POST

Parameters

Hostname: Host name of the endpoint, as defined in the AP-REST deployment.

Resource: The resource to be used, which is /v4/reprotect.

Request Body

• User: Name of the user executing the API.
• Payload:
  • dataElement: Name of the data element used to initially protect the data. This field is mandatory.
  • newDataElement: Name of the data element used to reprotect the data. This field is mandatory.
  • data: Data to be protected. This field is mandatory.
  • externalIv: External Initialization Vector (IV) used for initially protecting the data.
  • newExternalIv: External IV used for reprotecting the data.
  • externaltweak: External tweak used for initially protecting the data.
  • newExternaltweak: External tweak used for reprotecting the data.

Result

This API reprotects the data.

Example 1

Without external IV and external tweak

$ curl --location --request POST 'https://<hostname>/v4/reprotect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"TE_A_N_S13_L0R0_Y_ST",newDataElement: TE_A_N_S13_L1R3_N,"data":["cEJPM2pF"],"encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 1

Without external IV and external tweak

The following response appears for the status code 200, if the API is invoked successfully.

    {
  "errorCount": 0,
  "results": [
    {
      "id": 1,
      "encoding": "base64",
      "data": [
        "bDlrdGhhbg=="
      ],
      "returnCode": 50
    }
  ]
}

Example 2

With external IV

$ curl --location --request POST 'https://<hostname>/v4/reprotect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"TE_A_N_S13_L0R0_Y_ST",newDataElement: TE_A_N_S13_L1R3_N,"data":["cEJPM2pF"],"externalIv":"cHJvdGVncml0eQ==","newExternalIv":"dJvKdGWndnM0eP==","encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 2

With external IV

The following response appears for the status code 200, if the API is invoked successfully.

   {
 "errorCount": 0,
 "results": [
   {
     "id": 1,
     "encoding": "base64",
     "data": [
       "c2Snd1mz"
     ],
     "returnCode": 50
   }
 ]
}

Example 3

With external tweak

$ curl --location --request POST 'https://<hostname>/v4/reprotect' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
--data '{"payload":[{"id":1,"dataElement":"FPE_FF1_LA_APIP_L0R0_ASTNI_M2.UTF8",newDataElement: FPE_FF1_LA_APIP_L1R1_ASTNI_M2.UTF8,"data":["cEJPM2pF"],"externaltweak":"eIvJdGKncnl8eS==","newExternaltweak_":"eKwLeHXoepN0fQ==","encoding":"base64"}],"user":"user1"}'
--cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  

Response 3

With external tweak

The following response appears for the status code 200, if the API is invoked successfully.

    {
  "errorCount": 0,
  "results": [
    {
      "id": 1,
      "encoding": "base64",
      "data": [
        "d2Tmd1nz"
      ],
      "returnCode": 50
    }
  ]
}

5.1.1.5 - HTTP GET doc

URI

https://hostname/v4/doc

Method

GET

Parameters

Hostname: Host name of the endpoint, as defined in the AP-REST deployment.

Resource: The resource to be used, which is /v4/doc.

Result

This API returns the document specification.

Example

$ curl --location --request GET 'https://<hostname>/v4/doc' \
--header 'Content-Type: application/json' \
--header 'X-Correlation-ID: k81d1fae-7dec-41g0-a765-90a0c31e6wf5' \
 --cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key

Response

The API returns the OpenAPI specifications YAML file.

5.1.1.6 - HTTP Headers

The client should send the required HTTP headers to the server to specify the type of data being sent in the payload. The content type also specifies the type of result being sent by the server to the client.

To send a JSON request and get a JSON response, specify the following HTTP header:

Content-Type: application/json

Only the Content-Type: application/json value is supported. It is mandatory to specify this value in the HTTP header.

To uniquely identify each HTTP request, specify the correlation ID in the HTTP header:

X-Correlation-ID: <Correlation ID>

Correlation ID is used in audit logs. This is an optional value.

5.1.2 - V4 AP REST HTTP Response Codes

The Correlation ID appears in the error message only if it has been specified in the HTTP header.

5.2 - Version 1 (V1) Application Protector API on REST

5.2.1 - List of REST APIs

This section describes the AP REST APIs available for protection and unprotection of data.

5.2.1.1 - HTTP GET version

URI

https://hostname/rest-v1/version

Method

GET

Parameters

Hostname: Host name of the endpoint, as defined in the AP-REST deployment

Resource: The resource to be used, which is /rest-v1/version

Result

This function returns the current version of the AP REST protector API.

Response

Example

$ curl 'https://<HostName>/rest-v1/version' --cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key

5.2.1.2 - HTTP POST protect

URI

https://hostname/rest-v1/protect

Method

POST

Parameters

Hostname: Host name of the endpoint, as defined in the AP-REST deployment

Resource: The resource to be used, which is /rest-v1/protect

Result

This API returns protected data.

The input data must always be Base64 encoded.

Example 1 - without external IV and external tweak

$ curl --location --request POST 'https://<hostname>/rest-v1/protect' \
--connect-to  "<hostname>:443:<AWS LoadBalancer>:443"  \
--header 'Content-Type: application/json' \
 --cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  --data '{ 
  "protect": {
    "policyusername": "Uername",
    "dataelementname": "DataElement1",
    "bulk":{
      "id": 1,
      "data": [
        {
          "id": 1,
          "content": "AFAAcgBvAHQAZQBnAHIAaQB0AHkAMQAyADMANA=="
        },
                {
          "id": 2,
          "content": "AFAAcgBvAHQAZQBnAHIAaQB0AHkAMQAyADMANA=="
        }
      ]
    }
  }
}'

Response 1 - without external IV and external tweak

The following response appears for the status code 200, if the API is invoked successfully.

{
   "protect":{
      "bulk":{
         "id":1,
         "returntype":"success",
         "data":[
            {
               "id":1,
               "returncode":"/rest-v1/returncodes/id/6",
               "returntype":"success",
               "content":"AGoAZABzAHIAdQBlAGMAagBaAEMAMQAyADMANA=="
            },
            {
               "id":2,
               "returncode":"/rest-v1/returncodes/id/6",
               "returntype":"success",
               "content":"AGoAZABzAHIAdQBlAGMAagBaAEMAMQAyADMANA=="
            }
         ]
      }
   }
}

Example 2 - with external IV

$ curl --location --request POST 'https://<hostname>/rest-v1/protect' \
--connect-to  "<hostname>:443:<AWS LoadBalancer>:443"  \
--header 'Content-Type: application/json' \
 --cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  --data '{ 
  "protect": {
    "policyusername": "Uername",
    "dataelementname": "DataElement1",
    "externaliv": "ZXh0ZXJuYWpdg=="
    "bulk":{
      "id": 1,
      "data": [
        {
          "id": 1,
          "content": "RW5eEN2RGZZaw=="
        },
                {
          "id": 2,
          "content": "cmZBcnJTRg=="
        }
      ]
    }
  }
}'

Response 2 - with external IV

The following response appears for the status code 200, if the API is invoked successfully.

{
   "protect":{
      "bulk":{
         "id":1,
         "returntype":"success",
         "data":[
            {
               "id":1,
               "returncode":"/rest-v1/returncodes/id/6",
               "returntype":"success",
               "content":"OG8xZW0QlQ3MQ=="
            },
            {
               "id":2,
               "returncode":"/rest-v1/returncodes/id/6",
               "returntype":"success",
               "content":"blg2Qm5Ddg=="
            }
         ]
      }
   }
}

Example 3 - with external tweak

$ curl --location --request POST 'https://<hostname>/rest-v1/protect' \
--connect-to  "<hostname>:443:<AWS LoadBalancer>:443"  \
--header 'Content-Type: application/json' \
 --cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  --data '{ 
  "protect": {
    "policyusername": "Uername",
    "dataelementname": "DataElement2_FPE",
    "externaltweak": "ZXh0ZXJuYWpdg=="
    "bulk":{
      "id": 1,
      "data": [
        {
          "id": 1,
          "content": "RW5eEN2RGZZaw=="
        },
                {
          "id": 2,
          "content": "cmZBcnJTRg=="
        }
      ]
    }
  }
}'

Response 3 - with external tweak

The following response appears for the status code 200, if the API is invoked successfully.

{
   "protect":{
      "bulk":{
         "id":1,
         "returntype":"success",
         "data":[
            {
               "id":1,
               "returncode":"/rest-v1/returncodes/id/6",
               "returntype":"success",
               "content":"MHM4OVpsRndIbA=="
            },
            {
               "id":2,
               "returncode":"/rest-v1/returncodes/id/6",
               "returntype":"success",
               "content":"VzFsNmd1Ng=="
            }
         ]
      }
   }
}

5.2.1.3 - HTTP POST unprotect

URI

https://hostname/rest-v1/unprotect

Method

POST

Parameters

Hostname: Host name of the endpoint, as defined in the AP-REST deployment

Resource: The resource to be used, which is /rest-v1/unprotect

Result

This API returns unprotected data.

The input data must always be Base64 encoded.

Example 1 - without external IV and external tweak

$ curl --request POST 'https://<hostname>/rest-v1/unprotect' \
--connect-to  "<hostname>:443:<AWS LoadBalancer>:443"  \
--header 'Content-Type: application/json' \
 --cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  --data '{ 
  "unprotect": {
    "policyusername": "UserName",
    "dataelementname": "DataElement1",
    "bulk":{
      "id": 1,
      "data": [
        {
          "id": 1,
          "content": "AFAAcgBvAHQAZQBnAHIAaQB0AHkAMQAyADMANA=="
        },
                {
          "id": 2,
          "content": "AFAAcgBvAHQAZQBnAHIAaQB0AHkAMQAyADMANA=="
        }
      ]
    }
  }
}'

Response 1 - without external IV and external tweak

The following response appears for the status code 200, if the API is invoked successfully.

{
   "unprotect":{
      "bulk":{
         "id":1,
         "returntype":"success",
         "data":[
            {
               "id":1,
               "returncode":"/rest-v1/returncodes/id/8",
               "returntype":"success",
               "content":"AGwATgBWAEwATAByAFIAUAB2AGcAMQAyADMANA=="
            },
            {
               "id":2,
               "returncode":"/rest-v1/returncodes/id/8",
               "returntype":"success",
               "content":"AGwATgBWAEwATAByAFIAUAB2AGcAMQAyADMANA=="
            }
         ]
      }
   }
}

Example 2 - with external IV

$ curl --request POST 'https://<hostname>/rest-v1/unprotect' \
--connect-to  "<hostname>:443:<AWS LoadBalancer>:443"  \
--header 'Content-Type: application/json' \
 --cacert iap-rest-ca.crt --cert iap-rest-client.crt  --key iap-rest-client.key  --data '{ 
  "unprotect": {
    "policyusername": "UserName",
    "dataelementname": "DataElement1",
    "externaliv": "ZXh0ZXJuYWpdg=="
    "bulk":{
      "id": 1,
      "data": [
        {
          "id": 1,
          "content": "OG8xZW0QlQ3MQ=="
        },
                {
          "id": 2,
          "content": "blg2Qm5Ddg=="
        }
      ]
    }
  }
}'

Response 2 - with external IV

The following response appears for the status code 200, if the API is invoked successfully.

{
   "unprotect":{
      "bulk":{
         "id":1,
         "returntype":"success",
         "data":[
            {
               "id":1,
               "returncode":"/rest-v1/returncodes/id/8",
               "returntype":"success",
               "content":"RW5eEN2RGZZaw=="
            },
            {
               "id":2,
               "returncode":"/rest-v1/returncodes/id/8",
               "returntype":"success",
               "content":"cmZBcnJTRg=="
            }
         ]
      }
   }
}