The Protegrity Provisioned Cluster (PPC) is the core framework that forms Protegrity AI Team Edition, designed to deliver a modern, cloud-native experience for data security and governance. Built on Kubernetes, PPC uses a containerized architecture that simplifies deployment and scaling. Using OpenTofu scripts and Helm charts, administrators can stand up clusters with minimal manual intervention, ensuring consistency and reducing operational overhead.
PPC introduces a suite of Protegrity Common Services (PCS) that act as the backbone for AI Team Edition features. These include ingress control for secure traffic routing, certificate management for request validation, and robust authentication and authorization services. PPC also integrates Insight for audit logging and analytics, leveraging OpenSearch and OpenDashboards for visualization and compliance reporting. Beyond this foundation, AI Team Edition delivers advanced capabilities such as policy management, anonymization, data discovery, semantic guardrails, and synthetic data generation — all orchestrated within the PPC cluster. This modular approach ensures scalability, security, and flexibility, making PPC a strategic enabler for organizations adopting cloud-first and containerized environments.
Note: It is recommended to install and use AI Team Edition on AWS to test and use all the features. Install and use on Azure to use the fixed set of protectors supported on Azure.
The documentation in this section is for PPC v1.1.0. To refer to the earlier version of the documentation, refer to the PPC documentation.
1 - Installing PPC on Amazon Web Services (AWS)
Steps to install PPC on AWS
The Protegrity Provisioned Cluster PPC is the core framework that forms the AI Team Edition. It is designed to deliver a modern, cloud-native experience for data security and governance. Built on Kubernetes, PPC uses a containerized architecture that simplifies deployment and scaling. Using OpenTofu scripts and Helm charts, administrators can create clusters with minimal manual intervention, ensuring consistency and reducing operational overhead.
1.1 - Prerequisites
Ensure that the following prerequisites are met before deploying the PPC.
Updating the Roles and Permissions using JSON
The roles and permissions are updated using the JSONs.
From the AWS Console, navigate to IAM > Policies > Create policy > JSON, and create the following JSONs.
Note: Before using the provided JSON, replace the AWS_ACCOUNT_ID and REGION values with those of the account and region where the resources are being deployed.
A newly provisioned EC2 instance must be used as the jump box for PPC deployment.
The jump box must be provisioned in the same AWS region where the PPC cluster will be deployed to ensure proper connectivity and access to regional resources.
Supported operating systems: RHEL 10, Debian 13, or Debian 12.
Note: Ensure to retain the original jump box used for installation. The Terraform state files required for cleaning and upgrading the cluster are available on this jump box. A snapshot of the jump box must be maintained to prevent data loss.
AWS Account Details
A valid AWS account where Amazon EKS is deployed.
The AWS account ID and AWS region must be identified in advance, as all resources are provisioned in the selected region.
Service Quotas
Verify that the AWS account has sufficient service quotas to support the deployment. At a minimum, ensure adequate limits for the following:
EC2 instances based on node group size and instance types.
VPC and networking limits, including subnets, route tables, and security groups.
Elastic IP addresses and Load balancers.
If required, request quota increases through the AWS Service Quotas console before proceeding.
Service Control Policies (SCPs)
The AWS account must not have SCPs that restrict required permissions. In particular, SCPs must not block the following actions:
eks:*
ec2:*
iam:PassRole
Restrictive SCPs may prevent successful cluster creation and resource provisioning.
Virtual Private Cloud (VPC)
An existing VPC must be available in the target AWS region.
The VPC should be configured to support Amazon EKS workloads.
Subnet Requirements
At least two private subnets must be available.
Subnets must be distributed across two or more Availability Zones (AZs).
Creating AWS KMS Key and AWS S3 Bucket
Amazon S3 Bucket: An Amazon S3 bucket is required to store critical data such as backups, configuration artifacts, and restore metadata used during installation and recovery workflows. Using a dedicated AWS S3 bucket helps ensure data durability, isolation, and controlled access during cluster operations.
AWS KMS Key: An AWS KMS customer‑managed key is required to encrypt data stored in the AWS S3 bucket. This ensures that sensitive data is protected at rest and allows customers to manage encryption policies, key rotation, and access control in accordance with their security requirements.
Note: The KMS key must allow access to the IAM roles used by the EKS cluster and related services.
The following section explains how to create AWS KMS Key and AWS S3 Bucket. This can be done from the AWS Web UI or using the script.
Create a KMS key for backup bucket. The KMS key created is referenced during installation and restore using its KMS ARN, and is validated by the installer.
Before you begin, ensure to have:
Access to the AWS account where the KMS key is created.
The KMS key can be in the same AWS account as the AWS S3 bucket, or in a different, cross‑account AWS account.
The user running the installer must have the permission kms:DescribeKey to describe the KMS key. If this permission is unavailable, then installation and restore fails.
The steps to create a KMS key are available at https://docs.aws.amazon.com/. Follow the KMS key creation steps, but ensure to select the following configurations.
On the Key configuration page: Select Key type as Symmetric. Select Key usage as Encrypt and decrypt.
These settings are required for encrypting and decrypting AWS S3 objects used by backup and restore operations.
On the Key Administrative Permissions page, select the users or roles that can manage the key. The key administrators do not automatically get permission to encrypt or decrypt data, unless these permissions are explicitly granted.
On the Define key usage permissions page, grant permissions to the principals that will use the key.
The user or role running the installation and restore must have the permission kms:DescribeKey to describe the key. This permission is mandatory because the installer validates the KMS key before proceeding. Without this, the installation or restore procedure fails, especially in cross‑account KMS scenarios.
On the Edit key policy - optional page, click Edit.
The KMS key policy controls the access to the encryption key and must be applied before creating the AWS S3 bucket.
If you are using AWS SSO IAM Identity Center, ensure that the IAM role ARN specified in the KMS key policy includes the full SSO path prefix: aws-reserved/sso.amazonaws.com/. For example: arn:aws:iam::<ACCOUNT_ID>:role/aws-reserved/sso.amazonaws.com/<SSO_ROLE_NAME> Omitting this path results in KMS key policy creation failures with an InvalidArnException.
The following example shows a key policy that:
Allows the PPC bootstrap user to use the KMS key for cluster creation and backup operations.
Allows the IAM role to encrypt and decrypt EKS backups.
Update the values of the following based on the environment:
DEPLOYMENT_AWS_ACCOUNT - AWS account ID.
CLUSTER_NAME - EKS cluster name.
SSO_OR_IAM_USER_ACCOUNT_ARN - ARN of the IAM role used to run the bootstrap script. The ARN format depends on your authentication method:
IAM role – Use the ARN returned by aws sts get-caller-identity.
AWS SSO (IAM Identity Center) – Convert the session ARN returned by aws sts get-caller-identity to a full IAM role ARN before using it in the KMS key policy.
If you are using AWS SSO (IAM Identity Center), the ARN returned by aws sts get-caller-identity is a session ARN and cannot be used directly in an AWS KMS key policy. AWS KMS requires the full IAM role ARN, including the aws-reserved/sso.amazonaws.com/ path. Without this, KMS key policy creation fails with InvalidArnException.
Retrieving the IAM role ARN for KMS key policy
To identify the role used to run the bootstrap script, run the following command:
aws sts get-caller-identity --query Arn --output text
Replace assumed-role/ with role/aws-reserved/sso.amazonaws.com/.
Remove the session suffix (everything after the last /).
Important: Before initiating restore operation, review and update the KMS key policy to reflect the restore CLUSTER_NAME. Even if the policy was already configured for the source cluster, it must be updated for the new restore cluster. If the policy continues to reference the source cluster name, the IAM role created during restore cannot decrypt the backup data, causing the restore to fail.
After the KMS key is created, note the KMS key ARN. This KMS key ARN is required while creating the AWS S3 backup bucket.
Create an AWS S3 Bucket encrypted with SSE‑KMS The AWS S3 bucket encrypted with SSE‑KMS is used as a backup bucket during installation and restore.
Before you begin, ensure to have:
Access to the AWS account where the AWS S3 bucket will be created.
Permission to create AWS S3 bucket.
The user running the installer must have permission to describe the KMS key. Without this permission, installation and restore fails.
The steps to create an AWS S3 bucket are available at https://docs.aws.amazon.com/. Follow the AWS S3 bucket creation steps, but ensure to set the following configurations as mentioned below.
In the Default Encryption section:
Select Encryption type as Server-side encryption with AWS Key Management Service keys (SSE-KMS).
Select the AWS KMS key ARN. If the KMS key is in a different AWS account than the AWS S3 bucket, then the key will not appear in the AWS console dropdown. In this case, enter the KMS key ARN manually.
Enable Bucket Key.
Automatic AWS KMS Key and S3 Bucket Provisioning
When deploying PPC using the bootstrap script, the AWS S3 bucket and KMS key can be provisioned automatically during the installation process, without any separate pre-installation steps.
During the bootstrap script execution, at step 5, you are prompted to choose how to provide the AWS S3 bucket for cluster backups:
Do you want to use an existing AWS S3 bucket, or provision a new one (with KMS key)?
1) Use existing
2) Provision new via Tofu (backup-infra)
Select Option 2 - Provision new via Tofu to have the bootstrap script automatically:
Create a new AWS KMS key.
Create a new AWS S3 bucket encrypted with SSE-KMS.
Associate the AWS S3 bucket with the KMS key.
At step 6, when prompted, provide a globally unique bucket name using the following naming rules:
Lowercase letters, digits, and hyphens only.
Between 3 and 63 characters.
Must not already exist.
Note: The AWS S3 bucket and KMS key are created in the same AWS account using this option. For cross-account KMS configurations, follow the steps in the Using AWS Web UI tab.
Use a dedicated AWS S3 bucket per cluster for backup and restore operations to ensure data and encryption isolation. Sharing a bucket across clusters increases the risk of cross-cluster data access or decryption due to IAM misconfiguration.
Blocking Public Access to the AWS S3 Bucket
The PPC bootstrap script does not configure AWS S3 Block Public Access at the per-bucket level. This is by design because many AWS Organizations enforce AWS S3 Block Public Access through Service Control Policies (SCPs) at the account or organization level. This causes per-bucket s3:PutBucketPublicAccessBlock calls to fail with AccessDenied. AWS enabled S3 Block Public Access by default for all new accounts created after April 2023. However, verify the settings manually before proceeding.
Warning: The AWS S3 backup bucket stores highly sensitive data. Ensure that AWS S3 Block Public Access is enforced at the AWS account or organization level to prevent accidental public exposure.
Verify that AWS S3 Block Public Access is enabled at the account level before deploying PPC:
Navigate to the Amazon AWS S3 console.
Click Account and organization settings from the left navigation.
Ensure all four settings are enabled and set to On:
Block public access to buckets and objects granted through new access control lists (ACLs)
Block public access to buckets and objects granted through any access control lists (ACLs)
Block public access to buckets and objects granted through new public bucket or access point policies
Block public and cross-account access to buckets and objects through any public bucket or access point policies
If it is set to Off, then click Edit, select the check boxes, and click Save changes.
Verify that the Block Public Access settings for this account are enabled using the following command:
All four fields BlockPublicAcls, IgnorePublicAcls, BlockPublicPolicy, RestrictPublicBuckets should be set to true.
Note: To block the permissions at the bucket level, navigate to Buckets > General purpose buckets > select the bucket > Permissions > Block public access (bucket settings).
Zone Requirements
The cluster requires a minimum of two availability zones for high availability. It is required to have two zones available while deploying a PPC.
Zones
Supported
Notes
1
No
Installation fails.
2
Yes
Minimum requirement for High Availability.
Zones are derived from the private subnets which are provided. Each subnet maps to exactly one availability zone. During the installation, the bootstrap script validates that the selected subnets are in different Availability Zones.
Note: Zones cannot be changed after cluster creation.
1.2 - Preparing for PPC deployment
Downloading and Extracting the Protegrity Cluster Template (PCT) for Deploying PPC
If the jump box is already set previously, run the ./bootstrap.sh --cloud aws destroy command in the directory where the cluster is installed. This ensures that the local repository on the jump box and the clusters are cleaned up before proceeding with a new installation.
Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform/OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior. Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as ./bootstrap.sh --cloud aws destroy.
Navigate to Product Management > Explore Products > AI Team Edition > Platform & Features.
Navigate to Platform Installation.
From the Actions column for Protegrity Provisioned Cluster, click the Download Product icon to download the PPC 1.1 archive.
Create a deployment directory on the jumpbox.
mkdir deployment && cd deployment
Copy the PCT to the deployment directory on the jump box.
Extract the PCT.
tar -xvf PPC-K8S-ALL_1.1.0.97.tar
1.3 - Deploying PPC
Steps to deploy the PPC cluster
By default, the PCT is configured to deploy a FIPS-enabled cluster using FIPS-compliant Bottlerocket AMIs.
Non-FIPS deployment: Before proceeding with the installation, complete the steps in Configuring PPC for Non-FIPS Deployment on AWS to modify the required Terraform configuration files. Then proceed with either the automated or component-based installation below.
FIPS deployment: No additional configuration is required. Proceed directly with either the automated or component-based installation below.
The PPC cluster can be deployed using one of the following methods:
Automated script-based installation: This method requires the bootstrap.sh script to deploy the PPC cluster. The script installs the required software and dependencies, prompts to provide the required information, and deploys the cluster automatically.
Manual component-based installation: This method requires to perform the deployment of each component to install the PPC cluster. The configuration files are edited to provide the required information and then deploy the cluster.
1.3.1 - Configuring PPC for Non-FIPS Deployment on AWS
Complete the steps provided in this section to configure the Protegrity Cluster Template (PCT) for a non-FIPS PPC deployment on AWS EKS.
Before you begin
By default, the PCT is configured to deploy a FIPS-enabled cluster using FIPS-compliant Bottlerocket AMIs. To deploy a standard, non-FIPS cluster, the following Terraform configuration files must be modified to use the standard (non-FIPS) Bottlerocket AMI types and SSM parameter paths before running the bootstrap script or the component-based installation.
For a non-FIPS PPC deployment on AWS EKS, perform the following steps:
Update the node group AMI type map in the /03_node_group/variables.tf file.
The node_group_ami_type_map variable defines the fallback EKS AMI type used per node architecture when node_group_ami_type is not explicitly set. The FIPS-enabled defaults must be replaced with the corresponding standard Bottlerocket AMI types.
a. Navigate to /03_node_group directory.
cd iac/aws/cluster/modules/03_node_group/
b. Edit the variables.tf file to make the following changes.
Change from (FIPS defaults):
variable"node_group_ami_type_map" {
description="Fallback EKS AMI type per architecture, used when node_group_ami_type is empty." type=map(string) default= {
amd64="BOTTLEROCKET_x86_64_FIPS" arm64="BOTTLEROCKET_ARM_64_FIPS" }
}
Change to (non-FIPS values):
variable"node_group_ami_type_map" {
description="Fallback EKS AMI type per architecture, used when node_group_ami_type is empty." type=map(string) default= {
amd64="BOTTLEROCKET_x86_64" arm64="BOTTLEROCKET_ARM_64" }
}
Update the Karpenter AMI SSM parameter path in the /05_helm_releases/main.tf file.
The karpenter_fips_ami_ssm_parameter value specifies the AWS Systems Manager (SSM) parameter path that Karpenter uses to resolve the correct Bottlerocket AMI for worker nodes. The default path targets the FIPS AMI. For a non-FIPS cluster, remove the -fips segment from the EKS version portion of the path.
a. Navigate to /05_helm_releases directory.
cd iac/aws/cluster/modules/05_helm_releases/
b. Edit the main.tf file to make the following changes.
Complete the steps provided in this section to deploy PPC in AWS EKS.
Note for non-FIPS deployments: If you completed the steps in Configuring PPC for Non-FIPS Deployment on AWS, then the required Terraform configuration files have already been updated with non-FIPS values. The bootstrap script reads these files directly. Do not modify 03_node_group/variables.tf or 05_helm_releases/main.tf files back to FIPS defaults. Reverting those changes will result in a FIPS cluster being deployed instead of a non-FIPS cluster.
Before you begin
Before running the bootstrap or resiliency scripts as the root user on RHEL, ensure that /usr/local/bin, and the AWS CLI binary path, if applicable, is included in the $PATH. Alternatively, run the script using a non-root user, such as ec2-user, where /usr/local/bin is already part of the default PATH.
The repository provides a bootstrap script that automatically installs or updates the following software on the jump box:
AWS CLI - Required to communicate with the AWS account.
OpenTofu - Required to manage infrastructure as code.
kubectl - Required to communicate with the Kubernetes cluster.
The bootstrap script also checks if you have the required permissions on AWS. It then sets up the EKS cluster and installs the microservices required for deploying the PPC.
Architecture Selection
The step [1b] Select Node Architecture in the bootstrap script prompts to select the node architecture interactively during deployment.
amd64 (x86-64-based environments such as standard Intel or AMD servers)
arm64 (ARM-based environments such as AWS Graviton)
Deploying the PPC
The bootstrap script prompts for variables to be set to complete your deployment. Follow the instructions on the screen:
./bootstrap.sh --cloud aws --arch amd64
The script prompts for the following variables.
Enter Cluster Name
The following characters are allowed:
Lowercase letters: a-z
Numbers: 0-9
Hyphens: -
The following characters are not allowed:
Uppercase letters: A-Z
Underscores: _
Spaces
Any special characters such as: / ? * + % ! @ # $ ^ & ( ) = [ ] { } : ; , .
Leading or trailing hyphens
Names longer than 31 characters (the cluster name must be 31 characters or fewer)
Note: Ensure that the cluster name does not exceed 31 characters. Cluster names longer than this limit can cause the bootstrap script to fail in subsequent installation steps. If the installation fails, specify a cluster name that is 31 characters or fewer and re-run the script. The script automatically applies the updated value and continues the bootstrap process.
After the cluster name is accepted, the script prompts you to select the node architecture.
Note: Select amd64 for standard Intel/AMD 64-bit environments. Select arm64 for ARM-based environments such as AWS Graviton instances.
Enter a VPC ID from the table
The script automatically retrieves the available VPCs. Enter the VPC ID where the cluster must be created.
Querying for subnets in VPC…
The script queries for the available VPC subnets and prompts to enter two private subnet IDs. Specify two private subnet IDs from different availability zones. The script automatically updates the VPC CIDR block based on the VPC details.
Enter FQDN
This is the Fully Qualified Domain Name (FQDN) for the ingress.
Warning: Ensure that the FQDN does not exceed 50 characters and only the following characters are used:
Do you want to use an existing S3 bucket, or provision a new one (with KMS key)?
1) Use existing
2) Provision new via Tofu (backup-infra)
Enter S3 bucket name
Option 1 – Use existing: Enter the name of an existing S3 bucket in your AWS account. The bucket must already be encrypted with SSE‑KMS.
Option 2 – Provision new via Tofu: The script uses OpenTofu (backup-infra) to create the S3 bucket and the corresponding KMS key on your behalf. Provide a globally unique bucket name when prompted:
Lowercase letters, digits, and hyphens only
Between 3 and 63 characters
Must not already exist
Use a dedicated S3 bucket per cluster for backup and restore operations to ensure data and encryption isolation. Sharing a bucket across clusters increases the risk of cross-cluster data access or decryption due to IAM misconfiguration. Dedicated buckets with unique IAM policies eliminate this risk.
During disaster management, OpenSearch restores only those snapshots that are created using the daily-insight-snapshots policy. For more information, refer to Backing up and restoring indexes.
Enter Image Registry Endpoint
The image repository from where the container images are retrieved. Use registry.protegrity.com:9443 for using the Protegrity Container Registry (PCR), else use the local repository endpoint for the local repository.
Expected format: <hostname>[:port].Do not include ‘https://’
Note: The container registry endpoint must be a Fully Qualified Domain Name (FQDN). Sub-paths like, my-registry.com/v2/path, are not supported by the Open Container Initiative (OCI) distribution specification.
Enter Registry Username
Enter the username for the registry mentioned in the previous step. Leave this entry blank if the registry does not require authentication.
Enter Registry Password or Access Token
Enter Password or Access Token for the registry.
Input is masked with * characters. Press Enter to keep the current value.
Leave this entry blank if the registry does not require authentication.
Enter Owner Email (for cluster tagging)
Enter email address in the format name@domain.tld. This email is used for the cluster ‘Owner’ tag and the platform owner contact.
Note: The cluster creation process can take 10-15 minutes.
If the session is terminated during installation due to network issues, power outage, and so on, then the installation stops. To restart the installation, run the script again:
# Navigate to setup directory ./bootstrap.sh --cloud aws
Important Information
Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform/OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior.
Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as ./bootstrap.sh --cloud aws destroy. To check the active kubectl context, run the following command: kubectl config current-context
Do not delete the original cluster deployment directory, as it contains the Terraform state files required for managing the deployment.
1.3.3 - Manual component-based installation
Complete the steps provided in this section to update the configuration files and deploy PPC in AWS EKS.
Before you begin
Ensure to have the following prerequisites before proceeding with the installation.
Jump box requirement
Ensure that the entire process is performed using the same jump box.
Installs all required CLI tools (OpenTofu, AWS CLI, kubectl, Helm, and others) on the jump box, either automatically via bootstrap.sh install-tools or manually.
Confirms all cluster nodes are in Ready state and all pods are in Running state.
1. Exporting AWS credentials
The installation process uses Terraform to provision and manage AWS resources, such as VPCs, EKS clusters, IAM roles, and S3 buckets. The AWS credentials must be exported as environment variables to authenticate these operations.
Note: While using temporary credentials, such as, using AWS STS, ensure the session token is included and that the credentials have not expired before proceeding.
If the components are installed by different users, ensure that each user must export their AWS credentials. This ensures that all the required access and permissions are available.
To export the AWS credentials, run the following commands.
The following tools and minimum versions are required for the component-based installation. Ensure they are installed on your jump box before proceeding, or use the bootstrap.sh install-tools command to install them automatically.
After downloading and extracting the PPC-K8S-ALL_1.1.0.97.tar, run the following command to install the required tools.
./bootstrap.sh install-tools --cloud aws
Alternatively, the following tools must be installed and available on the jump box before starting.
Tools Name
Description
Updated Version
OpenTofu
Required to manage infrastructure as code (IaC) and provision cloud resources.
1.11.5
AWS CLI
Required to communicate with AWS and manage AWS resources.
2.36.7
kubectl
Required to communicate with and manage Kubernetes clusters.
1.36.1
Helm
Required to deploy and manage Kubernetes packages and applications.
3.21.0
skopeo
Required to inspect, copy, and transfer container images between registries.
1.18.0
jq
Required to parse, filter, and process JSON data.
1.7
bc
Required for version comparisons and resource calculations.
1.07.1
ssh-keygen
Required to generate and manage SSH keys for secure access to deployment environments.
OpenSSH_10.0p2 Debian-7+deb13u4
oras
Required to pull OCI artifacts, including non-container deployment packages, from registries.
1.3.3
cmctl
Required to validate, manage, and troubleshoot cert-manager TLS certificates.
2.5.0
3. Provisioning the S3 bucket
This step is required if a new S3 bucket is to be provisioned for storing the backup data. If an existing S3 bucket is to be used, note the bucket name and KMS key ARN required for the installation process and skip to the next step.
a. To navigate to 00_backup_bucket directory, run the following command.
cd iac/aws/cluster/modules/00_backup_bucket
b. Edit the terraform.tfvars file to provide the following information.
c. To apply the terraform configuration, run the following command.
tofu init
tofu plan # review the role assignments that will be created
tofu apply # type "yes" when prompted
After the S3 bucket is provisioned successfully, note the bucket name and KMS key ARN. These values are required during the installation process.
d. To sync the Terraform state file to the S3 bucket, run the following command.
# Navigate to the bootstrap-scripts directory
cd <package-root>/bootstrap-scripts
# Run the script to sync the backup data to the S3 bucket
./sync-backup-bucket-state-to-s3.sh
4. Modifying and applying terraform.tfvars files
The iac/aws/cluster/modules/ directory contains the Terraform modules for the PPC installation. To enter details in the terraform.tfvars file, navigate to the appropriate module directory and edit the file accordingly.
After modifying the terraform.tfvars file, apply the changes
01_iam_roles module
a. To navigate to 01_iam_roles directory, run the following command.
cd iac/aws/cluster/modules/01_iam_roles
b. Edit the terraform.tfvars file to provide the following information.
c. To apply the terraform configuration, run the following command.
tofu init
tofu plan # review the role assignments that will be created
tofu apply # type "yes" when prompted
d. After the configurations are successfully applied, verify the required resources are created.To verify if the resources are created successfully, run the following command.
aws iam list-roles --query "Roles[?contains(RoleName, 'doc-test')].[RoleName, Arn]" --output table
The EKS cluster IAM role, Node group IAM role, and Service account IAM roles must be created.
Additionally, note the role ARNs from the Terraform output. These values are required in subsequent modules.
02_eks_cluster module
a. To navigate to 02_eks_cluster directory, run the following command.
cd iac/aws/cluster/modules/02_eks_cluster
b. Edit the terraform.tfvars file to provide the following information.
Ensure that the security group name must not start with sg.
c. To apply the terraform configuration, run the following command.
tofu init
tofu plan # review the role assignments that will be created
tofu apply # type "yes" when prompted
d. After the configurations are successfully applied, verify the required resources are created.To verify if the resources are created successfully, run the following command.
# Confirm the cluster is ACTIVEaws eks describe-cluster --name <cluster-name> --region <region> \
--query "cluster.{Name:name, Status:status, Version:version, Endpoint:endpoint}"\
--output table
# Verify kubectl can reach the API serverkubectl get nodes # expected: no nodes yet (node group is Stage 03)# Verify the worker security group existsaws ec2 describe-security-groups \
--filters "Name=group-name,Values=<security-group-name>*"\
--region <region> \
--query "SecurityGroups[].{Name:GroupName, ID:GroupId}"\
--output table
# Verify VPC CNI and kube-proxy add-ons are ACTIVEaws eks list-addons --cluster-name <cluster-name> --region <region> --output table
03_node_group module
a. To navigate to 03_node_group directory, run the following command.
cd iac/aws/cluster/modules/03_node_group
b. Edit the terraform.tfvars file to provide the following information.
Note: Use the private subnet IDs from the VPC where the EKS cluster is provisioned while installing the 02_eks_cluster module.
Note for non-FIPS deployments: If you completed the steps in Configuring PPC for Non-FIPS Deployment on AWS, the node_group_ami_type_map default values in variables.tf file have already been changed to BOTTLEROCKET_x86_64 (amd64) or BOTTLEROCKET_ARM_64 (arm64). Do not add or override node_group_ami_type with a FIPS value in terraform.tfvars, as this will reintroduce FIPS AMIs.
c. To apply the terraform configuration, run the following command.
tofu init
tofu plan # review the role assignments that will be created
tofu apply # type "yes" when prompted
d. After the configurations are successfully applied, verify the required resources are created.To verify if the resources are created successfully, run the following command.
Ensure that the Node group status is ACTIVE and the desired number of nodes are in Ready state.
04_eks_addons module
a. To navigate to 04_eks_addons directory, run the following command.
cd iac/aws/cluster/modules/04_eks_addons
b. Edit the terraform.tfvars file to provide the following information.
aws_region = "<region>"
eks_cluster_name = "<cluster-name>"
backup_bucket = "<s3-bucket-name>" #
# --- Upgrade mode only ---
# Set to "~/.kube/config" when upgrading an existing cluster;
# leave empty ("") for a fresh install.
kubeconfig_path = ""
c. To apply the terraform configuration, run the following command.
tofu init
tofu plan # review the role assignments that will be created
tofu apply # type "yes" when prompted
Note: This stage waits for all nodes from Stage 03 to reach Ready status before installing add-ons.
d. After the configurations are successfully applied, verify the required resources are created.To verify if the resources are created successfully, run the following command.
# List all EKS add-ons and confirm they are ACTIVEaws eks list-addons --cluster-name <cluster-name> --region <region> --output table
# Check add-on health detailsfor addon in coredns metrics-server aws-ebs-csi-driver snapshot-controller eks-pod-identity-agent;doecho"--- $addon ---" aws eks describe-addon --cluster-name <cluster-name> --addon-name $addon --region <region> \
--query "addon.{Name:addonName, Status:status, Version:addonVersion}" --output table
done# Verify the default storage class existskubectl get storageclass
# Confirm CoreDNS and Metrics Server pods are runningkubectl get pods -n kube-system -l k8s-app=kube-dns
kubectl get pods -n kube-system -l k8s-app=metrics-server
05_helm_releases module
a. To navigate to 05_helm_releases directory, run the following command.
cd iac/aws/cluster/modules/05_helm_releases
b. Edit the terraform.tfvars file to provide the following information.
The default node OS is BOTTLEROCKET_x86_64_FIPS. While providing the node_os value, ensure that the value must be the same as nnode_group_ami_type value obtained from the 03_node_group module.
For obtaining the value of node_os, run the following command.
kubectl get nodes -o jsonpath='{.items[0].status.nodeInfo.osImage}'; echo
Provide the node_os value in the terraform.tfvars file.
d. To apply the terraform configuration, run the following command.
tofu init
tofu plan # review the role assignments that will be created
tofu apply # type "yes" when prompted
Note: This process may take 15–20 minutes to get completed. Karpenter nodes are provisioned on demand after the NodePool is created.
e. After the configurations are successfully applied, verify the required resources are created.To verify if the resources are created successfully, run the following command.
# Confirm all Helm releases are deployedhelm list -A
# Verify Karpenter controller is runningkubectl get pods -n karpenter
# Verify Karpenter NodePool and EC2NodeClass existkubectl get nodepools
kubectl get ec2nodeclasses
# Verify Velero is runningkubectl get pods -n pty-backup-recovery
# Verify PPC platform podskubectl get pods -A
# Verify Insight podskubectl get pods -n pty-insight
# Verify Envoy Gatewaykubectl get pods -n api-gateway
# Check all pods across the cluster are healthykubectl get pods -A --field-selector=status.phase!=Running,status.phase!=Succeeded
5. Verifying the installation
To verify the installation, run the following command.
kubectl get nodes
kubectl get pods -A
All the nodes should be in the Ready state and all the pods should be in the Running state.
1.4 - Installing Features and Protectors
Installing the features and protectors
Before you begin
Ensure that PPC is successfully installed before installing the features or protectors.
The Protegrity Provisioned Cluster (PPC) CLI provides a command-line interface for managing policies, users, and log forwarding. Access is established over SSH using the private key generated during the PPC installation.
1.5.1 - Prerequisites
The deployment includes a CLI container that provides command-line access to the Protegrity Management CLI using SSH, on both Linux and Windows.
To access the PPC CLI, ensure that the following prerequisites are met.
SSH Keys:
The SSH private key that corresponds to the public key configured in the pty-cli pod is required.
Network Access:
Ensure to have network connectivity to the cluster.
Resolve FQDN:
Use Route 53 configuration on AWS to resolve the PPC FQDN specified during the installation to the internal load balancer.
For more information, refer to Prerequisites.
The private key to access the CLI pod will be in the .ssh directory.
Under Connection > SSH > Auth, browse and select your private key file (.ppk format)
Username: ptyitusr
1.5.2 - Accessing the PPC CLI
Once connected, the Protegrity CLI welcome banner is displayed. Enter the following parameters when prompted:
Username: Application username
Password: Application password
For more information about the default credentials, refer to the Release Notes.
The CLI supports three main command categories:
pim: Policy Information Management commands for data protection policies.
admin: User, Role, Permission, and Group management commands.
insight: Log forwarding to external SIEM and syslog servers.
Note: Ensure that at least one additional backup administrator user is configured with the same administrative privileges as the primary admin user. If the primary admin account is locked or its credentials are lost, restoring the system from a backup is the only recovery option.
1.6 - Login to PPC
The PPC provides a web-based user interface for managing data protection policies, users, and roles. Access the UI through a browser using the cluster’s FQDN. This section covers the prerequisites for resolving the FQDN and the steps to log in.
1.6.1 - Prerequisites
Before accessing the Web UI, configure AWS Route 53 to resolve the PPC Fully Qualified Domain Name (FQDN) to the internal load balancer. This section lists the network and DNS requirements you need to verify, and the steps to map the FQDN to the load balancer.
Use Route 53 configuration on AWS to resolve the PPC FQDN specified during the installation to the internal load balancer.
Ensure that the instance is using the AWS-provided DNS server, such as, VPC CIDR + 2.
Verify that enableDnsHostnames and enableDnsSupport are set to true in the VPC settings.
Verify the Security Group of the load balancer. Ensure that Inbound traffic is allowed on the required ports, such as, 80 and 443, from the client instance’s IP or Security Group.
Keep the following information ready:
VPC ID: The ID of the VPC for the client instances and the Load Balancer. For example, vpc-0123456789.
Internal ELB DNS Name: The DNS name of the load balancer. For example, internal-abcdefghi123456-123456789.us-east-1.amazonaws.com.
Target FQDN: The FQDN for PPC. For example, mysite.aws.com.
Find the AWS Load Balancer address.
kubectl get gateway -A
The output appears similar to the following:
NAMESPACE NAME CLASS ADDRESS PROGRAMMED AGE
api-gateway pty-main envoy a814fae40464d462da9ca685921699c0-1ccfcf98919adf90.elb.us-east-1.amazonaws.com True 34m
Verify that the gateway address resolves to an internal IP, using the following command.
ping <gateway_address>
Where <gateway_address> is the ADDRESS value from the previous step.
The output confirms that the hostname resolves to a private VPC IP address (10.x.x.x range), indicating the load balancer is internal and accessible only within the AWS VPC.
The output appears similar to the following:
PING a814fae40464d462da9ca685921699c0-1ccfcf98919adf90.elb.us-east-1.amazonaws.com (10.31.4.172) 56(84) bytes of data.
Map the PPC FQDN to the load balancer using Route 53.
For more information about configuring Route 53, refer to the AWS documentation.
1.6.2 - Log in to PPC
After completing the prerequisites, log in to the web-based user interface for managing data protection policies, users, and roles. This page describes the steps to access the Web UI using the Fully Qualified Domain Name (FQDN) provided during the installation process.
Access the PPC using the FQDN provided during the installation process.
Enter the following parameters to log in and view the Insight Dashboard.
Username: Application username
Password: Application password
For more information about the default credentials, refer to the Release Notes.
Complete the steps provided in this section to upgrade a PPC to v1.1.0.
Before you begin
Before starting an upgrade, ensure the following conditions are met:
Access to the jump box associated with the PPC v1.0.0 cluster is required. This must be the same jump box used during the PPC v1.0.0 installation.
Maintain both, previous and current installation directories. Do not delete the original cluster deployment directory, as it contains the Terraform state files required for managing the deployment.
Create a Velero backup and an OpenSearch snapshot to ensure data recovery is possible, in case of an upgrade failure.
Maintain a backup of the state file in the AWS S3 backup bucket.
If logs are forwarded to an external SIEM, then from the PPC CLI, run the insight delete syslog or the insight delete fluentd as per your configuration. After the upgrade is complete, re-configure the SIEM. For more information about SIEM configuration, refer to Sending logs to a SIEM.
The context is set to the cluster where the upgrade operation is to be performed. Run the following command.
kubectl config current-context
The cluster name in the output must match the cluster installed with PPC v1.0.0.
Note: Before running the bootstrap or resiliency scripts as the root user on RHEL, ensure that /usr/local/bin, and the AWS CLI binary path, if applicable, is included in the $PATH. Alternatively, run the script using a non-root user, such as ec2-user where /usr/local/bin is already part of the default $PATH.
Upgrading PPC
To upgrade the PPC from v1.0.0 to v1.1.0, perform the following steps:
Create a deployment_110 directory on the jump box and navigate to it.
Navigate to Product Management > Explore Products > AI Team Edition > Platform & Features.
Navigate to Platform Installation.
From the Actions column for Protegrity Provisioned Cluster, click the Download Product icon and download the PPC 1.1 archive to the deployment_110 directory.
Extract the archive contents.
tar -xvf PPC-K8S-ALL_1.1.0.97.tar
Copy the Terraform state file from the v1.0.0 cluster installation to the deployment_110 directory created for upgrade.
where <v1.0-cluster-dir> is the path to the original PPC v1.0.0 cluster installation directory on the jump box, and <v1.1.0-cluster-dir> is the path to the new directory created for upgrade.
Verify the active context points to the correct cluster using the following command.
kubectl config current-context
Initiate the upgrade from the deployment_110 directory using the following command.
If logs were forwarded to an external SIEM before the upgrade, then SIEM log forwarding was disabled as part of the pre-upgrade prerequisites. After completing the upgrade, update the settings to resume sending logs to the external SIEM.
During upgrade, both the previous and the new PPC installation directories must be retained.
The original directory contains Terraform state and deployment metadata.
The new directory contains the updated configuration and binaries.
Both directories are required for proper management of the deployment.
Recovering from a Failed Upgrade
If the session is terminated during upgrade due to network issues, power outage, or any such issues, then the upgrade process stops. The ./upgrade.sh script resumes the upgrade.
To restart the process, run the following command:
Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform or OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior. Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as ./bootstrap.sh --cloud aws destroy. To check the active kubectl context, run the following command: kubectl config current-context
Reverting to a previous version of PPC
To revert to a previous version of PPC, restore the cluster using the steps in Restoring the PPC.
1.8 - Backing up the PPC
Complete the steps provided in this section to backup a PPC deployment.
Protegrity Provisioned Cluster (PPC) supports backup of Insight indexes and cluster configurations. Use the procedures below to update the scheduled backups of your PPC deployment.
Backing up Velero backup schedule
To take a backup at an interval of 10 minutes, the cron job must be executed before the schedule. This ensures that the latest configurations are included and are available in the Storage Account.By default, the cron job is executed after every 3 hours and schedule is executed after every 3.5 hours.
For taking the backups, the frequency of schedule and cron job can be changed, as required.
Perform the following steps to update the frequency.
To update the frequency of the cron job, run the following command.
Insight indexes are backed up automatically using a scheduled job. To manually back up the indexes, perform the following steps:
Log in to the Insight Dashboard.
Select the main menu.
Navigate to Management > Snapshot Management > Snapshot policies.
Click the daily-insight-snapshots policy.
Click Edit.
Update the following parameters:
Snapshot schedule
Frequency: Custom (Cron expression)
Cron expression: */15 * * * *
Click Update.
Wait for the next scheduled snapshot to complete. Verify that the snapshot is created from the snapshot status.
Note: The snapshot schedule is set to run every 15 minutes for creating the manual backup. Restore the default settings after the manual backup is complete. Before upgrading, disable the scheduler job after the backup is created. Ensure that the default parameters are restored and the job is re-enabled after the upgrade.
1.9 - Restoring the PPC
Complete the steps provided in this section to restore a PPC deployment using an existing backup.
Before you begin
Before starting a restore, ensure the following conditions are met:
An existing backup is available. Backups are taken automatically as part of the default installation using scheduled backup mechanisms. These backups are stored in an AWS S3 bucket configured during the original installation.
Access to the original backup AWS S3 bucket. During restore, the same S3 bucket that was used during the original installation must be specified.
Before initiating the restore, review and update the KMS key policy to reflect the restore cluster name.
Even if the policy was already configured for the source cluster, it must be updated for the new restore cluster. If the policy continues to reference the source cluster name, the IAM role created during restore cannot decrypt the backup data, causing the restore to fail.
Permissions to read from the S3 bucket. The user performing the restore must have sufficient permissions to access the backup data stored in the bucket.
A Kubernetes cluster is created. Restore is performed as part of creating a cluster, not on an existing one. Restore is only supported during a fresh installation flow.
While the backup is taken from the source cluster, do not perform Create, Read, Update, or Delete (CRUD) operations on the source cluster. This ensures backup consistency and prevents data corruption during restore.
Before restoring to a new cluster, if the source cluster is accessible, disable the backup operations on the source cluster by setting the backup storage location to read‑only. This ensures that no additional backup data is written during the restore process.
To disable the backup operation on the source cluster, run the following command:
If the source cluster is not accessible, this step can be skipped.
During Disaster management, the backup data is used to restore the cluster and the OpenSearch indexes using snapshots. However, Insight restores OpenSearch data only from the most recent snapshot created by the daily-insight-snapshots policy. For more information, refer to Backing up and restoring indexes.
Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform or OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior. Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as ./bootstrap.sh --cloud aws destroy.
The repository provides a bootstrap script that automatically installs or updates the following software on the jump box:
AWS CLI - Required to communicate with your AWS account.
bc - Required for version comparisons and resource calculations.
cmctl - Required to validate and manage TLS certificates.
kubectl - Required to communicate with the Kubernetes cluster.
OpenTofu - Required to manage infrastructure as code.
ORAS - Required to pull OCI artifacts such as deployment packages.
skopeo - Required to copy container images between registries.
ssh-keygen - Required to secure SSH access to deployment environments.
The bootstrap script also checks if you have the required permissions on AWS. It then sets up the EKS cluster and installs the microservices required for deploying the PCS on a PPC.
Note: Before running the bootstrap or resiliency scripts as the root user on RHEL, ensure that /usr/local/bin (and the AWS CLI binary path, if applicable) is included in the $PATH. Alternatively, run the script using a non-root user (such as ec2-user) where /usr/local/bin is already part of the default PATH.
Architecture Selection
amd64 (x86-64-based environments such as standard Intel or AMD servers)
arm64 (ARM-based environments such as AWS Graviton)
The bootstrap script prompts you to select the node architecture interactively during deployment (step 1b).
Restoring the PPC
Ensure that the PCT is downloaded and extracted on the jump box.
For more information, refer to Preparing for PPC deployment.
Run the following command to initiate restore using an existing backup:
./bootstrap.sh --cloud aws --restore
To deploy PPC on ARM-based infrastructure, specify the architecture explicitly:
The bootstrap script asks for variables to be set to complete the deployment. Follow the instructions on the screen.
The --restore command enables the restore mode for the installation. It initiates restoration of data from the configured AWS S3 backup bucket. This process must be followed on a fresh jump box.
The script prompts for the following variables.
Enter Cluster Name
Ensure that the cluster name does not match the name of the source cluster. Reusing an existing cluster name during restore can lead to discrepancies during cluster installation.
This same cluster name must already be updated in the KMS key policy. If this update is not performed, the restore process fails because the new cluster cannot decrypt the backup data.
Ensure that the cluster name does not exceed 31 characters. Cluster names longer than this limit can cause the bootstrap script to fail in subsequent installation steps. If the installation fails because the cluster name exceeds the 31-character limit, correct the name and re-run the script.
Correction: Choose a cluster name with 31 characters or fewer.
Retry: Execute the installation command again with the updated name. The script will automatically handle the update and proceed with the bootstrap process.
The following characters are allowed:
Lowercase letters: a-z
Numbers: 0-9
Hyphens: -
The following characters are not allowed:
Uppercase letters: A-Z
Underscores: _
Spaces
Any special characters such as: / ? * + % ! @ # $ ^ & ( ) = [ ] { } : ; , .
Leading or trailing hyphens
More than 31 characters
After the cluster name is accepted, the script prompts you to select the node architecture.
Note: Select amd64 for standard Intel/AMD 64-bit environments. Select arm64 for ARM-based environments such as AWS Graviton instances.
Enter a VPC ID from the table
The script automatically retrieves the available VPCs. Enter the VPC ID where the cluster must be created.
Querying for subnets in VPC…
The script queries for the available VPC subnets and prompts to enter two private subnet IDs. Specify two private subnet IDs from different availability zones. The script then automatically updates the VPC CIDR block based on the VPC details.
Enter FQDN
This is the Fully Qualified Domain Name (FQDN) for the ingress.
While entering the FQDN, ensure that:
contains at least two labels separated by a dot, such as, app.example.com
does not start or end with a dot or hyphen
does not contain consecutive dot character
label length must be between 1–63 characters
each label must start and end with a letter or digit
the last label must start only with a letter
Warning: Ensure that the FQDN does not exceed 50 characters and only the following characters are used:
Lowercase letters: a-z
Numbers: 0-9
Special characters: - .
Restore mode requires an existing S3 bucket.
Proceeding to existing bucket validation in Step 6.
Use a dedicated S3 bucket per cluster for backup and restore operations to ensure data and encryption isolation. Sharing a bucket across clusters increases the risk of cross-cluster data access or decryption due to IAM misconfiguration. Dedicated buckets with unique IAM policies eliminate this risk.
Select backup
During restore, the script prompts to manually select a backup from the available backups stored in the S3 bucket. User input is required to either restore from the latest backup or choose a specific backup from the list.
If n is selected, then the script displays a list of available backups (latest first) and prompts to select one by number:
Available backups (latest first):
[1] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx
[2] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx
Select a backup number (1-2):
After entering the backup number, the chosen backup is used for the restore, and the installation continues.
Enter Insight backup
During restore, the script prompts to manually select the required insight backup from the available backups stored in the S3 bucket. User input is required to either restore from the latest backup or choose a specific backup from the list.
If n is selected, then the script displays a list of available backups (latest first) and prompts to select one by number:
Available backups (latest first):
[1] <timestamp> insight-configmaps-secrets-xxxxxxxxx
[2] <timestamp> insight-configmaps-secrets-xxxxxxxxx
Select a backup number (1-2):
After entering the backup number, the chosen backup is used for the restore, and the installation continues.
Enter Image Registry Endpoint
The image repository from where the container images are retrieved.
Expected format: <hostname>[:port].Do not include ‘https://’
Note: The container registry endpoint must be a FQDN (Fully Qualified Domain Name). Sub-paths like, my-registry.com/v2/path, are not supported by the OCI distribution specification.
Enter Registry Username
Enter the username for the registry mentioned in the previous step. Leave this entry blank if the registry does not require authentication.
Enter Registry Password or Access Token
Enter Password or Access Token for the registry.
Input is masked with * characters. Press Enter to keep the current value.
Leave this entry blank if the registry does not require authentication.
Enter Owner Email (for cluster tagging)
Enter email address in the format name@domain.tld. This email is used for the cluster ‘Owner’ tag and the platform owner contact.
Note: The cluster creation process can take approximately 15-20 minutes.
If the session is terminated during restore due to network issues, power outage, and so on, then the restore stops. To restart the process, run the following commands:
# Navigate to deployment directory and run the following command./bootstrap.sh --cloud aws destroy
# Restore the cluster./bootstrap.sh --cloud aws --restore
Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform or OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior. Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as ./bootstrap.sh --cloud aws destroy. To check the active kubectl context, run the following command: kubectl config current-context
After the restore to the new cluster is completed successfully and all required validation and migration activities are finished, the source cluster can be deleted.
1.10 - Deleting PPC
Steps to delete the cluster
Prerequisites
Authentication
The deletion process uses OpenTofu to destroy AWS resources, such as EKS clusters, IAM roles, S3 buckets, and KMS keys. The AWS credentials must be exported as environment variables to authenticate these operations.
Note: While using temporary credentials, such as, using AWS STS, ensure the session token is included and that the credentials have not expired before proceeding.
To export the AWS credentials, run the following commands.
To verify that the credentials are valid and the correct AWS identity is being used, run the following command.
aws sts get-caller-identity
Required Permissions
The same IAM principal (or one with equivalent permissions) that installed the cluster, can delete the cluster. The principal must have cluster-creator admin access to the EKS Kubernetes API.
The required IAM Permissions are listed in the following table:
The following tools must be installed and available on the machine before proceeding with deletion:
tofu (OpenTofu)
kubectl (configured with cluster context)
aws CLI
helm
Cleaning up the EKS Resources
There are two ways to destroy all created resources, including the EKS cluster and related components.
Method 1: Bootstrap script
Method 2: OpenTofu modules
Approach
Automated, single command
Manual, module-by-module
Control
Fully automated; no per-stage visibility
Full control and visibility over each stage
Best for
Standard teardowns where speed is preferred
Troubleshooting, partial cleanup, or auditing each deletion step
Effort
Minimal
Higher; requires navigating each module directory
Method 1: Using the bootstrap script
From the folder where bootstrap script is present, run the following command:
./bootstrap.sh --cloud aws destroy
Method 2: Using OpenTofu to delete all modules
This method displays how to delete a PPC cluster on AWS by destroying the OpenTofu modules in reverse order. Destroying modules one at a time gives full control and visibility over each stage.
Warning: This process is irreversible. All cluster resources, data, and backups stored in the S3 bucket will be permanently deleted.
These modules must be destroyed in reverse order: 05 → 04 → 03 → 02 → 01 → 00.
To delete the modules, perform the following steps:
To delete the helm releases resource, perform the following steps:
# Navigate to 05_helm_releases directorycd /iac/aws/cluster/modules/05_helm_releases
# Initialize providerstofu init
# Delete the helm releasetofu destroy # type "yes" when prompted
To delete the EKS Addons resource, run the following command.
# Navigate to 04_eks_addons directorycd /iac/aws/cluster/modules/04_eks_addons
# Initialize providerstofu init
# Delete the eks addonstofu destroy # type "yes" when prompted
To delete the Node Group resource, run the following command.
# Navigate to 03_node_group directorycd /iac/aws/cluster/modules/03_node_group
# Initialize providerstofu init
# Delete the node grouptofu destroy # type "yes" when prompted
To delete the EKS Cluster resource, run the following command.
# Navigate to 02_eks_cluster directorycd /iac/aws/cluster/modules/02_eks_cluster
# Initialize providerstofu init
# Delete the eks clustertofu destroy # type "yes" when prompted
To delete the IAM Roles resource, run the following command.
# Navigate to 01_iam_roles directorycd /iac/aws/cluster/modules/01_iam_roles
# Initialize providerstofu init
# Delete the iam rolestofu destroy # type "yes" when prompted
To delete the Backup Bucket (S3 + KMS) resource, perform the following steps.
# 1.Navigate to 00_backup_bucket directorycd /iac/aws/cluster/modules/00_backup_bucket
# 2. Download the state file from S3 (synced there during install)aws s3 cp \
s3://<bucket-name>/infrastructure-manifest/<cluster-name>/states/00_backup_bucket.tfstate \
./terraform.tfstate
# 3. Initialize providerstofu init
# 4. Enable force_destroy on the bucket (updates only the flag)tofu apply -var="force_destroy_bucket=true"
# 5. Destroy all resourcestofu destroy -var="force_destroy_bucket=true"# type "yes" when prompted
Note: After destroying all the modules, delete the CloudWatch Logs group created by EKS control plane logging using the following command.
Verify the access to the cluster, using the following command.
kubectl get nodes
1.12 - Restoring the SSH keys and PCT from backed up Terraform state file
Complete the steps provided in this section to restore the SSH keys and PCT from a backed up Terraform state file of a deleted v1.1.0 jump box.
This section describes the procedure to rebuild a usable PPC v1.1.0 deployment workspace on a jump box from an existing cluster and a backed up Terraform state file.
Before you Begin
Ensure the following requirements are met.
The state file from all the following modules must be available on the AWS S3 backup bucket.
00_backup_bucket
01_iam_roles
02_eks_cluster
03_node_group
04_eks_addons
05_helm_releases
Access to the same AWS account and target EKS cluster.
AWS CLI credentials with required permissions.
Perform the following steps to recover PPC
Create a workspace.
To create a working directory for PCT, run the following command.
mkdir -p deployment
cd deployment
Download and extract the PCT
Download the PCT package into the deployment directory and extract it.For more information about downloading and extracting the PCT, refer to Preparing for PPC deployment.
After successfully extracting the template, this directory behaves as recovered installation workspace.
Install required dev tools
Run the following script.
cdbootstrap-scripts/./setup-devtools-aws.sh
Set kubectl to the target EKS cluster
To set kubectl to the target EKS cluster, run the following command.
A jump box serves as a secure, controlled access point for executing installation commands and managing cluster resources. It ensures that all operations originate from a trusted environment with direct network access to the PPC cluster, avoiding connectivity issues that may arise from local machines with restricted network paths or inconsistent tooling.
Use a dedicated Debian or Red Hat jump box created in Microsoft Azure. Do not use a jump box hosted on any other cloud.
The jump box must be provisioned in the same Azure region where the PPC cluster will be deployed to ensure proper connectivity and access to regional resources.
For example, if the PPC cluster is deployed in eastus, the jump box must also be created in eastus. Placing the jump box in a different region, such as westeurope, may result in network access failures or latency issues when connecting to cluster endpoints.
Microsoft Azure Region
For automated installation
Identify the Azure region where the cluster is to be deployed.
The deployment region is automatically detected from the resource group’s location during the bootstrap process. Ensure that the resource group is created in the intended deployment region before running the installer.
Microsoft Azure Resource IDs from Infrastructure Team
Deployment Resource Group
Before proceeding with the installation, identify the Azure Resource Group where the cluster is to be deployed.
For automated deployment, the value of this resource group name must be specified during the input prompt while running the ./bootstrap.sh script.
For component-based deployment, the value of this resource group name must be specified in the resource_group_name field in the terraform.tfvars files, wherever required.
The following resources are provisioned within the resource group specified during deployment.
RBAC Role Assignments (references existing UAMIs; no new resources created)
02_AKS Cluster
AKS Cluster, Federated Credentials
03_Node Pool
AKS User Node Pool
04_AKS Addons
Storage Classes, Cluster Add-ons
05_Helm Releases
Kubernetes Namespaces, Secrets, Helm Charts
Pre-existing Resources
The following resources must be provisioned by the IT team before installation.
They can reside in any resource group in the same region. The installer references each resource by its entire Azure resource ID.
Ensure the deploying identity, such as, service principal or managed identity, has the appropriate permissions on these resources.
Resource
Description
Variable in terraform.tfvars
Example Resource ID
AKS User-Assigned Managed Identity
Identity used by the AKS cluster to interact with Azure resources such as Private DNS Zones and Virtual Networks.
uami_id
/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-applianceframework Must have Private DNS Zone and VNet custom role permissions. For more information, refer to AKS User-Assigned Managed Identity Permissions.
Velero User-Assigned Managed Identity
Identity used by Velero to access Azure Blob Storage for cluster backup and restore operations.
The AKS User-Assigned Managed Identity (id-aks-applianceframework) must be granted a custom role with the following permissions on the resource group containing the Private DNS Zone and Virtual Network. These permissions enable AKS to manage private DNS records and join virtual networks during cluster operations.
To configure User-Assigned Managed Identity with a custom Azure role containing the permissions required for AKS deployment and management operations within the specified resource group, perform the following steps:
1. Creating a Custom Role
In the Azure portal, navigate to Resource Groups.
Select the resource group that contains the AKS resources.
Open Access Control (IAM).
Select Add > Add custom role.
Provide a name and description for the custom role.
Review the permissions and create the custom role.
Note: These permissions are required when using a private AKS cluster with a pre-existing Private DNS Zone (private_dns_zone_id). Without them, cluster provisioning and DNS resolution will fail.
2. Assigning the custom role to the Managed Identity
On the Azure portal, navigate to Managed Identities.
Select Create.
Specify the following:
Subscription
Resource group
Managed identity name
Region
Select Review + Create and then Create.
Wait for the User-Assigned Managed Identity (UAMI) to be provisioned.
Open the newly created User-Assigned Managed Identity.
Select Azure role assignments.
Select Add role assignment.
Set the scope to the target resource group.
Select the custom role created in the previous section.
Assign the role to the User-Assigned Managed Identity.
Verify that the role assignment appears under the managed identity’s role assignments.
Zone Requirements
The cluster requires a minimum of two availability zones for high availability.
Three-zone cluster deployments are supported and recommended for production.
No. of Zones
Supported
Notes
1
No
Installation fails.
2
Yes
Minimum requirement for High Availability.
3
Yes
Recommended for production environments.
Ensure the following requirements are met before deploying:
Requirement
Detail
Region support
The target Azure region must support availability zones.
VM SKU availability
The chosen VM size must be available in all the specified zones.
Modules:02_aks_cluster, 03_node_pool
Note: The bootstrap script does not prompt for zone selection, it uses the default three zone configuration.
The default zone configuration is ["1", "2", "3"]. Not all Azure regions support three availability zones, and not all VM SKUs are available in every zone. Using a zone that does not support your VM SKU causes cluster creation to fail.
Important:system_node_pool_zones (module 02) and node_pool_zones (module 03) must use the same set of zones. If the zones differ, disks provisioned in one zone cannot be attached to nodes rescheduled into another zone.
Before deploying, verify zone and VM SKU availability:
# List availability zones for your regionaz account list-locations \
--query "[?name=='<region>'].{Region:name, Zones:availabilityZoneMappings}"\
-o table
# Check which zones support your VM SKUaz vm list-skus \
--location <region> \
--size Standard_D4ds_v4 \
--query "[].{Name:name, Zones:locationInfo[0].zones}"\
-o table
Replace <region> with your Azure region, for example eastus.
If zone adjustment is required, navigate to iac/azure/cluster/modules and set matching values in both terraform.tfvars files:
# 02_aks_cluster/terraform.tfvars
system_node_pool_zones=["1", "2"] # adjust to the zones your region supports
# 03_node_pool/terraform.tfvars
node_pool_zones=["1", "2"] # must match system_node_pool_zones exactly
The step [1b] Select Node Architecture in the bootstrap script prompts to select the node architecture interactively during deployment.
The default system node pool VM size (Standard_D2ds_v4, 2 vCPU / 8 GiB RAM) can become resource-constrained under production load when Karpenter, CoreDNS, and CSI drivers run concurrently on the same node. Use the following or larger for production system node pools.
amd64 (x86-64)
Node Pool
VM Size
vCPUs
Memory
System
Standard_D2as_v5
2
8 GB
User
Standard_D8as_v5
8
32 GB
arm64
Node Pool
VM Size
vCPUs
Memory
System
Standard_D2ps_v5
2
8 GB
User
Standard_D8ps_v5
8
32 GB
In 02_aks_cluster/terraform.tfvars, edit the following value:
system_node_pool_vm_size="Standard_D4ds_v5" # or Standard_D8ds_v4 for heavier loads
Downloading and extracting the Protegrity Cluster Template (PCT) for deploying PPC
The Protegrity Cluster Template (PCT) is a pre-packaged deployment archive that contains the infrastructure-as-code scripts, Helm charts, and bootstrap tooling required to provision a PPC on your target cloud platform. It automates the creation of the Kubernetes cluster, networking, identity configuration, and installation of Protegrity microservices. Download and extract the PCT on your jump box before starting the deployment process.
Note: If there is an existing cluster from a previous install, clean up your local repository on the jump box and any existing clusters by running ./bootstrap.sh --cloud azure destroy from /deployment before proceeding.
During installation, the system may prompt for the system password and require sign-in to Microsoft Azure. If the Azure CLI is not already logged in, the bootstrap script automatically runs az login --use-device-code. A device-code prompt similar to the following appears.
[YYYY-MM-DD HH:MM:SS] Azure CLI not logged in. Triggering az login...
To sign in, use a web browser to open the page https://login.microsoft.com/device and enter the code XXXXXXXX to authenticate.
To sign-in to Microsoft Azure, perform the following steps:
Open the displayed URL in a browser, and enter the code shown in the terminal.
Complete the SSO sign-in and follow the on-screen instructions.
After successful authentication, the script continues automatically.
To download and extract PCT, perform the following steps:
Navigate to Product Management > Explore Products > AI Team Edition > Platform & Features.
Navigate to Platform Installation.
From the Actions column for Protegrity Provisioned Cluster, click the Download Product icon to download the PPC 1.1 archive.
Create a deployment_110 directory on the jumpbox.
mkdir deployment_110 && cd deployment_110
Copy the PCT to the deployment_110 directory on the jump box.
Extract the PCT.
tar -xvf PPC-K8S-ALL_1.1.0.97.tar
2.3 - Deploying PPC
Steps to deploy the PPC cluster
By default, the PCT is configured to deploy a FIPS-enabled cluster using FIPS-compliant node OS images.
Non-FIPS deployment: Before proceeding with the installation, complete the steps in Configuring PPC for Non-FIPS Deployment on Microsoft Azure to modify the required Terraform configuration files. Then proceed with either the automated or component-based installation below.
FIPS deployment: No additional configuration is required. Proceed directly with either the automated or component-based installation below.
The PPC cluster can be deployed using one of the following methods:
Automated script-based installation: This method requires the bootstrap.sh script to deploy the PPC cluster. The script installs the required software and dependencies, prompts to provide the required information, and deploys the cluster automatically.
Manual component-based installation: This method requires to perform the deployment of each component to install the PPC cluster. The configuration files are edited to provide the required information and then deploy the cluster.
Note: A PPC can be deployed in regions where minimum two zones are present.
2.3.1 - Production configuration
Review and override these settings before deploying to production for high availability, data safety, and security compliance.
The default terraform.tfvars values are optimized for development and staging environments. Before deploying to production, override the settings below to ensure high availability, data safety, and security compliance.
Applying the changes
Script-based installation: After the bootstrap script generates the terraform.tfvars files, edit them before the script runs tofu apply, or modify and re-apply individually using OpenTofu after installation.
By default, the backup storage account uses Locally Redundant Storage (LRS), which stores data only within a single datacenter. When AKS nodes span multiple availability zones, an LRS-backed disk cannot be reattached to a node in a
different zone during failover, causing pod scheduling to fail.
Recommendations:
Use ZRS (Zone-Redundant Storage) when nodes are spread across availability zones in the same region.
Use GRS (Geo-Redundant Storage) if cross-region disaster recovery is required.
Use Premium_ZRS as the storage class for Kubernetes persistent volumes.
In 00_backup_storage/terraform.tfvars:
storage_account_replication_type="ZRS" # use "GRS"forcross-regionredundancy
System node pool sizing
Module:02_aks_cluster
The default system node pool VM size (Standard_D2ds_v4, 2 vCPU / 8 GiB RAM) can become resource-constrained under production load when Karpenter, CoreDNS, and CSI drivers run concurrently on the same node.
Recommendation: Use Standard_D4ds_v4 (4 vCPU / 16 GiB) or larger for production system node pools.
In 02_aks_cluster/terraform.tfvars, edit the following value:
system_node_pool_vm_size="Standard_D4ds_v4" # or Standard_D8ds_v4 for heavier loads
Key Vault SKU
Module:00_backup_storage
The default Key Vault SKU is standard, which uses software-protected keys. For organizations with compliance requirements (such as FIPS 140-2 Level 2 or Level 3), the premium SKU provides HSM-backed encryption keys.
Recommendation: Use premium when your security or compliance policy requires HSM-backed key protection.
In 00_backup_storage/terraform.tfvars:
key_vault_sku="premium"
Note: Confirm with your compliance team whether HSM-backed keys are required before changing this setting. The premium SKU incurs additional Azure costs.
Availability zones
Modules:02_aks_cluster, 03_node_pool
The default zone configuration is ["1", "2", "3"]. Not all Azure regions support three availability zones, and not all VM SKUs are available in every zone. Using a zone that does not support your VM SKU causes cluster creation to
fail.
Important:system_node_pool_zones (module 02) and node_pool_zones (module 03) must use the same set of zones. If the zones differ, disks provisioned in one zone cannot be attached to nodes rescheduled into another
zone.
Before deploying, verify zone and VM SKU availability:
# List availability zones for your regionaz account list-locations \
--query "[?name=='<region>'].{Region:name, Zones:availabilityZoneMappings}"\
-o table
# Check which zones support your VM SKUaz vm list-skus \
--location <region> \
--size Standard_D4ds_v4 \
--query "[].{Name:name, Zones:locationInfo[0].zones}"\
-o table
Replace <region> with your Azure region, for example eastus.
In both 02_aks_cluster/terraform.tfvars and 03_node_pool/terraform.tfvars, set matching values:
# 02_aks_cluster/terraform.tfvars
system_node_pool_zones=["1", "2"] # adjust to the zones your region supports
# 03_node_pool/terraform.tfvars
node_pool_zones=["1", "2"] # must match system_node_pool_zones exactly
AKS cluster tier (sku_tier)
Module:02_aks_cluster
The sku_tier setting controls the AKS API server uptime SLA, available node capacity, and supported features.
Tier
Uptime SLA
Notes
Free
No SLA
Suitable for development and testing only
Standard
99.9% (99.95% with Availability Zones)
Recommended for production
Premium
99.9% (99.95% with Availability Zones) + LTS
Required if long-term support is needed
Recommendation: Set sku_tier = "Standard" for all production deployments. Use "Premium" if your organization requires long-term support (LTS) guarantees.
In 02_aks_cluster/terraform.tfvars:
sku_tier="Standard"
Note:Standard and Premium tiers incur an additional hourly cost per cluster. See the AKS pricing page for current rates.
Storage account network access
Module:00_backup_storage
The backup storage account is created with public network access enabled by default. In production, restrict access using your organization’s existing network controls.
Azure provides three options:
Method
Isolation level
Typical use
Private Endpoints
Assigns the storage account a private IP inside your VNet. Removes it from public internet.
Recommended for most enterprise deployments.
VNet service endpoints
Routes traffic through the VNet without a private IP.
Lower cost alternative to Private Endpoints.
Azure Firewall / NSG
Network-layer traffic filtering.
Supplement to either option above.
Apply the appropriate controls after deployment. The example below uses the service endpoint approach:
# Allow access from a specific VNet subnetaz storage account network-rule add \
--account-name <storage-account-name> \
--resource-group <resource-group> \
--vnet-name <vnet-name> \
--subnet <subnet-name>
# Deny all other public accessaz storage account update \
--name <storage-account-name> \
--resource-group <resource-group> \
--default-action Deny
Note: Consult your organization’s security team to select the appropriate access control method before deploying to production.
Setting
Default
Production recommendation
Module
storage_account_replication_type
LRS
ZRS or GRS
00_backup_storage
system_node_pool_vm_size
Standard_D2ds_v4
Standard_D4ds_v4 or larger
02_aks_cluster
key_vault_sku
standard
premium (if HSM required)
00_backup_storage
system_node_pool_zones
["1","2","3"]
Verify per region/SKU; match node_pool_zones
02_aks_cluster
node_pool_zones
["1","2","3"]
Must match system_node_pool_zones
03_node_pool
sku_tier
Free
Standard or Premium
02_aks_cluster
Storage account network access
Public (default)
Private Endpoints, VNet service endpoints, or Firewall
Post-deploy
2.3.2 - Configuring PPC for Non-FIPS Deployment on Microsoft Azure
Complete the steps provided in this section to configure the Protegrity Cluster Template (PCT) for a non-FIPS PPC deployment on Azure AKS.
Before you begin
By default, the Azure PPC installation enables FIPS (Federal Information Processing Standards). Follow these steps only if you need to install a non-FIPS cluster. For the standard FIPS-enabled installation, refer to Installing PPC on Microsoft Azure.
Note: The Script-based and Component-based installation instructions reference default FIPS-enabled settings. When following either installation method, do not revert or overwrite the values configured on this page. Keep fips_enabled set to false in 02_aks_cluster and 03_node_pool.
2.3.3 - Automated script-based installation
Complete the steps provided in this section to deploy PPC in Microsoft Azure AKS.
Before you begin
Ensure that all the required resources are in the same resource-group. If the resources are in different resource groups, then ensure to move the resources to the same group before proceeding for PPC deployment.
The repository provides a bootstrap script that automatically installs or updates the following tools on the jump box:
Azure CLI - Required to communicate with your Microsoft Azure account.
kubectl - Required to communicate with the Kubernetes cluster.
OpenTofu - Required to manage infrastructure as code.
oras: Required to pull non‑container, generic OCI artifacts from the registry that are not handled by standard container tooling.
Note: If you are performing a non-FIPS deployment, ensure that you have already completed the steps in the section Configuring PPC for Non-FIPS Deployment on Microsoft Azure before proceeding. Do not revert fips_enabled to true or change node_os back to a FIPS-compliant image during the steps below.
Architecture Selection
The step [1b] Select Node Architecture in the bootstrap script prompts to select the node architecture interactively during deployment.
amd64 (x86-64)
Node Pool
VM Size
vCPUs
Memory
System
Standard_D2as_v5
2
8 GB
User
Standard_D8as_v5
8
32 GB
arm64
Node Pool
VM Size
vCPUs
Memory
System
Standard_D2ps_v5
2
8 GB
User
Standard_D8ps_v5
8
32 GB
The bootstrap script prompts you to select the node architecture interactively during deployment (step 1b).
Deploying the PPC
The bootstrap script prompts for variables to be set to complete the deployment. Run the following command and follow the instructions on the screen.
./bootstrap.sh --cloud azure
The script prompts for the following variables.
Enter AKS Cluster Name
The following characters are allowed:
Lowercase letters: a-z
Numbers: 0-9
Hyphens: -
The following characters are not allowed:
Uppercase letters: A-Z
Underscores: _
Spaces
Any special characters such as: / ? * + % ! @ # $ ^ & ( ) = [ ] { } : ; , .
Leading or trailing hyphens
Names longer than 10 characters (the cluster name must be 10 characters or fewer)
Note: Ensure that the cluster name does not exceed 10 characters. Cluster names longer than this limit can cause the bootstrap script to fail in subsequent installation steps. If the installation fails because the cluster name exceeds the 10-character limit, correct the name and re-run the script.
Correction: Choose a cluster name with 31 characters or fewer.
Retry: Execute the installation command again with the updated name. The script will automatically handle the update and proceed with the bootstrap process.
After the cluster name is accepted, the script prompts you to select the node architecture.
[1b] Select Node Architecture
[1b] Select Node Architecture
1) amd64 (x86_64) — e.g. Standard_D8as_v5
2) arm64 (ARM/Ampere) — e.g. Standard_D8ps_v5
Enter 1 to select amd64 or 2 to select arm64.
Note: Select amd64 for standard Intel/AMD 64-bit environments. Select arm64 for ARM-based environments such as AWS Graviton instances.
Querying for available Resource Groups
The script queries for the available Resource Groups.
Enter the Resource Group name from the table []
The script then automatically detects the location and subscription ID of the resource group.
Enter UAMI Resource ID
Provide the complete Azure resource ID for the UAMI used by AKS in the following format:
The script attempts to automatically detect network settings:
Virtual network address space
Service CIDR
DNS service IP
If the detection fails, then default values configured in the terraform.tfvars file are used.
Enter FQDN [name.domain.com]
This is the Fully Qualified Domain Name for the ingress.
Warning: Ensure that the FQDN does not exceed 50 characters and only the following characters are used:
Lowercase letters: a-z
Numbers: 0-9
Special characters: - .
Storage Account and Key Vault provisioning
Choose whether to use existing resources or create new resources:
1) Use existing
2) Provision new via Tofu
Enter 1 if an encrypted Storage Account and Key Vault are already provisioned for this cluster. The installer prompts for the Storage Account name, Key Vault name, backup container, Key Vault key name, and the Velero UAMI Resource ID.
Enter 2 to allow the installer to create a new Storage Account and Key Vault with the velero container, the pty-backup-key encryption key, and a Velero UAMI automatically. Only the new resource names are required.
Enter Velero UAMI Resource ID (required)
Enter the resource ID of the dedicated Velero User-Assigned Managed Identity (UAMI) in the format:
The script automatically validates the UAMI and detects the UAMI Client ID.
Enter Image Registry Endpoint
The image repository from where the container images are retrieved.
Expected format: <hostname>[:port].
Do not include ‘https://’
Enter Registry Username
Enter the username for the registry mentioned in the previous step. Leave this entry blank if the registry does not require authentication.
Enter Registry Password or Access Token
Enter Password or Access Token for the registry.
Input is masked with * characters. Press Enter to keep the current value.
Leave this entry blank if the registry does not require authentication.
Enter Owner Email (for cluster tagging)
Enter email address in the format name@domain.tld. This email is used for the cluster ‘Owner’ tag and the platform owner contact.
Note: The cluster creation process can take 10-15 minutes.
If the session is terminated during installation due to network issues, power outage, and so on, then the installation stops. To restart the installation, run the script again:
# Navigate to setup directory ./bootstrap.sh --cloud azure
After the bootstrap script is completed, verify the cluster and workloads using the following commands:
# Confirm nodes are Readykubectl get nodes
# Confirm NFA workloads are Runningkubectl get pods -A
2.3.4 - Manual component-based installation
Complete the steps provided in this section to update the configuration files and deploy PPC in Microsoft Azure AKS.
This section is intended for enterprise customers who need full control over each stage of the AKS cluster deployment. Use this path when individual infrastructure stages must be reviewed or approved separately, when integration with an existing CI/CD pipeline is required, or when corporate change-management controls prevent running an all-in-one automated script. Each stage is self-contained and can be executed by different teams on the same jump box.
Before deploying to production: The default values are optimized for development and staging. Review the Production configuration section to override settings for high availability, data safety, and security compliance before running tofu apply.
Note: If you are performing a non-FIPS deployment, ensure you have already completed Configuring PPC for Non-FIPS Deployment on Microsoft Azure before proceeding. Do not revert fips_enabled to true when performing the following steps.
Deploying PPC using component-based installation
To install PPC, complete the following steps in order. Each step is described in detail in the sections below.
Installs all required CLI tools (OpenTofu, Azure CLI, kubectl, Helm, and others) on the jump box, either automatically via bootstrap.sh install-tools --cloud azure or manually.
Authenticates with Azure using az login and verifies access to the target subscription, resource group, managed identities, subnet, and private DNS zone.
(Skip if using existing resources) Provisions a new Azure Storage Account, Blob container, Key Vault, and encryption key for storing PPC backup data using the 00_backup_storage Terraform module.
Provisions the private AKS cluster with a system node pool, configures networking (Azure CNI Overlay and Cilium), enables Workload Identity, and creates federated identity credentials for Velero and OpenSearch.
Creates default and backup Kubernetes storage classes using Azure Managed Disks (Premium LRS) and removes the built-in default storage class annotation.
Confirms all cluster nodes are in Ready state and all pods are in Running state.
Before you begin
Ensure to have the following prerequisites before proceeding with the installation.
Access and permissions
Requirement
Details
Azure RBAC
Contributor and User Access Administrator on the target subscription or resource group.
Jump box
Debian or Red Hat VM in Azure with internet access to pull the release archive.
Required Roles
The Azure principal must have AKS cluster admin access.
The required Azure RBAC Roles are listed in the following table:
Role
Scope
Used by
Contributor
Resource group
Create AKS cluster, node pool, storage account, Key Vault
User Access Administrator
Resource group
Create role assignments (module 01)
Storage Blob Data Contributor
Backup storage account
Access state files in backend, upload module 00 state
Key Vault Administrator
Key Vault
Create Key Vault and keys (module 00)
AKS Cluster Admin
AKS cluster
Kubernetes API access for Helm, kubectl operations (modules 04, 05)
Deployment Resource Group
All resources created by the installer are provisioned in a single Azure resource group specified using resource_group_name in each terraform.tfvars file. This resource group must exist before running the installer.
The following resources are created inside this resource group.
Note: Skip this step if an Azure Storage Account, backup container, Key Vault, and key already exist for this cluster. Note these values. They will be needed in next steps.
Navigate to the 00_backup_storage directory.
cd iac/azure/cluster/modules/00_backup_storage
Open terraform.tfvars and fill in the required values.
To apply the terraform configuration, run the following command.
tofu init
tofu plan # review the role assignments that will be created tofu apply # type "yes" when prompted
Note the Storage Account, container, Key Vault, and key values for future steps.
4. Filling in each module’s terraform.tfvars
Each module under iac/azure/cluster/modules/ contains its own terraform.tfvars file. There is no single shared configuration file, so the terraform.tfvars file in every module must be updated before running the deployment.
Verify that the following values are the same across all modules:
resource_group_name
aks_cluster_name
azure_subscription_id
storage_account_name
storage_container_name
01_identity module
This stage creates RBAC role assignments for the pre-existing User-Assigned Managed Identities (UAMIs) required by AKS, Velero, and OpenSearch.
Resource
Purpose
Managed Identity Operator role
Allows AKS to assign the UAMI to nodes
Virtual Machine Contributor role
Allows AKS to manage VMs in the node resource group
Disk Snapshot Contributor role
Allows Velero to create and manage disk snapshots
Storage Blob Data Contributor (Velero)
Allows Velero to read/write backup blobs
Key Vault Crypto User role
Allows Velero to encrypt/decrypt backups with the Key Vault key
Storage Blob Data Contributor (OpenSearch)
Allows OpenSearch to store snapshot data in blob storage
To update the 01_identity module, perform the following steps.
a. Navigate to the iac/azure/cluster/modules/01_identity/ directory.
cd iac/azure/cluster/modules/01_identity/
b. Edit the terraform.tfvars file to provide the required values.
resource_group_name = "<resource-group>" # e.g. "aks-rg-prod"
aks_cluster_name = "<cluster-name>" # e.g. "prod-aks-cluster"
azure_subscription_id = "<subscription-id>" # e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"
storage_account_name = "<storage-account-name>" # e.g. "myclusterstateaccount"
storage_container_name = "velero"
# -------------------------------------------------
# Pre-existing managed identities (provided by IT)
# -------------------------------------------------
uami_id = "</subscriptions/<subscription-id>/resourceGroups/<it-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-applianceframework>"
velero_uami_id = "</subscriptions/<subscription-id>/resourceGroups/<velero-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-velero>" # required
opensearch_uami_id = "</subscriptions/<subscription-id>/resourceGroups/<opensearch-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/aks-opensearch-identity>" # required
# -------------------------------------------------
# Key Vault (leave empty to skip KV role assignment)
# -------------------------------------------------
key_vault_name = "<key-vault-name>" # e.g. "my-key-vault"
c. Save the file and exit the editor.
d. To initialise, plan, and apply the changes, run the following commands.
tofu init
tofu plan # review the role assignments that will be created tofu apply # type "yes" when prompted
e. To verify the resources created in this stage, run the following commands
For AKS UAMI:
# Retrieve the Principal ID of the AKS User-Assigned Managed Identity (UAMI)az identity show \
--ids "<uami-resource-id>"\
--query principalId \
--output tsv
#Use the returned Principal ID to verify the role assignments:
az role assignment list \
--assignee "<aks-uami-principal-id>" \
--output table
For Velero UAMI
# Retrieve the Principal ID of the Velero UAMIaz identity show \
--ids "<velero-uami-principal-id>"\
--query principalId \
--output tsv
#Use the returned Principal ID to verify the role assignments:
az role assignment list \
--assignee "<velero-uami-principal-id>" \
--output table
For OpenSearch UAMI
# Retrieve the Principal ID of the OpenSearch UAMIaz identity show \
--ids "<opensearch-uami-principal-id>"\
--query principalId \
--output tsv
#Use the returned Principal ID to verify the role assignments:
az role assignment list \
--assignee "<opensearch-uami-principal-id>" \
--output table
02_aks_cluster module
This stage provisions the private AKS cluster with a system node pool, configures networking (Azure CNI Overlay and Cilium), enables Workload Identity, and creates federated identity credentials for Velero and OpenSearch.
Resource
Purpose
AKS cluster
Private Kubernetes control plane with OIDC issuer and Workload Identity enabled
System node pool
2-node pool for core Kubernetes components (CoreDNS, metrics-server, etc.)
Federated credential (Velero)
Allows the Velero service account to authenticate as the Velero UAMI
Federated credential (OpenSearch)
Allows the OpenSearch service account to authenticate as the OpenSearch UAMI
Kubeconfig context
Auto-configures kubectl access to the new cluster
To update the 02_aks_cluster module, perform the following steps.
a. Navigate to the iac/azure/cluster/modules/02_aks_cluster/ directory.
b. Edit the terraform.tfvars file to provide the required values.
resource_group_name="<resource-group>"# e.g. "aks-rg-prod"aks_cluster_name="<cluster-name>"# e.g. "prod-aks-cluster"azure_subscription_id="<subscription-id>"# e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"storage_account_name="<storage-account-name>"# e.g. "myclusterstateaccount"storage_container_name="velero"# -------------------------------------------------# Identity (UAMI for kubelet and federated credentials)# -------------------------------------------------uami_id="</subscriptions/<subscription-id>/resourceGroups/<it-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-applianceframework>"# full UAMI resource ID (same as Stage 01)velero_uami_id="</subscriptions/<subscription-id>/resourceGroups/<velero-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-velero>"# full Velero UAMI resource ID (for federated credential)opensearch_uami_id="</subscriptions/<subscription-id>/resourceGroups/<opensearch-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/aks-opensearch-identity>"# full OpenSearch UAMI resource ID (for federated credential)# -------------------------------------------------# Cluster# -------------------------------------------------azure_location="<region>"# e.g. "eastus" (auto-detected from resource group)subnet_id="</subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>/subnets/<subnet>>"# e.g. "/subscriptions/.../subnets/snet-aks-applianceframework"private_dns_zone_id="</subscriptions/<subscription-id>/resourceGroups/<dns-resource-group>/providers/Microsoft.Network/privateDnsZones/privatelink.eastus.azmk8s.io>"# e.g. "/subscriptions/.../privateDnsZones/privatelink.eastus.azmk8s.io"# -------------------------------------------------# Networking (service CIDR + DNS IP)# -------------------------------------------------service_cidr="<10.0.0.0/16>"# auto-derived by updatevartf.sh (non-overlapping with the VNet)dns_service_ip="<10.220.0.10>"# auto-derived: the .10 host of the chosen service_cidrpod_cidr="<10.0.0.0/16>"# auto-derived: non-overlapping with the VNet and service_cidr# -------------------------------------------------# System node pool# -------------------------------------------------architecture="amd64"# amd64 or arm64system_node_pool_vm_size=""# leave empty to auto-select based on architecturesystem_node_pool_node_count=2system_node_pool_disk_size_gb=50system_node_pool_os_sku="AzureLinux"system_node_pool_zones=["1","2","3"]fips_enabled=true# set false for a non-FIPS node OS image (immutable)# -------------------------------------------------# Ownership / tagging# -------------------------------------------------owner_email="<name@domain.tld>"# optional; blank = auto-detect from the Azure CLI identity
c. Save the file and exit the editor.
d. To initialise, plan, and apply the changes, run the following commands.
tofu init
tofu plan # review the role assignments that will be created tofu apply # type "yes" when prompted
e. To verify the resources created in this stage, run the following commands.
# Confirm the AKS cluster is in Succeeded stateaz aks show --name <cluster-name> --resource-group <resource-group> \
--query "{Name:name, State:provisioningState, Version:kubernetesVersion, Fqdn:privateFqdn}"\
--output table
A sample output is displayed below:
Name State Version Fqdn
---------------- --------- --------- -----------------------------------------------
<cluster-name> Succeeded 1.35 <cluster-name>.privatelink.<region>.azmk8s.io
# Verify kubectl can reach the clusterkubectl get nodes # expect 2 system nodes in Ready state
az identity federated-credential list \
--identity-name <opensearch-uami-name> \
--resource-group <opensearch-uami-resource-group> --output table
03_node_pool module
This stage creates a user-managed node pool for application workloads. Node Auto Provisioning (NAP) handles autoscaling; no cluster autoscaler is configured on this pool.
Resource
Purpose
User node pool (userpool)
Worker nodes for NFA platform, Insight, and other application pods
To update the 03_node_pool module, perform the following steps.
a. Navigate to the iac/azure/cluster/modules/03_node_pool/ directory.
b. Edit the terraform.tfvars file to provide the required values.
resource_group_name = "<resource-group>" # e.g. "aks-rg-prod"
aks_cluster_name = "<cluster-name>" # e.g. "prod-aks-cluster"
azure_subscription_id = "<subscription-id>" # e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"
storage_account_name = "<storage-account-name>" # e.g. "myclusterstateaccount"
storage_container_name = "velero"
# -------------------------------------------------
# Node pool
# -------------------------------------------------
subnet_id = "</subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>/subnets/<subnet>>" # same subnet as AKS nodes (Stage 02)
architecture = "amd64" # amd64 or arm64
vm_size = "" # leave empty for architecture default
node_pool_disk_size_gb = 120
node_pool_desired_size = 3
node_pool_zones = ["1", "2", "3"]
fips_enabled = true # set false for a non-FIPS node OS image (immutable)
c. Save the file and exit the editor.
d. To initialise, plan, and apply the changes, run the following commands.
tofu init
tofu plan # review the role assignments that will be created tofu apply # type "yes" when prompted
Expected time for this stage is approximately 5 to 10 minutes for node pool provisioning.
e. To verify the resources created in this stage, run the following commands.
# Confirm the user node pool exists and is Succeededaz aks nodepool show --cluster-name <cluster-name> \
--resource-group <resource-group> --name userpool \
--query "{Name:name, State:provisioningState, Count:count, VmSize:vmSize, OsSku:osSku}"\
--output table
A sample output is displayed below:
Name State Count VmSize OsSku
-------- --------- ------- ---------- ----------
userpool Succeeded <count> <vm-size> AzureLinux
# Verify all nodes are Ready (system + user nodes)kubectl get nodes -o wide
# Confirm user pool nodes have the expected labelkubectl get nodes -l agentpool=userpool
04_aks_addons module
This stage creates the default and backup Kubernetes storage classes using Azure Managed Disks (Premium LRS) and removes the built-in default storage class annotation.
Resource
Purpose
ebs-sc storage class
Default storage class (Premium LRS, WaitForFirstConsumer, expandable)
ebs-bkp storage class
Backup storage class used by Velero for PVC snapshots
Remove built-in default SC
Ensures only ebs-sc is marked as the cluster default
To update the 04_aks_addons module, perform the following steps.
a. Navigate to the iac/azure/cluster/modules/04_aks_addons/ directory.
cd iac/azure/cluster/modules/04_aks_addons/
b. Edit the terraform.tfvars file to provide the required values.
resource_group_name = "<resource-group>" # e.g. "aks-rg-prod"
aks_cluster_name = "<cluster-name>" # e.g. "prod-aks-cluster"
azure_subscription_id = "<subscription-id>" # e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"
storage_account_name = "<storage-account-name>" # e.g. "myclusterstateaccount"
storage_container_name = "velero"
c. Save the file and exit the editor.
d. To initialise, plan, and apply the changes, run the following commands.
tofu init
tofu plan # review the role assignments that will be created tofu apply # type "yes" when prompted
e. To verify the resources created in this stage, run the following commands.
# List storage classes and confirm ebs-sc is the defaultkubectl get storageclass
# Confirm the built-in 'default' storage class is no longer the defaultkubectl get storageclass default -o jsonpath='{.metadata.annotations.storageclass\.kubernetes\.io/is-default-class}'# expected output: empty or "false"
This stage deploys all application Helm charts, configures Velero backup with the Azure plugin, and bootstraps the NFA platform and Insight analytics stack.
Resource
Purpose
Velero Helm release
Backup and disaster recovery with Azure Blob Storage and disk snapshots
NFA / Eclipse Helm release
Main platform stack (cert-manager, Keycloak, API Gateway, services)
Insight Helm release
Log aggregation and analytics (OpenSearch, Dashboards, Fluentd)
Reloader Helm release
Auto-restarts workloads on ConfigMap or Secret changes
Envoy Gateway Helm release
API gateway and ingress proxy
VolumeSnapshotClass
CSI snapshot class (csi-azure-vsc) for Azure Disk snapshots
Namespaces & registry secrets
Creates namespaces with image-pull secrets for the private registry
To update the 05_helm_releases module, perform the following steps.
a. Navigate to the iac/azure/cluster/modules/05_helm_releases/ directory.
cd iac/azure/cluster/modules/05_helm_releases/
b. Run the following commands and note the values.
# Get the Velero Client IDaz identity show \
--name id-aks-velero \
--resource-group aks-ppc \
--query clientId \
-o tsv
# Get the OpenSearch Client IDaz identity show \
--name aks-ppc01-opensearch-identity \
--resource-group aks-ppc \
--query clientId \
-o tsv
c. Edit the terraform.tfvars file to provide the required values.
resource_group_name="<resource-group>"# e.g. "aks-rg-prod"aks_cluster_name="<cluster-name>"# e.g. "prod-aks-cluster"azure_subscription_id="<subscription-id>"# e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"storage_account_name="<storage-account-name>"# e.g. "myclusterstateaccount"storage_container_name="velero"# -------------------------------------------------# Networking# -------------------------------------------------subnet_id="</subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>/subnets/<subnet>>"# same subnet as Stages 02/03 (for VNet address space lookup)pod_cidr=""# optional override; empty = auto-detected from the live AKS network profilearchitecture="amd64"# amd64 or arm64node_os="AzureLinux_FIPS"# should mirror stage-03 user node pool os_sku + fips_enabled# -------------------------------------------------# Registry# -------------------------------------------------global_image_reg="<registry endpoint>"# e.g. "harbor.example.com"harbor_secret="<harbour secret>"# e.g. "harbor-pull-secret"ingress_fqdn="<FQDN>"# e.g. "apps.example.com"# -------------------------------------------------# Ownership / tagging# -------------------------------------------------owner_email="<name@domain.tld>"# optional; blank = auto-detect from the Azure CLI identity# -------------------------------------------------# Restore / backup# -------------------------------------------------restore=false# set to true to restore from a previous backupbackup_name="authnz-postgresql-schedule-backup"# Velero backup name used when restore = trueinsight_backup_name=""# Insight Velero backup name (K8s resources only); leave empty to auto-resolve latest# -------------------------------------------------# Workload Identity# -------------------------------------------------velero_uami_client_id="<velero_uami_client_id>"# client ID of the Velero UAMI opensearch_uami_client_id="<opensearch_uami_client_id>"# client ID of the OpenSearch UAMI
c. Save the file and exit the editor.
d. Export registry credentials using environment variables. Do not put them in the tfvars file.
The PPC provides a web-based user interface for managing data protection policies, users, and roles. Access the UI through a browser using the cluster’s FQDN. This section covers the prerequisites for resolving the FQDN and the steps to log in.
To access the Web UI, map the gateway hostname to the Microsoft Azure Load Balancer IP address in the local hosts file.
Get gateway details: Find the hostname and the Microsoft Azure Load Balancer address.
kubectl get gateway -A
The output will look similar to:
NAMESPACE NAME CLASS ADDRESS PROGRAMMED AGE
api-gateway pty-main envoy 10.221.8.33 True 5h7m
Update the hosts file: Add an entry mapping the ingress FQDN to the IP.
Linux:/etc/hosts
Windows:C:\Windows\System32\drivers\etc\hosts
Example entry:
10.221.8.33 <FQDN given during cluster installation>
Use the same FQDN provided during the installation process..
Access the UI in the browser.
URL:https://<user-provided-fqdn>
Enter the following parameters to log in and view the Insight Dashboard.
Username: Application username
Password: Application password
For more information about the default credentials, refer to the Release Notes.
2.6 - Accessing the PPC CLI
The Protegrity Provisioned Cluster (PPC) CLI provides a command-line interface for managing policies, users, and log forwarding. Access is established over SSH using the private key generated during the PPC installation.
The deployment includes a CLI container that provides command-line access to the Protegrity Management CLI using SSH, on both Linux and Windows.
Prerequisites
SSH key: The private key generated during bootstrap at ~/.ssh/<cluster_name>_user_svc, matches the public key configured in the pty-cli pod.
Network access: Ensure you have connectivity to the AKS cluster’s ingress Load Balancer.
Hosts file: Same as Web UI access. Map the ingress FQDN to the Load Balancer IP.
The private key is placed under ~/.ssh/<cluster_name>_user_svc after bootstrap completes, where <cluster_name> is the AKS cluster name provided during installation.
From the project root directory, run the following command:
Connection > SSH > Auth: Browse to your private key (.ppk format)
Username: ptyitusr
CLI usage
Once connected, the Protegrity CLI welcome banner is displayed. Enter the following parameters when prompted:
Username: Application username
Password: Application password
For more information about the default credentials, refer to the Release Notes.
The CLI supports three command categories:
pim: Policy Information Management commands for data protection policies.
admin: User, role, permission, group, and email management commands.
insight: Log forwarding to external SIEM and syslog servers.
Note: Ensure that at least one additional backup administrator user is configured with the same administrative privileges as the primary admin user. If the primary admin account is locked or its credentials are lost, restoring the system from a backup is the only recovery option.
2.7 - Deleting PPC
Steps to delete the cluster
Prerequisites
Authentication
The deletion process uses OpenTofu to destroy Azure resources, such as AKS clusters, managed identities, storage accounts, and Key Vaults. The Azure CLI must be authenticated to perform these operations.
Note: While using temporary credentials, such as service principal tokens or managed identity tokens, ensure that the credentials have not expired before proceeding.
To authenticate and set the correct subscription, run the following commands.
az login
az account set --subscription <subscription-id>
To verify that the credentials are valid and the correct Azure subscription is being used, run the following command.
az account show
Required Resource Providers
The following Azure resource providers must be registered in the subscription before deletion can proceed. If any provider is missing, the destroy operation may fail when attempting to manage the associated resources.
Microsoft.ContainerService
Microsoft.Storage
Microsoft.KeyVault
Microsoft.ManagedIdentity
Microsoft.Network
Microsoft.Compute
Required permissions
The following permissions are required to delete the PPC cluster. Create a custom role with these permissions and assign it to the managed identity used for deletion.
The following tools must be installed and available on the machine before proceeding with deletion:
tofu (OpenTofu)
kubectl (configured with cluster context)
az CLI (authenticated)
helm
Cleaning up the AKS Resources
There are two ways to destroy all created resources, including the AKS cluster and related components.
Method 1: Using the bootstrap script
From the folder where bootstrap script is present, run the following command:
./bootstrap.sh --cloud azure destroy
Method 2: Using OpenTofu to delete all modules
This method displays how to delete a PPC cluster on Azure by destroying the OpenTofu modules in reverse order. Destroying modules one at a time gives full control and visibility over each stage.
Warning: This process is irreversible. All cluster resources, data, and backups stored in the backup storage will be permanently deleted.
These modules must be destroyed in reverse order: 05 → 04 → 03 → 02 → 01 → 00.
To delete the modules, perform the following steps:
To delete the helm releases resource, run the following command.
# Navigate to 05_helm_releases directorycd /iac/azure/cluster/modules/05_helm_releases
# Initialize providerstofu init
# Delete the helm releasetofu destroy # type "yes" when prompted
To delete the AKS Addons resource, run the following command.
# Navigate to 04_aks_addons directorycd /iac/azure/cluster/modules/04_aks_addons
# Initialize providerstofu init
# Delete the aks addonstofu destroy # type "yes" when prompted
To delete the Node Pool resource, run the following command.
# Navigate to 03_node_pool directorycd /iac/azure/cluster/modules/03_node_pool
# Initialize providerstofu init
# Delete the node pooltofu destroy # type "yes" when prompted
To delete the AKS Cluster resource, run the following command.
# Navigate to 02_aks_cluster directorycd /iac/azure/cluster/modules/02_aks_cluster
# Initialize providerstofu init
# Delete the aks clustertofu destroy # type "yes" when prompted
To delete the Identity resource, run the following command.
# Navigate to 01_identity directorycd /iac/azure/cluster/modules/01_identity
# Initialize providerstofu init
# Delete the identitytofu destroy # type "yes" when prompted
To delete the Backup Storage (Backup storage account + container) resource, run the following command.
# Navigate to 00_backup_storage directorycd /iac/azure/cluster/modules/00_backup_storage
# Initialize providerstofu init
# Destroy all resources (Storage Account, Key Vault, container, encryption key)tofu destroy # type "yes" when prompted
2.8 - Backing up the PPC
Complete the steps provided in this section to backup a PPC deployment.
Protegrity Provisioned Cluster (PPC) supports backup of Insight indexes and cluster configurations. Use the procedures below to update the scheduled backups of your PPC deployment.
Backing up Velero backup schedule
To take a backup at an interval of 10 minutes, the cron job must be executed before the schedule. This ensures that the latest configurations are included and are available in the Storage Account.By default, the cron job is executed after every 3 hours and schedule is executed after every 3.5 hours.
For taking the backups, the frequency of schedule and cron job can be changed, as required.
Perform the following steps to update the frequency.
To update the frequency of the cron job, run the following command.
Insight indexes are backed up automatically using a scheduled job. To manually back up the indexes, complete the following steps:
Log in to the Insight Dashboard.
Select the main menu.
Navigate to Management > Snapshot Management > Snapshot policies.
Click the daily-insight-snapshots policy.
Click Edit.
Update the following parameters:
Snapshot schedule
Frequency: Custom (Cron expression)
Cron expression: */15 * * * *
Click Update.
Wait for the next scheduled snapshot to complete. Verify that the snapshot is created from the snapshot status.
Note: The snapshot schedule is set to run every 15 minutes for creating the manual backup. Restore the default settings after the manual backup is complete. Before upgrading, disable the scheduler job after the backup is created. Ensure that the default parameters are restored and the job is re-enabled after the upgrade.
2.9 - Restoring the PPC
Complete the steps provided in this section to restore a PPC deployment using an existing backup.
Before you begin
Before starting a restore, ensure the following conditions are met:
An existing backup is available. Backups are taken automatically as part of the default installation using scheduled backup mechanisms. These backups are stored in a Storage Account configured during the original installation.
Access to the original backup Storage Account. During restore, the same Storage Account that was used during the original installation must be specified.
Permissions to read from the Storage Account. The user performing the restore must have sufficient permissions to access the backup data stored in the account.
A new Kubernetes cluster is created. Restore is performed as part of creating a new cluster, not on an existing one. Restore is only supported during a fresh installation flow.
While the backup is taken from the source cluster, do not perform Create, Read, Update, or Delete (CRUD) operations on the source cluster. This ensures backup consistency and prevents data corruption during restore.
Before restoring to a new cluster, if the source cluster is accessible, disable the backup operations on the source cluster by setting the backup storage location to read‑only. This ensures that no additional backup data is written during the restore process.
To disable the backup operation on the source cluster, run the following command:
If the source cluster is not accessible, this step can be skipped.
During Disaster management, the backup data is used to restore the cluster and the OpenSearch indexes using snapshots. However, Insight restores OpenSearch data only from the most recent snapshot created by the daily-insight-snapshots policy. For more information, refer to Backing up and restoring indexes.
Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform/OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior. Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as ./bootstrap.sh --cloud azure destroy.
The repository provides a bootstrap script that automatically installs or updates the following software on the jump box:
Azure CLI - Required to communicate with your Microsoft Azure account.
OpenTofu - Required to manage infrastructure as code.
kubectl - Required to communicate with the Kubernetes cluster.
skopeo - Required to copy container images between registries.
bc - Required for version comparisons and resource calculations.
ORAS - Required to pull OCI artifacts such as deployment packages.
ssh-keygen - Required to secure SSH access to deployment environments.
cmctl - Required to validate and manage TLS certificates.
The bootstrap script also checks if you have the required permissions on Azure. It then sets up the AKS cluster and installs the microservices required for deploying the PCS on a PPC.
Note: Before running the bootstrap or resiliency scripts as the root user on RHEL, ensure that /usr/local/bin (and the Azure CLI binary path, if applicable) is included in the $PATH. Alternatively, run the script using a non-root user where /usr/local/bin is already part of the default PATH.
Architecture Selection
amd64 (x86-64-based environments such as Standard_D8as_v5)
arm64 (ARM-based environments such as Standard_D8ps_v5)
The bootstrap script prompts you to select the node architecture interactively during deployment (step 1b).
Restoring the PPC
Ensure that the PCT is downloaded and extracted on the jump box.
For more information, refer to Preparing for PPC deployment.
Run the following command to initiate restore using an existing backup:
./bootstrap.sh --cloud azure --restore
The script prompts for the following variables.
Enter AKS Cluster Name [opencluster-v1]
The following characters are allowed:
Lowercase letters: a-z
Numbers: 0-9
Hyphens: -
The following characters are not allowed:
Uppercase letters: A-Z
Underscores: _
Spaces
Any special characters such as: / ? * + % ! @ # $ ^ & ( ) = [ ] { } : ; , .
Leading or trailing hyphens
Names must only contain up to 10 characters
While creating a cluster, the cluster name must:
Start with a lowercase letter.
End with a letter or number.
Not contain consecutive hyphens.
Note: Ensure that the cluster name does not exceed 10 characters. Cluster names longer than this limit can cause the bootstrap script to fail in subsequent installation steps.
After the cluster name is accepted, the script prompts you to select the node architecture.
[1b] Select Node Architecture
Select the required architecture.
[1b] Select Node Architecture
1) amd64 (x86_64) — e.g. Standard_D8as_v5
2) arm64 (ARM/Ampere) — e.g. Standard_D8ps_v5
Enter 1 to select amd64 or 2 to select arm64.
Note: Select amd64 for Standard_D8as_v5 environments. Select arm64 for ARM-based environments such as Standard_D8ps_v5.
Querying for available Resource Groups
The script queries for the available Resource Groups.
Enter the Resource Group name from the table []
The script then automatically detects the location and subscription ID of the resource group.
Enter UAMI Resource ID
Provide the complete Azure resource ID for the UAMI used by AKS in the following format:
The script attempts to automatically detect network settings:
Virtual network address space
Service CIDR
Pod CIDR
DNS service IP
Enter FQDN [name.domain.com]
This is the Fully Qualified Domain Name for the ingress.
While entering the FQDN, ensure that:
contains at least two labels separated by a dot, such as, app.example.com
does not start or end with a dot or hyphen
does not contain consecutive dot character
label length must be between 1–63 characters
each label must start and end with a letter or digit
the last label must start only with a letter
Warning: Ensure that the FQDN does not exceed 253 characters and only the following characters are used:
Lowercase letters: a-z
Numbers: 0-9
Special characters: - .
Storage Account and Key Vault provisioning
During restore, the script prompts for existing Storage Account and Key Vault configuration. Provide the existing values in this step.
Storage Account & Key Vault provisioning
Restore mode requires existing Storage Account and Key Vault configuration.
Skipping provisioning choice; provide existing values in this step.
Enter existing Storage Account Name:
<bucket-name>
Storage account '<bucket-name>' found in resource group '<resource-group>'.
Using container: velero
Enter existing Key Vault Name:
<key-vault-name>
During restore, the script prompts to manually select a backup from the available backups stored in the Storage Account. User input is required to either restore from the latest backup or choose a specific backup from the list.
If n is selected, then the script displays a list of available backups (latest first) and prompts to select one by number:
Available backups (latest first):
[1] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx
[2] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx
Select a backup number (1-2):
After entering the backup number, the chosen backup is used for the restore, and the installation continues.
Enter Insight backup
During restore, the script prompts to manually select the required insight backup from the available backups stored in the Storage Account. User input is required to either restore from the latest backup or choose a specific backup from the list.
If n is selected, then the script displays a list of available backups (latest first) and prompts to select one by number:
Available backups (latest first):
[1] <timestamp> insight-configmaps-secrets-xxxxxxxxx
[2] <timestamp> insight-configmaps-secrets-xxxxxxxxx
Select a backup number (1-2):
After entering the backup number, the chosen backup is used for the restore, and the installation continues.
Enter Velero UAMI Resource ID
Enter the resource ID of the dedicated Velero User-Assigned Managed Identity (UAMI) in the format:
The script automatically validates the UAMI and detects the UAMI Client ID.
Enter Image Registry Endpoint
The image repository from where the container images are retrieved.
Expected format: <hostname>[:port].
Do not include ‘https://’
Enter Registry Username
Enter the username for the registry mentioned in the previous step. Leave this entry blank if the registry does not require authentication.
Enter Registry Password or Access Token
Enter Password or Access Token for the registry.
Input is masked with * characters.
Leave this entry blank if the registry does not require authentication.
Enter Owner Email (for cluster tagging)
Enter email address in the format name@domain.tld. This email is used for the cluster ‘Owner’ tag and the platform owner contact.
Note: The cluster creation process can take approximately 15-20 minutes.
If the session is terminated during installation due to network issues, power outage, and so on, then the installation stops. To restart the installation, run the script again:
# Navigate to deployment directory and run the following command./bootstrap.sh --cloud azure destroy
# Restore the cluster./bootstrap.sh --cloud azure --restore
After the bootstrap script is completed, verify the cluster and workloads using the following commands:
# Confirm nodes are Readykubectl get nodes
# Confirm NFA workloads are Runningkubectl get pods -A
2.10 - Troubleshooting
Accessing the PPC CLI
Permission denied for publickey: Ensure the correct private key ~/.ssh/<cluster_name>_user_svc is used and matches the authorized_keys in the pod.
Connection refused: Verify the load balancer IP and hosts file configuration.
Key format issues: Ensure the private key is in the correct format . For example, OpenSSH format for Linux or macOS, .ppk for PuTTY.
Component installation issues
Helm chart not found: Run helm repo update to refresh the repository cache.
Namespace already exists: Drop the --create-namespace flag if the namespace is already created.
CRD conflicts: If cert-manager CRDs already exist, skip the CRD installation step.
Pod not starting: Inspect logs with kubectl logs <pod> -n <namespace> and kubectl describe pod <pod> -n <namespace>.
State lock error after interrupted installation
Issue: When an installation is interrupted mid-apply (Ctrl+C, network drop, SSH disconnect, or jump box crash), subsequent installation attempts fail with the following error:
Error: Error acquiring the state lock
Error message: state blob is already locked
Lock Info:
ID: 9377a903-94d1-4b19-3e57-1ee92b18bed5
Path: https://<storage>.blob.core.windows.net/velero/infrastructure-manifest%2F<cluster>%2Fstates%2F01_identity.tfstate
Operation: OperationTypeApply
Description: OpenTofu acquires a lease on the state blob in Azure Blob Storage before applying changes. If the process is terminated without a graceful shutdown, the lease is not released automatically. This can occur due to Ctrl+C, network disconnection, SSH session timeout, or a jump box reboot.
Resolution: Force-unlock the state using the lock ID from the error message, then re-run the installation.
cd iac/azure/cluster/modules/<failed-stage>
tofu init
tofu force-unlock <lock-id>
Replace <failed-stage> with the module directory that failed (for example, 01_identity) and <lock-id> with the ID shown in the error output.
2.11 - Restoring the SSH keys and PCT from backed up Terraform state file
Complete the steps provided in this section to restore the SSH keys and PCT from a backed up Terraform state file of a deleted v1.1.0 jump box.
This section describes the procedure to rebuild a usable PPC v1.1.0 deployment workspace on a jump box from an existing cluster and a backed up Terraform state file.
Before you Begin
Ensure the following requirements are met.
The state file from all the following modules must be available on the Storage Account.
00_backup_bucket
01_iam_roles
02_eks_cluster
03_node_group
04_eks_addons
05_helm_releases
Access to the same Microsoft Azure account and target AKS cluster.
Azure CLI credentials with required permissions.
Perform the following steps to recover PPC
Create a workspace.
To create a working directory for PCT, run the following command.
mkdir -p deployment
cd deployment
Download and extract the PCT
Download the PCT package into the deployment directory and extract it.For more information about downloading and extracting the PCT, refer to Preparing for PPC deployment.
After successfully extracting the template, this directory behaves as recovered installation workspace.
Login to Microsoft Azure as a Service Principal
To login to Microsoft Azure as a Service Principal, run the following command
az login -u <username> -p <password> -t <tenant-id>