Installing PPC on Microsoft Azure

• 1: Prerequisites
• 2: Preparing for PPC deployment
• 3: Deploying PPC
• 3.1: Production configuration
• 3.2: Configuring PPC for Non-FIPS Deployment on Microsoft Azure
• 3.3: Automated script-based installation
• 3.4: Manual component-based installation
• 4: Installing Features and Protectors
• 5: Login to PPC
• 6: Accessing the PPC CLI
• 7: Deleting PPC
• 8: Backing up the PPC
• 9: Restoring the PPC
• 10: Troubleshooting
• 11: Restoring the SSH keys and PCT from backed up Terraform state file

1 - Prerequisites

Microsoft Azure Resource Providers

The following Microsoft Azure resource providers are registered.

• Microsoft.ContainerService
• Microsoft.Network
• Microsoft.Compute
• Microsoft.Storage
• Microsoft.KeyVault
• Microsoft.ManagedIdentity

Permissions to deploy AKS

Update the following custom role permissions.

Before you begin

• Identify the resource group where the AKS cluster will be created.
• Ensure that a custom role is already created in the subscription or resource group.
• Complete this configuration before starting the AKS installation process.

To update the permission for the custom role, perform the following steps:

1. Navigate to the required resource group in the Azure portal.

2. Open Access control (IAM) for the selected resource group.

3. Locate the existing custom role.

4. Select the custom role and click Edit.

5. Select JSON and click Edit.

   Update the role by adding the following permissions:

  {
    "id": "/subscriptions/<SUBSCRIPTION-ID>/providers/Microsoft.Authorization/roleDefinitions/<ROLE-DEFINITION-ID>",
    "properties": {
        "roleName": "<ROLE-NAME>",
        "description": "",
        "assignableScopes": [
            "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RG-Name>"
        ],
        "permissions": [
            {
                "Actions": [
                    "Microsoft.Authorization/roleAssignments/read",
                    "Microsoft.Authorization/roleAssignments/write",
                    "Microsoft.Authorization/roleDefinitions/read",
                    "Microsoft.Authorization/roleDefinitions/write",
                    "Microsoft.ContainerService/managedClusters/read",
                    "Microsoft.ContainerService/managedClusters/write",
                    "Microsoft.KeyVault/vaults/read",
                    "Microsoft.KeyVault/vaults/write",
                    "Microsoft.ManagedIdentity/userAssignedIdentities/assign/action",
                    "Microsoft.ManagedIdentity/userAssignedIdentities/read",
                    "Microsoft.Network/virtualNetworks/read",
                    "Microsoft.Network/virtualNetworks/subnets/join/action",
                    "Microsoft.Network/virtualNetworks/subnets/read",
                    "Microsoft.Resources/subscriptions/resourceGroups/read",
                    "Microsoft.Storage/locations/checknameavailability/read",
                    "Microsoft.Storage/storageAccounts/blobServices/containers/read",
                    "Microsoft.Storage/storageAccounts/blobServices/containers/write",
                    "Microsoft.Storage/storageAccounts/blobServices/read",
                    "Microsoft.Storage/storageAccounts/blobServices/write",
                    "Microsoft.Storage/storageAccounts/fileServices/read",
                    "Microsoft.Storage/storageAccounts/fileServices/write",
                    "Microsoft.Storage/storageAccounts/listKeys/action",
                    "Microsoft.Storage/storageAccounts/read",
                    "Microsoft.Storage/storageAccounts/write",
                    "Microsoft.Network/privateDnsZones/read",
                    "Microsoft.Network/privateDnsZones/A/read",
                    "Microsoft.Network/privateDnsZones/A/write",
                    "Microsoft.Network/privateDnsZones/write",
                    "Microsoft.Network/privateDnsZones/join/action",
                    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/read",
                    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/write",
                    "Microsoft.ContainerService/managedClusters/listClusterUserCredential/action",
                    "Microsoft.ContainerService/managedClusters/agentPools/read",
                    "Microsoft.ContainerService/managedClusters/agentPools/write",
                    "Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/write",
                    "Microsoft.KeyVault/vaults/read"
                ],
                "notActions": [],
                "dataActions": [],
                "notDataActions": []
            }
        ]
    }
}

6. Click Save after updating the permissions.
7. Click Review + update to finalise the changes.

Jump Box or Local Machine

A jump box serves as a secure, controlled access point for executing installation commands and managing cluster resources. It ensures that all operations originate from a trusted environment with direct network access to the PPC cluster, avoiding connectivity issues that may arise from local machines with restricted network paths or inconsistent tooling.

• Use a dedicated Debian or Red Hat jump box created in Microsoft Azure. Do not use a jump box hosted on any other cloud.

• The jump box must be provisioned in the same Azure region where the PPC cluster will be deployed to ensure proper connectivity and access to regional resources.

  For example, if the PPC cluster is deployed in eastus, the jump box must also be created in eastus. Placing the jump box in a different region, such as westeurope, may result in network access failures or latency issues when connecting to cluster endpoints.

Microsoft Azure Region

For automated installation

Identify the Azure region where the cluster is to be deployed.

The deployment region is automatically detected from the resource group’s location during the bootstrap process. Ensure that the resource group is created in the intended deployment region before running the installer.

All resources must be in the same region.
For more information about Microsoft Azure resources, refer to Microsoft Azure Resource IDs from Infrastructure Team.

For component-based installation

Identify the Azure region where the cluster is to be deployed.

Region must be specified as azure_location in the terraform.tfvars file during the 00_backup_storage and 02_aks_cluster stages.

All resources must be in the same region.
For more information about Microsoft Azure resources, refer to Microsoft Azure Resource IDs from Infrastructure Team.

Microsoft Azure Resource IDs from Infrastructure Team

Deployment Resource Group

Before proceeding with the installation, identify the Azure Resource Group where the cluster is to be deployed.

• For automated deployment, the value of this resource group name must be specified during the input prompt while running the ./bootstrap.sh script.
• For component-based deployment, the value of this resource group name must be specified in the resource_group_name field in the terraform.tfvars files, wherever required.

The following resources are provisioned within the resource group specified during deployment.

Pre-existing Resources

The following resources must be provisioned by the IT team before installation.

They can reside in any resource group in the same region. The installer references each resource by its entire Azure resource ID.

Ensure the deploying identity, such as, service principal or managed identity, has the appropriate permissions on these resources.

AKS User-Assigned Managed Identity Permissions

The AKS User-Assigned Managed Identity (id-aks-applianceframework) must be granted a custom role with the following permissions on the resource group containing the Private DNS Zone and Virtual Network. These permissions enable AKS to manage private DNS records and join virtual networks during cluster operations.

To configure User-Assigned Managed Identity with a custom Azure role containing the permissions required for AKS deployment and management operations within the specified resource group, perform the following steps:

1. Creating a Custom Role

1. In the Azure portal, navigate to Resource Groups.
2. Select the resource group that contains the AKS resources.
3. Open Access Control (IAM).
4. Select Add > Add custom role.
5. Provide a name and description for the custom role.
6. Select JSON and add the following permissions.

{
  "Name": "<ROLE-NAME>",
  "Description": "",
  "AssignableScopes": [
    "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RESOURCE-GROUP>"
  ],
  "Actions": [
    "Microsoft.Network/privateDnsZones/read",
    "Microsoft.Network/privateDnsZones/A/read",
    "Microsoft.Network/privateDnsZones/A/write",
    "Microsoft.Network/privateDnsZones/A/delete",
    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/read",
    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/write",
    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/delete",
    "Microsoft.Network/virtualNetworks/subnets/read",
    "Microsoft.Network/virtualNetworks/subnets/join/action",
    "Microsoft.Network/privateDnsZones/write",
    "Microsoft.Network/virtualNetworks/join/action"
  ],
  "NotActions": [],
  "DataActions": [],
  "NotDataActions": []
}

7. Review the permissions and create the custom role.

Note: These permissions are required when using a private AKS cluster with a pre-existing Private DNS Zone (private_dns_zone_id). Without them, cluster provisioning and DNS resolution will fail.

2. Assigning the custom role to the Managed Identity

1. On the Azure portal, navigate to Managed Identities.

2. Select Create.

3. Specify the following:

   • Subscription
   • Resource group
   • Managed identity name
   • Region

4. Select Review + Create and then Create.

   Wait for the User-Assigned Managed Identity (UAMI) to be provisioned.

5. Open the newly created User-Assigned Managed Identity.

6. Select Azure role assignments.

7. Select Add role assignment.

8. Set the scope to the target resource group.

9. Select the custom role created in the previous section.

10. Assign the role to the User-Assigned Managed Identity.

11. Verify that the role assignment appears under the managed identity’s role assignments.

Zone Requirements

The cluster requires a minimum of two availability zones for high availability. Three-zone cluster deployments are supported and recommended for production.

Ensure the following requirements are met before deploying:

Modules: 02_aks_cluster, 03_node_pool

Note: The bootstrap script does not prompt for zone selection, it uses the default three zone configuration.

The default zone configuration is ["1", "2", "3"]. Not all Azure regions support three availability zones, and not all VM SKUs are available in every zone. Using a zone that does not support your VM SKU causes cluster creation to fail.

Important: system_node_pool_zones (module 02) and node_pool_zones (module 03) must use the same set of zones. If the zones differ, disks provisioned in one zone cannot be attached to nodes rescheduled into another zone.

Before deploying, verify zone and VM SKU availability:

# List availability zones for your region
az account list-locations \
  --query "[?name=='<region>'].{Region:name, Zones:availabilityZoneMappings}" \
  -o table

# Check which zones support your VM SKU
az vm list-skus \
  --location <region> \
  --size Standard_D4ds_v4 \
  --query "[].{Name:name, Zones:locationInfo[0].zones}" \
  -o table

Replace <region> with your Azure region, for example eastus.

If zone adjustment is required, navigate to iac/azure/cluster/modules and set matching values in both terraform.tfvars files:

# 02_aks_cluster/terraform.tfvars
system_node_pool_zones = ["1", "2"]    # adjust to the zones your region supports

# 03_node_pool/terraform.tfvars
node_pool_zones = ["1", "2"]           # must match system_node_pool_zones exactly

For more information on editing the 02_aks_cluster, and 03_node_pool modules, refer to Manual component-based installation.

System node pool sizing

The step [1b] Select Node Architecture in the bootstrap script prompts to select the node architecture interactively during deployment.

The default system node pool VM size (Standard_D2ds_v4, 2 vCPU / 8 GiB RAM) can become resource-constrained under production load when Karpenter, CoreDNS, and CSI drivers run concurrently on the same node. Use the following or larger for production system node pools.

amd64 (x86-64)

arm64

In 02_aks_cluster/terraform.tfvars, edit the following value:

system_node_pool_vm_size = "Standard_D4ds_v5"   # or Standard_D8ds_v4 for heavier loads

For more information on editing the 02_aks_cluster module, refer to Manual component-based installation.

AKS cluster tier

The sku_tier setting controls the AKS API server uptime SLA, available node capacity, and supported features.

Set sku_tier = "Standard" for all production deployments. Use "Premium" if your organization requires long-term support (LTS) guarantees.

In 02_aks_cluster/terraform.tfvars, edit the following value:

sku_tier = "Standard"

For more information on editing the 02_aks_cluster module, refer to Manual component-based installation.

2 - Preparing for PPC deployment

The Protegrity Cluster Template (PCT) is a pre-packaged deployment archive that contains the infrastructure-as-code scripts, Helm charts, and bootstrap tooling required to provision a PPC on your target cloud platform. It automates the creation of the Kubernetes cluster, networking, identity configuration, and installation of Protegrity microservices. Download and extract the PCT on your jump box before starting the deployment process.

Note: If there is an existing cluster from a previous install, clean up your local repository on the jump box and any existing clusters by running ./bootstrap.sh --cloud azure destroy from /deployment before proceeding.

During installation, the system may prompt for the system password and require sign-in to Microsoft Azure. If the Azure CLI is not already logged in, the bootstrap script automatically runs az login --use-device-code. A device-code prompt similar to the following appears.

[YYYY-MM-DD HH:MM:SS] Azure CLI not logged in. Triggering az login...
To sign in, use a web browser to open the page https://login.microsoft.com/device and enter the code XXXXXXXX to authenticate.

To sign-in to Microsoft Azure, perform the following steps:

1. Open the displayed URL in a browser, and enter the code shown in the terminal.
2. Complete the SSO sign-in and follow the on-screen instructions.
3. After successful authentication, the script continues automatically.

To download and extract PCT, perform the following steps:

1. Log in to the My.Protegrity portal.

2. Navigate to Product Management > Explore Products > AI Team Edition > Platform & Features.

3. Navigate to Platform Installation.

4. From the Actions column for Protegrity Provisioned Cluster, click the Download Product icon to download the PPC 1.1 archive.

5. Create a deployment_110 directory on the jumpbox.

   mkdir deployment_110 && cd deployment_110
   

6. Copy the PCT to the deployment_110 directory on the jump box.

7. Extract the PCT.

   tar -xvf PPC-K8S-ALL_1.1.0.97.tar
   

3 - Deploying PPC

By default, the PCT is configured to deploy a FIPS-enabled cluster using FIPS-compliant node OS images.

Non-FIPS deployment: Before proceeding with the installation, complete the steps in Configuring PPC for Non-FIPS Deployment on Microsoft Azure to modify the required Terraform configuration files. Then proceed with either the automated or component-based installation below.

FIPS deployment: No additional configuration is required. Proceed directly with either the automated or component-based installation below.

The PPC cluster can be deployed using one of the following methods:

Automated script-based installation: This method requires the bootstrap.sh script to deploy the PPC cluster. The script installs the required software and dependencies, prompts to provide the required information, and deploys the cluster automatically.

Manual component-based installation: This method requires to perform the deployment of each component to install the PPC cluster. The configuration files are edited to provide the required information and then deploy the cluster.

Note: A PPC can be deployed in regions where minimum two zones are present.

3.1 - Production configuration

The default terraform.tfvars values are optimized for development and staging environments. Before deploying to production, override the settings below to ensure high availability, data safety, and security compliance.

Applying the changes

• Script-based installation: After the bootstrap script generates the terraform.tfvars files, edit them before the script runs tofu apply, or modify and re-apply individually using OpenTofu after installation.

• Component-based installation: Edit the relevant terraform.tfvars for each module before running tofu apply.

Storage redundancy

Modules: 00_backup_storage, 04_aks_addons

By default, the backup storage account uses Locally Redundant Storage (LRS), which stores data only within a single datacenter. When AKS nodes span multiple availability zones, an LRS-backed disk cannot be reattached to a node in a different zone during failover, causing pod scheduling to fail.

Recommendations:

• Use ZRS (Zone-Redundant Storage) when nodes are spread across availability zones in the same region.
• Use GRS (Geo-Redundant Storage) if cross-region disaster recovery is required.
• Use Premium_ZRS as the storage class for Kubernetes persistent volumes.

In 00_backup_storage/terraform.tfvars:

storage_account_replication_type = "ZRS"   # use "GRS" for cross-region redundancy

System node pool sizing

Module: 02_aks_cluster

The default system node pool VM size (Standard_D2ds_v4, 2 vCPU / 8 GiB RAM) can become resource-constrained under production load when Karpenter, CoreDNS, and CSI drivers run concurrently on the same node.

Recommendation: Use Standard_D4ds_v4 (4 vCPU / 16 GiB) or larger for production system node pools.

In 02_aks_cluster/terraform.tfvars, edit the following value:

system_node_pool_vm_size = "Standard_D4ds_v4"   # or Standard_D8ds_v4 for heavier loads

Key Vault SKU

Module: 00_backup_storage

The default Key Vault SKU is standard, which uses software-protected keys. For organizations with compliance requirements (such as FIPS 140-2 Level 2 or Level 3), the premium SKU provides HSM-backed encryption keys.

Recommendation: Use premium when your security or compliance policy requires HSM-backed key protection.

In 00_backup_storage/terraform.tfvars:

key_vault_sku = "premium"

Note: Confirm with your compliance team whether HSM-backed keys are required before changing this setting. The premium SKU incurs additional Azure costs.

Availability zones

Modules: 02_aks_cluster, 03_node_pool

The default zone configuration is ["1", "2", "3"]. Not all Azure regions support three availability zones, and not all VM SKUs are available in every zone. Using a zone that does not support your VM SKU causes cluster creation to fail.

Important: system_node_pool_zones (module 02) and node_pool_zones (module 03) must use the same set of zones. If the zones differ, disks provisioned in one zone cannot be attached to nodes rescheduled into another zone.

Before deploying, verify zone and VM SKU availability:

# List availability zones for your region
az account list-locations \
  --query "[?name=='<region>'].{Region:name, Zones:availabilityZoneMappings}" \
  -o table

# Check which zones support your VM SKU
az vm list-skus \
  --location <region> \
  --size Standard_D4ds_v4 \
  --query "[].{Name:name, Zones:locationInfo[0].zones}" \
  -o table

Replace <region> with your Azure region, for example eastus.

In both 02_aks_cluster/terraform.tfvars and 03_node_pool/terraform.tfvars, set matching values:

# 02_aks_cluster/terraform.tfvars
system_node_pool_zones = ["1", "2"]    # adjust to the zones your region supports

# 03_node_pool/terraform.tfvars
node_pool_zones = ["1", "2"]           # must match system_node_pool_zones exactly

AKS cluster tier (sku_tier)

Module: 02_aks_cluster

The sku_tier setting controls the AKS API server uptime SLA, available node capacity, and supported features.

Recommendation: Set sku_tier = "Standard" for all production deployments. Use "Premium" if your organization requires long-term support (LTS) guarantees.

In 02_aks_cluster/terraform.tfvars:

sku_tier = "Standard"

Note: Standard and Premium tiers incur an additional hourly cost per cluster. See the AKS pricing page for current rates.

Storage account network access

Module: 00_backup_storage

The backup storage account is created with public network access enabled by default. In production, restrict access using your organization’s existing network controls.

Azure provides three options:

Apply the appropriate controls after deployment. The example below uses the service endpoint approach:

# Allow access from a specific VNet subnet
az storage account network-rule add \
  --account-name <storage-account-name> \
  --resource-group <resource-group> \
  --vnet-name <vnet-name> \
  --subnet <subnet-name>

# Deny all other public access
az storage account update \
  --name <storage-account-name> \
  --resource-group <resource-group> \
  --default-action Deny

Note: Consult your organization’s security team to select the appropriate access control method before deploying to production.

3.2 - Configuring PPC for Non-FIPS Deployment on Microsoft Azure

Before you begin

• By default, the Azure PPC installation enables FIPS (Federal Information Processing Standards). Follow these steps only if you need to install a non-FIPS cluster. For the standard FIPS-enabled installation, refer to Installing PPC on Microsoft Azure.

• Complete all prerequisites before proceeding.

• Perform these steps after downloading and extracting the PCT and before proceeding with either the Script-based installation or the Component-based installation.

The following files require changes:

Configuring PPC for Non-FIPS Deployment on Azure

For a non-FIPS PPC deployment on Azure AKS, perform the following steps:

1. Disable FIPS in the AKS Cluster Module.

a. Navigate to the 02_aks_cluster module.

cd iac/azure/cluster/modules/02_aks_cluster/

b. Edit the terraform.tfvars file. Locate the fips_enabled parameter and set it to false.

fips_enabled                  = false

c. Save the file and exit the editor.

2. Disable FIPS in the Node Pool Module.

a. Navigate to the 03_node_pool module.

cd iac/azure/cluster/modules/03_node_pool/

b. Edit the terraform.tfvars file. Locate the fips_enabled parameter and set it to false.

fips_enabled                  = false

c. Save the file and exit the editor.

After saving all these configuration changes, proceed with deploying the cluster using one of the following methods:

• Script-based installation: Run the bootstrap.sh script to deploy the cluster automatically.

• Component-based installation: Apply each Terraform module individually.

Note: The Script-based and Component-based installation instructions reference default FIPS-enabled settings. When following either installation method, do not revert or overwrite the values configured on this page. Keep fips_enabled set to false in 02_aks_cluster and 03_node_pool.

3.3 - Automated script-based installation

Before you begin

• Ensure that all the required resources are in the same resource-group. If the resources are in different resource groups, then ensure to move the resources to the same group before proceeding for PPC deployment.

• The repository provides a bootstrap script that automatically installs or updates the following tools on the jump box:

  • Azure CLI - Required to communicate with your Microsoft Azure account.
  • Helm - Required to manage Kubernetes packages.
  • jq - Required to parse JSON.
  • kubectl - Required to communicate with the Kubernetes cluster.
  • OpenTofu - Required to manage infrastructure as code.
  • oras: Required to pull non‑container, generic OCI artifacts from the registry that are not handled by standard container tooling.

Note: If you are performing a non-FIPS deployment, ensure that you have already completed the steps in the section Configuring PPC for Non-FIPS Deployment on Microsoft Azure before proceeding. Do not revert fips_enabled to true or change node_os back to a FIPS-compliant image during the steps below.

Architecture Selection

The step [1b] Select Node Architecture in the bootstrap script prompts to select the node architecture interactively during deployment.

• amd64 (x86-64)

• arm64

The bootstrap script prompts you to select the node architecture interactively during deployment (step 1b).

Deploying the PPC

The bootstrap script prompts for variables to be set to complete the deployment. Run the following command and follow the instructions on the screen.

./bootstrap.sh --cloud azure

The script prompts for the following variables.

1. Enter AKS Cluster Name

   The following characters are allowed:

   • Lowercase letters: a-z
   • Numbers: 0-9
   • Hyphens: -

   The following characters are not allowed:

   • Uppercase letters: A-Z
   • Underscores: _
   • Spaces
   • Any special characters such as: / ? * + % ! @ # $ ^ & ( ) = [ ] { } : ; , .
   • Leading or trailing hyphens
   • Names longer than 10 characters (the cluster name must be 10 characters or fewer)

   Note: Ensure that the cluster name does not exceed 10 characters. Cluster names longer than this limit can cause the bootstrap script to fail in subsequent installation steps. If the installation fails because the cluster name exceeds the 10-character limit, correct the name and re-run the script.
   

   • Correction: Choose a cluster name with 31 characters or fewer.
     
   • Retry: Execute the installation command again with the updated name. The script will automatically handle the update and proceed with the bootstrap process.

   After the cluster name is accepted, the script prompts you to select the node architecture.

   [1b] Select Node Architecture

   [1b] Select Node Architecture
     1) amd64 (x86_64) — e.g. Standard_D8as_v5
     2) arm64 (ARM/Ampere) — e.g. Standard_D8ps_v5
   

   Enter 1 to select amd64 or 2 to select arm64.

   Note: Select amd64 for standard Intel/AMD 64-bit environments. Select arm64 for ARM-based environments such as AWS Graviton instances.

2. Querying for available Resource Groups

   The script queries for the available Resource Groups.

   Enter the Resource Group name from the table []

   The script then automatically detects the location and subscription ID of the resource group.

3. Enter UAMI Resource ID

   Provide the complete Azure resource ID for the UAMI used by AKS in the following format:

   /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<identity-name>
   

   The UAMI client ID is detected automatically.

4. Enter AKS Subnet Resource ID

   Provide the complete resource ID of the pre-existing subnet used for AKS nodes in the following format:

   /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/virtualNetworks/<vnet-name>/subnets/<subnet-name>
   

5. Enter Private DNS Zone Resource ID

   Provide the Private DNS zone ID used by the AKS private cluster in the following format:

   /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/privateDnsZones/privatelink.<region>.azmk8s.io
   

   The script attempts to automatically detect network settings:

   • Virtual network address space
   • Service CIDR
   • DNS service IP

   If the detection fails, then default values configured in the terraform.tfvars file are used.

6. Enter FQDN [name.domain.com]

   This is the Fully Qualified Domain Name for the ingress.

   Warning: Ensure that the FQDN does not exceed 50 characters and only the following characters are used:

   • Lowercase letters: a-z
   • Numbers: 0-9
   • Special characters: - .

7. Storage Account and Key Vault provisioning

   Choose whether to use existing resources or create new resources:

   1) Use existing
   2) Provision new via Tofu
   

   • Enter 1 if an encrypted Storage Account and Key Vault are already provisioned for this cluster. The installer prompts for the Storage Account name, Key Vault name, backup container, Key Vault key name, and the Velero UAMI Resource ID.

   • Enter 2 to allow the installer to create a new Storage Account and Key Vault with the velero container, the pty-backup-key encryption key, and a Velero UAMI automatically. Only the new resource names are required.

8. Enter Velero UAMI Resource ID (required)

   Enter the resource ID of the dedicated Velero User-Assigned Managed Identity (UAMI) in the format:

   /subscriptions/<subscription-id>/resourceGroups/<velero-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name>
   

   The script automatically validates the UAMI and detects the UAMI Client ID.

9. Enter OpenSearch UAMI Resource ID (required)

   Enter the resource ID of the OpenSearch Velero User-Assigned Managed Identity (UAMI) in the format:

   /subscriptions/<subscription-id>/resourceGroups/<opensearch-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name>
   

   The script automatically validates the UAMI and detects the UAMI Client ID.

10. Enter Image Registry Endpoint

The image repository from where the container images are retrieved.

Expected format: <hostname>[:port].

Do not include ‘https://’

11. Enter Registry Username

Enter the username for the registry mentioned in the previous step. Leave this entry blank if the registry does not require authentication.

12. Enter Registry Password or Access Token

Enter Password or Access Token for the registry.

Input is masked with * characters. Press Enter to keep the current value.

Leave this entry blank if the registry does not require authentication.

13. Enter Owner Email (for cluster tagging)

Enter email address in the format name@domain.tld. This email is used for the cluster ‘Owner’ tag and the platform owner contact.

Note: The cluster creation process can take 10-15 minutes.

If the session is terminated during installation due to network issues, power outage, and so on, then the installation stops. To restart the installation, run the script again:

# Navigate to setup directory 
./bootstrap.sh --cloud azure

After the bootstrap script is completed, verify the cluster and workloads using the following commands:

# Confirm nodes are Ready
kubectl get nodes

# Confirm NFA workloads are Running
kubectl get pods -A

3.4 - Manual component-based installation

This section is intended for enterprise customers who need full control over each stage of the AKS cluster deployment. Use this path when individual infrastructure stages must be reviewed or approved separately, when integration with an existing CI/CD pipeline is required, or when corporate change-management controls prevent running an all-in-one automated script. Each stage is self-contained and can be executed by different teams on the same jump box.

Before deploying to production: The default values are optimized for development and staging. Review the Production configuration section to override settings for high availability, data safety, and security compliance before running tofu apply.

Note: If you are performing a non-FIPS deployment, ensure you have already completed Configuring PPC for Non-FIPS Deployment on Microsoft Azure before proceeding. Do not revert fips_enabled to true when performing the following steps.

Deploying PPC using component-based installation

To install PPC, complete the following steps in order. Each step is described in detail in the sections below.

Before you begin

Ensure to have the following prerequisites before proceeding with the installation.

Access and permissions

Required Roles

The Azure principal must have AKS cluster admin access.

The required Azure RBAC Roles are listed in the following table:

Deployment Resource Group

All resources created by the installer are provisioned in a single Azure resource group specified using resource_group_name in each terraform.tfvars file. This resource group must exist before running the installer.

The following resources are created inside this resource group.

Installing the tools

After downloading and extracting the PPC-K8S-ALL_1.1.0.97.tar, run the following command to install the required tools.

./bootstrap.sh install-tools --cloud azure

Alternatively, the following tools must be installed and available on the jumpbox before starting:

Values to collect before starting

Gather the following values before populating each stage terraform.tfvars in Step 4.

Deploying the PPC

To deploy the PPC using component-based installation, perform the following steps:

1. Authenticating with Azure CLI

To authenticate with the Azure CLI on the jump box, run the following command.

az login
az account show

Ensure the target subscription, resource group, managed identities, subnet, and private DNS zone are accessible before continuing.

2. Downloading and extracting the release PCT

To download and extract the PCT, refer to Preparing for PPC deployment.

3. Provisioning backup storage and key vault

This is an optional step.

Note: Skip this step if an Azure Storage Account, backup container, Key Vault, and key already exist for this cluster. Note these values. They will be needed in next steps.

1. Navigate to the 00_backup_storage directory.

cd iac/azure/cluster/modules/00_backup_storage

2. Open terraform.tfvars and fill in the required values.

    resource_group_name    = "<resource-group>"
    aks_cluster_name       = "<cluster-name>"
    storage_account_name   = "<storage-account-name>"
    storage_container_name = "velero"
    key_vault_name         = "<key-vault-name>"
    key_name               = "pty-backup-key"

    azure_subscription_id = "<subscription-id>"
    azure_location        = "<region>"        # e.g. eastus

3. Save the file and exit the editor.

4. To apply the terraform configuration, run the following command.

    tofu init
    tofu plan          # review the role assignments that will be created
    tofu apply         # type "yes" when prompted

5. Note the Storage Account, container, Key Vault, and key values for future steps.

4. Filling in each module’s terraform.tfvars

Each module under iac/azure/cluster/modules/ contains its own terraform.tfvars file. There is no single shared configuration file, so the terraform.tfvars file in every module must be updated before running the deployment.

Verify that the following values are the same across all modules:

• resource_group_name
• aks_cluster_name
• azure_subscription_id
• storage_account_name
• storage_container_name

1. 01_identity module

This stage creates RBAC role assignments for the pre-existing User-Assigned Managed Identities (UAMIs) required by AKS, Velero, and OpenSearch.

To update the 01_identity module, perform the following steps.

a. Navigate to the iac/azure/cluster/modules/01_identity/ directory.

    cd iac/azure/cluster/modules/01_identity/

b. Edit the terraform.tfvars file to provide the required values.

    resource_group_name    = "<resource-group>" # e.g. "aks-rg-prod"
    aks_cluster_name       = "<cluster-name>" # e.g. "prod-aks-cluster"
    azure_subscription_id  = "<subscription-id>" # e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"
    storage_account_name   = "<storage-account-name>" # e.g. "myclusterstateaccount"
    storage_container_name = "velero"

    # -------------------------------------------------
    # Pre-existing managed identities (provided by IT)
    # -------------------------------------------------
    uami_id = "</subscriptions/<subscription-id>/resourceGroups/<it-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-applianceframework>"

    velero_uami_id = "</subscriptions/<subscription-id>/resourceGroups/<velero-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-velero>" # required

    opensearch_uami_id = "</subscriptions/<subscription-id>/resourceGroups/<opensearch-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/aks-opensearch-identity>" # required

    # -------------------------------------------------
    # Key Vault (leave empty to skip KV role assignment)
    # -------------------------------------------------
    key_vault_name = "<key-vault-name>" # e.g. "my-key-vault"

c. Save the file and exit the editor.

d. To initialise, plan, and apply the changes, run the following commands.

    tofu init
    tofu plan          # review the role assignments that will be created
    tofu apply         # type "yes" when prompted

e. To verify the resources created in this stage, run the following commands

For AKS UAMI:

# Retrieve the Principal ID of the AKS User-Assigned Managed Identity (UAMI)
az identity show \
  --ids "<uami-resource-id>" \
  --query principalId \
  --output tsv

#Use the returned Principal ID to verify the role assignments:
az role assignment list \
  --assignee "<aks-uami-principal-id>" \
  --output table

For Velero UAMI

# Retrieve the Principal ID of the Velero UAMI
az identity show \
  --ids "<velero-uami-principal-id>" \
  --query principalId \
  --output tsv

#Use the returned Principal ID to verify the role assignments:
az role assignment list \
  --assignee "<velero-uami-principal-id>" \
  --output table

For OpenSearch UAMI

# Retrieve the Principal ID of the OpenSearch UAMI
az identity show \
  --ids "<opensearch-uami-principal-id>" \
  --query principalId \
  --output tsv

#Use the returned Principal ID to verify the role assignments:
az role assignment list \
  --assignee "<opensearch-uami-principal-id>" \
  --output table

2. 02_aks_cluster module

This stage provisions the private AKS cluster with a system node pool, configures networking (Azure CNI Overlay and Cilium), enables Workload Identity, and creates federated identity credentials for Velero and OpenSearch.

To update the 02_aks_cluster module, perform the following steps.

a. Navigate to the iac/azure/cluster/modules/02_aks_cluster/ directory.

    cd iac/azure/cluster/modules/02_aks_cluster/

Note (Non-FIPS deployment): If you are deploying a non-FIPS cluster, ensure fips_enabled remains false as configured in the section Configuring PPC for Non-FIPS Deployment on Microsoft Azure. Do not change this value to true.

b. Edit the terraform.tfvars file to provide the required values.

    resource_group_name    = "<resource-group>" # e.g. "aks-rg-prod"
    aks_cluster_name       = "<cluster-name>" # e.g. "prod-aks-cluster"
    azure_subscription_id  = "<subscription-id>" # e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"
    storage_account_name   = "<storage-account-name>" # e.g. "myclusterstateaccount"
    storage_container_name = "velero"

    # -------------------------------------------------
    # Identity (UAMI for kubelet and federated credentials)
    # -------------------------------------------------
    uami_id            = "</subscriptions/<subscription-id>/resourceGroups/<it-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-applianceframework>" # full UAMI resource ID (same as Stage 01)
    velero_uami_id     = "</subscriptions/<subscription-id>/resourceGroups/<velero-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-aks-velero>" # full Velero UAMI resource ID (for federated credential)
    opensearch_uami_id = "</subscriptions/<subscription-id>/resourceGroups/<opensearch-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/aks-opensearch-identity>" # full OpenSearch UAMI resource ID (for federated credential)

    # -------------------------------------------------
    # Cluster
    # -------------------------------------------------
    azure_location      = "<region>" # e.g. "eastus"  (auto-detected from resource group)
    subnet_id           = "</subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>/subnets/<subnet>>" # e.g. "/subscriptions/.../subnets/snet-aks-applianceframework"
    private_dns_zone_id = "</subscriptions/<subscription-id>/resourceGroups/<dns-resource-group>/providers/Microsoft.Network/privateDnsZones/privatelink.eastus.azmk8s.io>" # e.g. "/subscriptions/.../privateDnsZones/privatelink.eastus.azmk8s.io"

    # -------------------------------------------------
    # Networking (service CIDR + DNS IP)
    # -------------------------------------------------
    service_cidr   = "<10.0.0.0/16>" # auto-derived by updatevartf.sh (non-overlapping with the VNet)
    dns_service_ip = "<10.220.0.10>" # auto-derived: the .10 host of the chosen service_cidr
    pod_cidr       = "<10.0.0.0/16>" # auto-derived: non-overlapping with the VNet and service_cidr

    # -------------------------------------------------
    # System node pool
    # -------------------------------------------------
    architecture                  = "amd64" # amd64 or arm64
    system_node_pool_vm_size      = ""      # leave empty to auto-select based on architecture
    system_node_pool_node_count   = 2
    system_node_pool_disk_size_gb = 50
    system_node_pool_os_sku       = "AzureLinux"
    system_node_pool_zones        = ["1", "2", "3"]
    fips_enabled                  = true # set false for a non-FIPS node OS image (immutable)

    # -------------------------------------------------
    # Ownership / tagging
    # -------------------------------------------------
    owner_email                   = "<name@domain.tld>" # optional; blank = auto-detect from the Azure CLI identity

c. Save the file and exit the editor.

d. To initialise, plan, and apply the changes, run the following commands.

    tofu init
    tofu plan          # review the role assignments that will be created
    tofu apply         # type "yes" when prompted

e. To verify the resources created in this stage, run the following commands.

# Confirm the AKS cluster is in Succeeded state
az aks show --name <cluster-name> --resource-group <resource-group> \
  --query "{Name:name, State:provisioningState, Version:kubernetesVersion, Fqdn:privateFqdn}" \
  --output table

A sample output is displayed below:

Name              State      Version    Fqdn
----------------  ---------  ---------  -----------------------------------------------
<cluster-name>    Succeeded  1.35       <cluster-name>.privatelink.<region>.azmk8s.io

# Verify kubectl can reach the cluster
kubectl get nodes    # expect 2 system nodes in Ready state

# Verify federated credentials exist
az identity federated-credential list \
  --identity-name <velero-uami-name> \
  --resource-group <velero-uami-resource-group> --output table

az identity federated-credential list \
  --identity-name <opensearch-uami-name> \
  --resource-group <opensearch-uami-resource-group> --output table

3. 03_node_pool module

This stage creates a user-managed node pool for application workloads. Node Auto Provisioning (NAP) handles autoscaling; no cluster autoscaler is configured on this pool.

To update the 03_node_pool module, perform the following steps.

a. Navigate to the iac/azure/cluster/modules/03_node_pool/ directory.

    cd iac/azure/cluster/modules/03_node_pool/

Note (Non-FIPS deployment): If you are deploying a non-FIPS cluster, ensure fips_enabled remains false as configured in Configuring PPC for Non-FIPS Deployment on Microsoft Azure. Do not change this value to true.

b. Edit the terraform.tfvars file to provide the required values.

    resource_group_name    = "<resource-group>" # e.g. "aks-rg-prod"
    aks_cluster_name       = "<cluster-name>" # e.g. "prod-aks-cluster"
    azure_subscription_id  = "<subscription-id>" # e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"
    storage_account_name   = "<storage-account-name>" # e.g. "myclusterstateaccount"
    storage_container_name = "velero"

    # -------------------------------------------------
    # Node pool
    # -------------------------------------------------
    subnet_id              = "</subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>/subnets/<subnet>>"      # same subnet as AKS nodes (Stage 02)
    architecture           = "amd64" # amd64 or arm64
    vm_size                = ""      # leave empty for architecture default
    node_pool_disk_size_gb = 120
    node_pool_desired_size = 3
    node_pool_zones        = ["1", "2", "3"]
    fips_enabled           = true # set false for a non-FIPS node OS image (immutable)

c. Save the file and exit the editor.

d. To initialise, plan, and apply the changes, run the following commands.

    tofu init
    tofu plan          # review the role assignments that will be created
    tofu apply         # type "yes" when prompted

Expected time for this stage is approximately 5 to 10 minutes for node pool provisioning.

e. To verify the resources created in this stage, run the following commands.

# Confirm the user node pool exists and is Succeeded
az aks nodepool show --cluster-name <cluster-name> \
  --resource-group <resource-group> --name userpool \
  --query "{Name:name, State:provisioningState, Count:count, VmSize:vmSize, OsSku:osSku}" \
  --output table

A sample output is displayed below:

Name      State      Count    VmSize      OsSku
--------  ---------  -------  ----------  ----------
userpool  Succeeded  <count>  <vm-size>   AzureLinux

# Verify all nodes are Ready (system + user nodes)
kubectl get nodes -o wide

# Confirm user pool nodes have the expected label
kubectl get nodes -l agentpool=userpool

4. 04_aks_addons module

This stage creates the default and backup Kubernetes storage classes using Azure Managed Disks (Premium LRS) and removes the built-in default storage class annotation.

To update the 04_aks_addons module, perform the following steps.

a. Navigate to the iac/azure/cluster/modules/04_aks_addons/ directory.

    cd iac/azure/cluster/modules/04_aks_addons/

b. Edit the terraform.tfvars file to provide the required values.

    resource_group_name    = "<resource-group>" # e.g. "aks-rg-prod"
    aks_cluster_name       = "<cluster-name>" # e.g. "prod-aks-cluster"
    azure_subscription_id  = "<subscription-id>" # e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"
    storage_account_name   = "<storage-account-name>" # e.g. "myclusterstateaccount"
    storage_container_name = "velero"

c. Save the file and exit the editor.

d. To initialise, plan, and apply the changes, run the following commands.

    tofu init
    tofu plan          # review the role assignments that will be created
    tofu apply         # type "yes" when prompted

e. To verify the resources created in this stage, run the following commands.

# List storage classes and confirm ebs-sc is the default
kubectl get storageclass

# Verify ebs-sc details
kubectl describe storageclass ebs-sc

# Confirm the built-in 'default' storage class is no longer the default
kubectl get storageclass default -o jsonpath='{.metadata.annotations.storageclass\.kubernetes\.io/is-default-class}'
# expected output: empty or "false"

A sample output is displayed below:

storage_class_names = {
  "backup" = "ebs-bkp"
  "default" = "ebs-sc"
}

5. 05_helm_releases module

This stage deploys all application Helm charts, configures Velero backup with the Azure plugin, and bootstraps the NFA platform and Insight analytics stack.

To update the 05_helm_releases module, perform the following steps.

a. Navigate to the iac/azure/cluster/modules/05_helm_releases/ directory.

    cd iac/azure/cluster/modules/05_helm_releases/

b. Run the following commands and note the values.

# Get the Velero Client ID
az identity show \
  --name id-aks-velero \
  --resource-group aks-ppc \
  --query clientId \
  -o tsv

# Get the OpenSearch Client ID
az identity show \
  --name aks-ppc01-opensearch-identity \
  --resource-group aks-ppc \
  --query clientId \
  -o tsv

c. Edit the terraform.tfvars file to provide the required values.

    resource_group_name    = "<resource-group>" # e.g. "aks-rg-prod"
    aks_cluster_name       = "<cluster-name>" # e.g. "prod-aks-cluster"
    azure_subscription_id  = "<subscription-id>" # e.g. "1e9ef7b6-cdc4-4a6b-98f7-a408ac1e19e0"
    storage_account_name   = "<storage-account-name>" # e.g. "myclusterstateaccount"
    storage_container_name = "velero"
    
    # -------------------------------------------------
    # Networking
    # -------------------------------------------------
    subnet_id    = "</subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>/subnets/<subnet>>" # same subnet as Stages 02/03 (for VNet address space lookup)
    pod_cidr     = ""  # optional override; empty = auto-detected from the live AKS network profile
    architecture = "amd64" # amd64 or arm64
    node_os      = "AzureLinux_FIPS" # should mirror stage-03 user node pool os_sku + fips_enabled

    # -------------------------------------------------
    # Registry
    # -------------------------------------------------
    global_image_reg = "<registry endpoint>" # e.g. "harbor.example.com"
    harbor_secret    = "<harbour secret>" # e.g. "harbor-pull-secret"
    ingress_fqdn     = "<FQDN>" # e.g. "apps.example.com"

    # -------------------------------------------------
    # Ownership / tagging
    # -------------------------------------------------
    owner_email = "<name@domain.tld>" # optional; blank = auto-detect from the Azure CLI identity

    # -------------------------------------------------
    # Restore / backup
    # -------------------------------------------------
    restore             = false                               # set to true to restore from a previous backup
    backup_name         = "authnz-postgresql-schedule-backup" # Velero backup name used when restore = true
    insight_backup_name = ""                                  # Insight Velero backup name (K8s resources only); leave empty to auto-resolve latest

    # -------------------------------------------------
    # Workload Identity
    # -------------------------------------------------
    velero_uami_client_id     = "<velero_uami_client_id>" # client ID of the Velero UAMI 
    opensearch_uami_client_id = "<opensearch_uami_client_id>" # client ID of the OpenSearch UAMI    

c. Save the file and exit the editor.

d. Export registry credentials using environment variables. Do not put them in the tfvars file.

    export TF_VAR_username="<registry-username>"
    export TF_VAR_password="<registry-password>"

e. To initialise, plan, and apply the changes, run the following commands.

    tofu init
    tofu plan          # review the role assignments that will be created
    tofu apply         # type "yes" when prompted

Expected time for this stage is approximately 15 to 20 minutes.

f. To verify the resources created in this stage, run the following commands.

# Confirm all Helm releases are deployed
helm list -A

# Verify Velero is running
kubectl get pods -n pty-backup-recovery

# Verify NFA platform pods
kubectl get pods -A

# Verify Insight pods
kubectl get pods -n pty-insight

# Verify Envoy Gateway
kubectl get pods -n api-gateway

# Verify VolumeSnapshotClass exists
kubectl get volumesnapshotclass

# Check all pods across the cluster are healthy
kubectl get pods -A --field-selector=status.phase!=Running,status.phase!=Succeeded

# Confirm the ingress gateway has an IP
kubectl get gateway -A

A sample output is displayed below:

nfa_release_status = "deployed"
user_svc_private_key = <sensitive>
velero_release_status = "deployed"

All the nodes must be in the Ready state and all the pods must be in the Running or Completed state.

Verifying the installation

To verify the installation, run the following command.

kubectl get nodes
kubectl get pods -A

4 - Installing Features and Protectors

Before you begin

Ensure that PPC is successfully installed before installing the features or protectors.

Installing Features

The following table lists the available features.

Installing Protectors

The following table lists the available protectors.

5 - Login to PPC

To access the Web UI, map the gateway hostname to the Microsoft Azure Load Balancer IP address in the local hosts file.

1. Get gateway details: Find the hostname and the Microsoft Azure Load Balancer address.

   kubectl get gateway -A
   

   The output will look similar to:

   NAMESPACE     NAME       CLASS   ADDRESS       PROGRAMMED   AGE
   api-gateway   pty-main   envoy   10.221.8.33   True         5h7m
   

2. Update the hosts file: Add an entry mapping the ingress FQDN to the IP.

   • Linux: /etc/hosts
   • Windows: C:\Windows\System32\drivers\etc\hosts

   Example entry:

   10.221.8.33    <FQDN given during cluster installation>
   

   Use the same FQDN provided during the installation process..

3. Access the UI in the browser.

   URL: https://<user-provided-fqdn>

   Enter the following parameters to log in and view the Insight Dashboard.

   • Username: Application username
   • Password: Application password

   For more information about the default credentials, refer to the Release Notes.

6 - Accessing the PPC CLI

The deployment includes a CLI container that provides command-line access to the Protegrity Management CLI using SSH, on both Linux and Windows.

Prerequisites

• SSH key: The private key generated during bootstrap at ~/.ssh/<cluster_name>_user_svc, matches the public key configured in the pty-cli pod.

• Network access: Ensure you have connectivity to the AKS cluster’s ingress Load Balancer.

• Hosts file: Same as Web UI access. Map the ingress FQDN to the Load Balancer IP.

The private key is placed under ~/.ssh/<cluster_name>_user_svc after bootstrap completes, where <cluster_name> is the AKS cluster name provided during installation.

• Linux
• Windows

ssh -i ~/.ssh/<cluster_name>_user_svc -p 22 ptyitusr@<your-fqdn>

ssh -i ~/.ssh/<cluster_name>_user_svc \
    -o StrictHostKeyChecking=no \
    -o UserKnownHostsFile=/dev/null \
    -p 22 ptyitusr@<your-fqdn>

ssh -i C:\path\to\<cluster_name>_user_svc -p 22 ptyitusr@<your-fqdn>

CLI usage

Once connected, the Protegrity CLI welcome banner is displayed. Enter the following parameters when prompted:

• Username: Application username
• Password: Application password

For more information about the default credentials, refer to the Release Notes.

The CLI supports three command categories:

• pim: Policy Information Management commands for data protection policies.
• admin: User, role, permission, group, and email management commands.
• insight: Log forwarding to external SIEM and syslog servers.

Note: Ensure that at least one additional backup administrator user is configured with the same administrative privileges as the primary admin user.
If the primary admin account is locked or its credentials are lost, restoring the system from a backup is the only recovery option.

7 - Deleting PPC

Prerequisites

Authentication

The deletion process uses OpenTofu to destroy Azure resources, such as AKS clusters, managed identities, storage accounts, and Key Vaults. The Azure CLI must be authenticated to perform these operations.

Note: While using temporary credentials, such as service principal tokens or managed identity tokens, ensure that the credentials have not expired before proceeding.

To authenticate and set the correct subscription, run the following commands.

az login
az account set --subscription <subscription-id>

To verify that the credentials are valid and the correct Azure subscription is being used, run the following command.

az account show

Required Resource Providers

The following Azure resource providers must be registered in the subscription before deletion can proceed. If any provider is missing, the destroy operation may fail when attempting to manage the associated resources.

• Microsoft.ContainerService
• Microsoft.Storage
• Microsoft.KeyVault
• Microsoft.ManagedIdentity
• Microsoft.Network
• Microsoft.Compute

Required permissions

The following permissions are required to delete the PPC cluster. Create a custom role with these permissions and assign it to the managed identity used for deletion.

{
  "Name": "<ROLE-NAME>",
  "Description": "",
  "AssignableScopes": [
    "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RESOURCE-GROUP>"
  ],
  "Actions": [
    "Microsoft.Storage/storageAccounts/listKeys/action",
    "Microsoft.Resources/subscriptions/resourceGroups/read",
    "Microsoft.ContainerService/managedClusters/read",
    "Microsoft.ContainerService/managedClusters/listClusterUserCredential/action",
    "Microsoft.Storage/storageAccounts/read",
    "Microsoft.ContainerService/managedClusters/agentPools/read",
    "Microsoft.ManagedIdentity/userAssignedIdentities/read",
    "Microsoft.ContainerService/managedClusters/agentPools/delete",
    "Microsoft.ContainerService/managedClusters/delete",
    "Microsoft.Storage/storageAccounts/blobServices/read",
    "Microsoft.Storage/storageAccounts/fileServices/read",
    "Microsoft.Storage/storageAccounts/blobServices/containers/read",
    "Microsoft.Storage/storageAccounts/write",
    "Microsoft.Authorization/roleAssignments/delete",
    "Microsoft.Storage/storageAccounts/blobServices/containers/delete",
    "Microsoft.KeyVault/vaults/delete",
    "Microsoft.KeyVault/vaults/read",
    "Microsoft.Storage/storageAccounts/delete"
  ],
  "NotActions": [],
  "DataActions": [],
  "NotDataActions": []
}

To create the custom role and assign it to the managed identity, follow the steps in AKS User-Assigned Managed Identity Permissions.

Required tools

The following tools must be installed and available on the machine before proceeding with deletion:

• tofu (OpenTofu)
• kubectl (configured with cluster context)
• az CLI (authenticated)
• helm

Cleaning up the AKS Resources

There are two ways to destroy all created resources, including the AKS cluster and related components.

Method 1: Using the bootstrap script

From the folder where bootstrap script is present, run the following command:

./bootstrap.sh --cloud azure destroy

Method 2: Using OpenTofu to delete all modules

This method displays how to delete a PPC cluster on Azure by destroying the OpenTofu modules in reverse order. Destroying modules one at a time gives full control and visibility over each stage.

Warning: This process is irreversible. All cluster resources, data, and backups stored in the backup storage will be permanently deleted.

The modules are located at:

~/iac/azure/cluster/modules/
├── 00_backup_storage
├── 01_identity
├── 02_aks_cluster
├── 03_node_pool
├── 04_aks_addons
└── 05_helm_releases

These modules must be destroyed in reverse order: 05 → 04 → 03 → 02 → 01 → 00.

To delete the modules, perform the following steps:

1. To delete the helm releases resource, run the following command.

# Navigate to 05_helm_releases directory
cd /iac/azure/cluster/modules/05_helm_releases

# Initialize providers
tofu init  

# Delete the helm release
tofu destroy        # type "yes" when prompted

2. To delete the AKS Addons resource, run the following command.

# Navigate to 04_aks_addons directory
cd /iac/azure/cluster/modules/04_aks_addons

# Initialize providers
tofu init 

# Delete the aks addons
tofu destroy        # type "yes" when prompted

3. To delete the Node Pool resource, run the following command.

# Navigate to 03_node_pool directory
cd /iac/azure/cluster/modules/03_node_pool

# Initialize providers
tofu init 

# Delete the node pool
tofu destroy        # type "yes" when prompted

4. To delete the AKS Cluster resource, run the following command.

# Navigate to 02_aks_cluster directory
cd /iac/azure/cluster/modules/02_aks_cluster

# Initialize providers
tofu init 

# Delete the aks cluster
tofu destroy        # type "yes" when prompted

5. To delete the Identity resource, run the following command.

# Navigate to 01_identity directory
cd /iac/azure/cluster/modules/01_identity

# Initialize providers
tofu init 

# Delete the identity
tofu destroy        # type "yes" when prompted

6. To delete the Backup Storage (Backup storage account + container) resource, run the following command.

# Navigate to 00_backup_storage directory
cd /iac/azure/cluster/modules/00_backup_storage

# Initialize providers
tofu init 

# Destroy all resources (Storage Account, Key Vault, container, encryption key)
tofu destroy        # type "yes" when prompted

8 - Backing up the PPC

Protegrity Provisioned Cluster (PPC) supports backup of Insight indexes and cluster configurations. Use the procedures below to update the scheduled backups of your PPC deployment.

Backing up Velero backup schedule

To take a backup at an interval of 10 minutes, the cron job must be executed before the schedule. This ensures that the latest configurations are included and are available in the Storage Account.
By default, the cron job is executed after every 3 hours and schedule is executed after every 3.5 hours.

For taking the backups, the frequency of schedule and cron job can be changed, as required.

Perform the following steps to update the frequency.

1. To update the frequency of the cron job, run the following command.

   $ kubectl patch cronjob authnz-postgresql-backup-recovery-cron-job \
   -n api-gateway \'
   --type=merge -p '{"spec":{"schedule":"<required frequency>"}}'
   

2. To update the frequency of the schedule, run the following command.

   $ kubectl patch schedule authnz-postgresql-schedule-backup \   
   -n pty-backup-recovery \
   --type=merge -p '{"spec":{"schedule":"<required frequency>"}}' 
   

For instance, to update the frequency of the backup schedule to 10 minutes, perform the following steps.

1. To update the cron job to every 8 minutes, run the following command.

   $ kubectl patch cronjob authnz-postgresql-backup-recovery-cron-job \
    -n api-gateway \
    --type=merge -p '{"spec":{"schedule":"8-58/10 * * * *"}}'
   

2. To update the schedule to every 10 minutes, run the following command.

   $ kubectl patch schedule authnz-postgresql-schedule-backup \
    -n pty-backup-recovery \
    --type=merge -p '{"spec":{"schedule":"*/10 * * * *"}}'
   

Backing up indexes

Insight indexes are backed up automatically using a scheduled job. To manually back up the indexes, complete the following steps:

1. Log in to the Insight Dashboard.
2. Select the main menu.
3. Navigate to Management > Snapshot Management > Snapshot policies.
4. Click the daily-insight-snapshots policy.
5. Click Edit.
6. Update the following parameters:
   • Snapshot schedule
     • Frequency: Custom (Cron expression)
     • Cron expression: */15 * * * *
7. Click Update.
8. Wait for the next scheduled snapshot to complete. Verify that the snapshot is created from the snapshot status.

Note: The snapshot schedule is set to run every 15 minutes for creating the manual backup. Restore the default settings after the manual backup is complete. Before upgrading, disable the scheduler job after the backup is created. Ensure that the default parameters are restored and the job is re-enabled after the upgrade.

9 - Restoring the PPC

Before you begin

Before starting a restore, ensure the following conditions are met:

• An existing backup is available. Backups are taken automatically as part of the default installation using scheduled backup mechanisms. These backups are stored in a Storage Account configured during the original installation.

• Access to the original backup Storage Account. During restore, the same Storage Account that was used during the original installation must be specified.

• Permissions to read from the Storage Account. The user performing the restore must have sufficient permissions to access the backup data stored in the account.

• A new Kubernetes cluster is created. Restore is performed as part of creating a new cluster, not on an existing one. Restore is only supported during a fresh installation flow.

• While the backup is taken from the source cluster, do not perform Create, Read, Update, or Delete (CRUD) operations on the source cluster. This ensures backup consistency and prevents data corruption during restore.

• Before restoring to a new cluster, if the source cluster is accessible, disable the backup operations on the source cluster by setting the backup storage location to read‑only. This ensures that no additional backup data is written during the restore process.

  To disable the backup operation on the source cluster, run the following command:

  kubectl patch backupstoragelocation default -n pty-backup-recovery --type merge -p '{"spec":{"accessMode":"ReadOnly"}}'
  

  If the source cluster is not accessible, this step can be skipped.

During Disaster management, the backup data is used to restore the cluster and the OpenSearch indexes using snapshots. However, Insight restores OpenSearch data only from the most recent snapshot created by the daily-insight-snapshots policy.
For more information, refer to Backing up and restoring indexes.

Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform/OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior.
Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as ./bootstrap.sh --cloud azure destroy.

The repository provides a bootstrap script that automatically installs or updates the following software on the jump box:

• Azure CLI - Required to communicate with your Microsoft Azure account.
• OpenTofu - Required to manage infrastructure as code.
• kubectl - Required to communicate with the Kubernetes cluster.
• Helm - Required to manage Kubernetes packages.
• jq - Required to parse JSON.
• skopeo - Required to copy container images between registries.
• bc - Required for version comparisons and resource calculations.
• ORAS - Required to pull OCI artifacts such as deployment packages.
• ssh-keygen - Required to secure SSH access to deployment environments.
• cmctl - Required to validate and manage TLS certificates.

The bootstrap script also checks if you have the required permissions on Azure. It then sets up the AKS cluster and installs the microservices required for deploying the PCS on a PPC.

Note: Before running the bootstrap or resiliency scripts as the root user on RHEL, ensure that /usr/local/bin (and the Azure CLI binary path, if applicable) is included in the $PATH. Alternatively, run the script using a non-root user where /usr/local/bin is already part of the default PATH.

Architecture Selection

• amd64 (x86-64-based environments such as Standard_D8as_v5)
• arm64 (ARM-based environments such as Standard_D8ps_v5)

The bootstrap script prompts you to select the node architecture interactively during deployment (step 1b).

Restoring the PPC

Ensure that the PCT is downloaded and extracted on the jump box. For more information, refer to Preparing for PPC deployment.

Run the following command to initiate restore using an existing backup:

./bootstrap.sh --cloud azure --restore

The script prompts for the following variables.

1. Enter AKS Cluster Name [opencluster-v1]

   The following characters are allowed:

   • Lowercase letters: a-z
   • Numbers: 0-9
   • Hyphens: -

   The following characters are not allowed:

   • Uppercase letters: A-Z
   • Underscores: _
   • Spaces
   • Any special characters such as: / ? * + % ! @ # $ ^ & ( ) = [ ] { } : ; , .
   • Leading or trailing hyphens
   • Names must only contain up to 10 characters

   While creating a cluster, the cluster name must:

   • Start with a lowercase letter.
   • End with a letter or number.
   • Not contain consecutive hyphens.

   Note: Ensure that the cluster name does not exceed 10 characters. Cluster names longer than this limit can cause the bootstrap script to fail in subsequent installation steps.
   

   After the cluster name is accepted, the script prompts you to select the node architecture.

   [1b] Select Node Architecture

   Select the required architecture.

   [1b] Select Node Architecture
     1) amd64 (x86_64) — e.g. Standard_D8as_v5
     2) arm64 (ARM/Ampere) — e.g. Standard_D8ps_v5
   

   Enter 1 to select amd64 or 2 to select arm64.

   Note: Select amd64 for Standard_D8as_v5 environments. Select arm64 for ARM-based environments such as Standard_D8ps_v5.

2. Querying for available Resource Groups

   The script queries for the available Resource Groups.

   Enter the Resource Group name from the table []

   The script then automatically detects the location and subscription ID of the resource group.

3. Enter UAMI Resource ID

   Provide the complete Azure resource ID for the UAMI used by AKS in the following format:

   /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<identity-name>
   

   The UAMI client ID is detected automatically.

4. Enter AKS Subnet Resource ID

   Provide the complete resource ID of the pre-existing subnet used for AKS nodes in the following format:

   /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/virtualNetworks/<vnet-name>/subnets/<subnet-name>
   

5. Enter Private DNS Zone Resource ID

   Provide the Private DNS zone ID used by the AKS private cluster in the following format:

   /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/privateDnsZones/privatelink.<region>.azmk8s.io
   

   The script attempts to automatically detect network settings:

   • Virtual network address space
   • Service CIDR
   • Pod CIDR
   • DNS service IP

6. Enter FQDN [name.domain.com]

   This is the Fully Qualified Domain Name for the ingress.

   While entering the FQDN, ensure that:

   • contains at least two labels separated by a dot, such as, app.example.com
   • does not start or end with a dot or hyphen
   • does not contain consecutive dot character
   • label length must be between 1–63 characters
   • each label must start and end with a letter or digit
   • the last label must start only with a letter

   Warning: Ensure that the FQDN does not exceed 253 characters and only the following characters are used:

   • Lowercase letters: a-z
   • Numbers: 0-9
   • Special characters: - .

7. Storage Account and Key Vault provisioning

   During restore, the script prompts for existing Storage Account and Key Vault configuration. Provide the existing values in this step.

   Storage Account & Key Vault provisioning
   
   Restore mode requires existing Storage Account and Key Vault configuration.
   Skipping provisioning choice; provide existing values in this step.
   
   Enter existing Storage Account Name:
   <bucket-name>
   Storage account '<bucket-name>' found in resource group '<resource-group>'.
   Using container: velero
   
   Enter existing Key Vault Name:
   <key-vault-name>
   

   During restore, the script prompts to manually select a backup from the available backups stored in the Storage Account. User input is required to either restore from the latest backup or choose a specific backup from the list.

   [restore] Backup selection — schedule: authnz-postgresql-schedule-backup
   Restore from latest backup? [Y/n]
   

   • Enter Y to restore from the most recent backup.
   • Enter n to manually select a backup.

   If n is selected, then the script displays a list of available backups (latest first) and prompts to select one by number:

   Available backups (latest first):
   [1] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx
   [2] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx
   Select a backup number (1-2):
   

   After entering the backup number, the chosen backup is used for the restore, and the installation continues.

   Enter Insight backup

   During restore, the script prompts to manually select the required insight backup from the available backups stored in the Storage Account. User input is required to either restore from the latest backup or choose a specific backup from the list.

   [restore] Insight backup selection — schedule: insight-configmaps-secrets
   Restore from latest backup? [Y/n]
   

   • Enter Y to restore from the most recent backup.
   • Enter n to manually select a backup.

   If n is selected, then the script displays a list of available backups (latest first) and prompts to select one by number:

   Available backups (latest first):
   [1] <timestamp>  insight-configmaps-secrets-xxxxxxxxx
   [2] <timestamp>  insight-configmaps-secrets-xxxxxxxxx
   Select a backup number (1-2):
   

   After entering the backup number, the chosen backup is used for the restore, and the installation continues.

8. Enter Velero UAMI Resource ID

   Enter the resource ID of the dedicated Velero User-Assigned Managed Identity (UAMI) in the format:

   /subscriptions/<subscription-id>/resourceGroups/<velero-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name>
   

   The script automatically validates the UAMI and detects the UAMI Client ID.

9. Enter OpenSearch UAMI Resource ID

   Enter the resource ID of the OpenSearch Velero User-Assigned Managed Identity (UAMI) in the format:

   /subscriptions/<subscription-id>/resourceGroups/<opensearch-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name>
   

   The script automatically validates the UAMI and detects the UAMI Client ID.

10. Enter Image Registry Endpoint

The image repository from where the container images are retrieved.

Expected format: <hostname>[:port].

Do not include ‘https://’

11. Enter Registry Username

Enter the username for the registry mentioned in the previous step. Leave this entry blank if the registry does not require authentication.

12. Enter Registry Password or Access Token

Enter Password or Access Token for the registry.

Input is masked with * characters.

Leave this entry blank if the registry does not require authentication.

13. Enter Owner Email (for cluster tagging)

Enter email address in the format name@domain.tld. This email is used for the cluster ‘Owner’ tag and the platform owner contact.

Note: The cluster creation process can take approximately 15-20 minutes.

If the session is terminated during installation due to network issues, power outage, and so on, then the installation stops. To restart the installation, run the script again:

# Navigate to deployment directory and run the following command
./bootstrap.sh --cloud azure destroy

# Restore the cluster
./bootstrap.sh --cloud azure --restore

After the bootstrap script is completed, verify the cluster and workloads using the following commands:

# Confirm nodes are Ready
kubectl get nodes

# Confirm NFA workloads are Running
kubectl get pods -A

10 - Troubleshooting

Accessing the PPC CLI

• Permission denied for publickey: Ensure the correct private key ~/.ssh/<cluster_name>_user_svc is used and matches the authorized_keys in the pod.

• Connection refused: Verify the load balancer IP and hosts file configuration.

• Key format issues: Ensure the private key is in the correct format . For example, OpenSSH format for Linux or macOS, .ppk for PuTTY.

Component installation issues

• Helm chart not found: Run helm repo update to refresh the repository cache.

• Namespace already exists: Drop the --create-namespace flag if the namespace is already created.

• CRD conflicts: If cert-manager CRDs already exist, skip the CRD installation step.

• Pod not starting: Inspect logs with kubectl logs <pod> -n <namespace> and kubectl describe pod <pod> -n <namespace>.

State lock error after interrupted installation

Issue: When an installation is interrupted mid-apply (Ctrl+C, network drop, SSH disconnect, or jump box crash), subsequent installation attempts fail with the following error:

Error: Error acquiring the state lock
Error message: state blob is already locked
Lock Info:
  ID:        9377a903-94d1-4b19-3e57-1ee92b18bed5
  Path:      https://<storage>.blob.core.windows.net/velero/infrastructure-manifest%2F<cluster>%2Fstates%2F01_identity.tfstate
  Operation: OperationTypeApply

Description: OpenTofu acquires a lease on the state blob in Azure Blob Storage before applying changes. If the process is terminated without a graceful shutdown, the lease is not released automatically. This can occur due to Ctrl+C, network disconnection, SSH session timeout, or a jump box reboot.

Resolution: Force-unlock the state using the lock ID from the error message, then re-run the installation.

cd iac/azure/cluster/modules/<failed-stage>
tofu init
tofu force-unlock <lock-id>

Replace <failed-stage> with the module directory that failed (for example, 01_identity) and <lock-id> with the ID shown in the error output.

11 - Restoring the SSH keys and PCT from backed up Terraform state file

This section describes the procedure to rebuild a usable PPC v1.1.0 deployment workspace on a jump box from an existing cluster and a backed up Terraform state file.

Before you Begin

Ensure the following requirements are met.

• The state file from all the following modules must be available on the Storage Account.

  • 00_backup_bucket
  • 01_iam_roles
  • 02_eks_cluster
  • 03_node_group
  • 04_eks_addons
  • 05_helm_releases

• Access to the same Microsoft Azure account and target AKS cluster.

• Azure CLI credentials with required permissions.

Perform the following steps to recover PPC

1. Create a workspace.

   To create a working directory for PCT, run the following command.

   mkdir -p deployment
   cd deployment
   

2. Download and extract the PCT

   Download the PCT package into the deployment directory and extract it.
   For more information about downloading and extracting the PCT, refer to Preparing for PPC deployment.

   After successfully extracting the template, this directory behaves as recovered installation workspace.

3. Login to Microsoft Azure as a Service Principal

   To login to Microsoft Azure as a Service Principal, run the following command

   az login -u <username> -p <password> -t <tenant-id>
   

4. Install required dev tools

   Run the following script.

   cd deployment/bootstrap-scripts
   ./setup-devtools-azure.sh
   

5. Initialize OpenTofu

   To initialize OpenTofu for the cluster module, run the following command,

   cd deployment/iac/azure/cluster/modules/05_helm_releases
   tofu init
   

6. Generate the SSH key for CLI access

   To generate the SSH key for CLI access

   tofu output -raw user_svc_private_key > ~/.ssh/user_svc.pem
   chmod 600 ~/.ssh/user_svc.pem
   

7. Configure kubectl to communicate with the AKS cluster

   To configure kubectl to communicate with the AKS cluster, run the following command.

   az aks get-credentials \
   --resource-group aks-ppc \
   --name           base-test
   

8. Verify the nodes and pods status

   To verify the nodes and pods status, run the following command.

   kubectl get nodes
   kubectl get pods -A