Restoring the PPC

Complete the steps provided in this section to restore a PPC deployment using an existing backup.

Before you begin

Before starting a restore, ensure the following conditions are met:

  • An existing backup is available. Backups are taken automatically as part of the default installation using scheduled backup mechanisms. These backups are stored in a Storage Account configured during the original installation.

  • Access to the original backup Storage Account. During restore, the same Storage Account that was used during the original installation must be specified.

  • Permissions to read from the Storage Account. The user performing the restore must have sufficient permissions to access the backup data stored in the account.

  • A new Kubernetes cluster is created. Restore is performed as part of creating a new cluster, not on an existing one. Restore is only supported during a fresh installation flow.

  • While the backup is taken from the source cluster, do not perform Create, Read, Update, or Delete (CRUD) operations on the source cluster. This ensures backup consistency and prevents data corruption during restore.

  • Before restoring to a new cluster, if the source cluster is accessible, disable the backup operations on the source cluster by setting the backup storage location to read‑only. This ensures that no additional backup data is written during the restore process.

    To disable the backup operation on the source cluster, run the following command:

    kubectl patch backupstoragelocation default -n pty-backup-recovery --type merge -p '{"spec":{"accessMode":"ReadOnly"}}'

    If the source cluster is not accessible, this step can be skipped.

During Disaster management, the backup data is used to restore the cluster and the OpenSearch indexes using snapshots. However, Insight restores OpenSearch data only from the most recent snapshot created by the daily-insight-snapshots policy.
For more information, refer to Backing up and restoring indexes.

Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform/OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior.
Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as ./bootstrap.sh --cloud azure destroy.

The repository provides a bootstrap script that automatically installs or updates the following software on the jump box:

  • Azure CLI - Required to communicate with your Microsoft Azure account.
  • OpenTofu - Required to manage infrastructure as code.
  • kubectl - Required to communicate with the Kubernetes cluster.
  • Helm - Required to manage Kubernetes packages.
  • jq - Required to parse JSON.
  • skopeo - Required to copy container images between registries.
  • bc - Required for version comparisons and resource calculations.
  • ORAS - Required to pull OCI artifacts such as deployment packages.
  • ssh-keygen - Required to secure SSH access to deployment environments.
  • cmctl - Required to validate and manage TLS certificates.

The bootstrap script also checks if you have the required permissions on Azure. It then sets up the AKS cluster and installs the microservices required for deploying the PCS on a PPC.

Note: Before running the bootstrap or resiliency scripts as the root user on RHEL, ensure that /usr/local/bin (and the Azure CLI binary path, if applicable) is included in the $PATH. Alternatively, run the script using a non-root user where /usr/local/bin is already part of the default PATH.

Architecture Selection

  • amd64 (x86-64-based environments such as Standard_D8as_v5)
  • arm64 (ARM-based environments such as Standard_D8ps_v5)

The bootstrap script prompts you to select the node architecture interactively during deployment (step 1b).

Restoring the PPC

Ensure that the PCT is downloaded and extracted on the jump box. For more information, refer to Preparing for PPC deployment.

Run the following command to initiate restore using an existing backup:

./bootstrap.sh --cloud azure --restore

The script prompts for the following variables.

  1. Enter AKS Cluster Name [opencluster-v1]

    The following characters are allowed:

    • Lowercase letters: a-z
    • Numbers: 0-9
    • Hyphens: -

    The following characters are not allowed:

    • Uppercase letters: A-Z
    • Underscores: _
    • Spaces
    • Any special characters such as: / ? * + % ! @ # $ ^ & ( ) = [ ] { } : ; , .
    • Leading or trailing hyphens
    • Names must only contain up to 10 characters

    While creating a cluster, the cluster name must:

    • Start with a lowercase letter.
    • End with a letter or number.
    • Not contain consecutive hyphens.

    Note: Ensure that the cluster name does not exceed 10 characters. Cluster names longer than this limit can cause the bootstrap script to fail in subsequent installation steps.

    After the cluster name is accepted, the script prompts you to select the node architecture.

    [1b] Select Node Architecture

    Select the required architecture.

    [1b] Select Node Architecture
  1) amd64 (x86_64) — e.g. Standard_D8as_v5
  2) arm64 (ARM/Ampere) — e.g. Standard_D8ps_v5

    Enter 1 to select amd64 or 2 to select arm64.

    Note: Select amd64 for Standard_D8as_v5 environments. Select arm64 for ARM-based environments such as Standard_D8ps_v5.

  2. Querying for available Resource Groups

    The script queries for the available Resource Groups.

    Enter the Resource Group name from the table []

    The script then automatically detects the location and subscription ID of the resource group.

  3. Enter UAMI Resource ID

    Provide the complete Azure resource ID for the UAMI used by AKS in the following format:

    /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<identity-name>

    The UAMI client ID is detected automatically.

  4. Enter AKS Subnet Resource ID

    Provide the complete resource ID of the pre-existing subnet used for AKS nodes in the following format:

    /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/virtualNetworks/<vnet-name>/subnets/<subnet-name>

  5. Enter Private DNS Zone Resource ID

    Provide the Private DNS zone ID used by the AKS private cluster in the following format:

    /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/privateDnsZones/privatelink.<region>.azmk8s.io

    The script attempts to automatically detect network settings:

    • Virtual network address space
    • Service CIDR
    • Pod CIDR
    • DNS service IP

  1. Enter FQDN [name.domain.com]

    This is the Fully Qualified Domain Name for the ingress.

    While entering the FQDN, ensure that:

    • contains at least two labels separated by a dot, such as, app.example.com
    • does not start or end with a dot or hyphen
    • does not contain consecutive dot character
    • label length must be between 1–63 characters
    • each label must start and end with a letter or digit
    • the last label must start only with a letter

    Warning: Ensure that the FQDN does not exceed 253 characters and only the following characters are used:

    • Lowercase letters: a-z
    • Numbers: 0-9
    • Special characters: - .

  2. Storage Account and Key Vault provisioning

    During restore, the script prompts for existing Storage Account and Key Vault configuration. Provide the existing values in this step.

    Storage Account & Key Vault provisioning

Restore mode requires existing Storage Account and Key Vault configuration.
Skipping provisioning choice; provide existing values in this step.

Enter existing Storage Account Name:
<bucket-name>
Storage account '<bucket-name>' found in resource group '<resource-group>'.
Using container: velero

Enter existing Key Vault Name:
<key-vault-name>

    During restore, the script prompts to manually select a backup from the available backups stored in the Storage Account. User input is required to either restore from the latest backup or choose a specific backup from the list.

    [restore] Backup selection — schedule: authnz-postgresql-schedule-backup
Restore from latest backup? [Y/n]
    • Enter Y to restore from the most recent backup.
    • Enter n to manually select a backup.

    If n is selected, then the script displays a list of available backups (latest first) and prompts to select one by number:

    Available backups (latest first):
[1] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx
[2] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx
Select a backup number (1-2):

    After entering the backup number, the chosen backup is used for the restore, and the installation continues.

    Enter Insight backup

    During restore, the script prompts to manually select the required insight backup from the available backups stored in the Storage Account. User input is required to either restore from the latest backup or choose a specific backup from the list.

    [restore] Insight backup selection — schedule: insight-configmaps-secrets
Restore from latest backup? [Y/n]
    • Enter Y to restore from the most recent backup.
    • Enter n to manually select a backup.

    If n is selected, then the script displays a list of available backups (latest first) and prompts to select one by number:

    Available backups (latest first):
[1] <timestamp>  insight-configmaps-secrets-xxxxxxxxx
[2] <timestamp>  insight-configmaps-secrets-xxxxxxxxx
Select a backup number (1-2):

    After entering the backup number, the chosen backup is used for the restore, and the installation continues.

  3. Enter Velero UAMI Resource ID

    Enter the resource ID of the dedicated Velero User-Assigned Managed Identity (UAMI) in the format:

    /subscriptions/<subscription-id>/resourceGroups/<velero-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name>

    The script automatically validates the UAMI and detects the UAMI Client ID.

  4. Enter OpenSearch UAMI Resource ID

    Enter the resource ID of the OpenSearch Velero User-Assigned Managed Identity (UAMI) in the format:

    /subscriptions/<subscription-id>/resourceGroups/<opensearch-resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name>

    The script automatically validates the UAMI and detects the UAMI Client ID.

  5. Enter Image Registry Endpoint

The image repository from where the container images are retrieved.

Expected format: <hostname>[:port].

Do not include ‘https://’

  1. Enter Registry Username

Enter the username for the registry mentioned in the previous step. Leave this entry blank if the registry does not require authentication.

  1. Enter Registry Password or Access Token

Enter Password or Access Token for the registry.

Input is masked with * characters.

Leave this entry blank if the registry does not require authentication.

  1. Enter Owner Email (for cluster tagging)

Enter email address in the format name@domain.tld. This email is used for the cluster ‘Owner’ tag and the platform owner contact.

Note: The cluster creation process can take approximately 15-20 minutes.

If the session is terminated during installation due to network issues, power outage, and so on, then the installation stops. To restart the installation, run the script again:

# Navigate to deployment directory and run the following command
./bootstrap.sh --cloud azure destroy

# Restore the cluster
./bootstrap.sh --cloud azure --restore

After the bootstrap script is completed, verify the cluster and workloads using the following commands:

# Confirm nodes are Ready
kubectl get nodes

# Confirm NFA workloads are Running
kubectl get pods -A

Last modified : July 29, 2026