Restoring the PPC
Before you begin
Before starting a restore, ensure the following conditions are met:
An existing backup is available. Backups are taken automatically as part of the default installation using scheduled backup mechanisms. These backups are stored in an AWS S3 bucket configured during the original installation.
Access to the original backup AWS S3 bucket. During restore, the same S3 bucket that was used during the original installation must be specified.
Before initiating the restore, review and update the KMS key policy to reflect the restore cluster name. Even if the policy was already configured for the source cluster, it must be updated for the new restore cluster. If the policy continues to reference the source cluster name, the IAM role created during restore cannot decrypt the backup data, causing the restore to fail.
Permissions to read from the S3 bucket. The user performing the restore must have sufficient permissions to access the backup data stored in the bucket.
A Kubernetes cluster is created. Restore is performed as part of creating a cluster, not on an existing one. Restore is only supported during a fresh installation flow.
While the backup is taken from the source cluster, do not perform Create, Read, Update, or Delete (CRUD) operations on the source cluster. This ensures backup consistency and prevents data corruption during restore.
Before restoring to a new cluster, if the source cluster is accessible, disable the backup operations on the source cluster by setting the backup storage location to read‑only. This ensures that no additional backup data is written during the restore process.
To disable the backup operation on the source cluster, run the following command:
kubectl patch backupstoragelocation default -n pty-backup-recovery --type merge -p '{"spec":{"accessMode":"ReadOnly"}}'If the source cluster is not accessible, this step can be skipped.
During Disaster management, the backup data is used to restore the cluster and the OpenSearch indexes using snapshots. However, Insight restores OpenSearch data only from the most recent snapshot created by the daily-insight-snapshots policy.
For more information, refer to Backing up and restoring indexes.
Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform or OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior.
Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as./bootstrap.sh --cloud aws destroy.
The repository provides a bootstrap script that automatically installs or updates the following software on the jump box:
- AWS CLI - Required to communicate with your AWS account.
- bc - Required for version comparisons and resource calculations.
- cmctl - Required to validate and manage TLS certificates.
- Helm - Required to manage Kubernetes packages.
- jq - Required to parse JSON.
- kubectl - Required to communicate with the Kubernetes cluster.
- OpenTofu - Required to manage infrastructure as code.
- ORAS - Required to pull OCI artifacts such as deployment packages.
- skopeo - Required to copy container images between registries.
- ssh-keygen - Required to secure SSH access to deployment environments.
The bootstrap script also checks if you have the required permissions on AWS. It then sets up the EKS cluster and installs the microservices required for deploying the PCS on a PPC.
Note: Before running the bootstrap or resiliency scripts as the root user on RHEL, ensure that /usr/local/bin (and the AWS CLI binary path, if applicable) is included in the $PATH. Alternatively, run the script using a non-root user (such as ec2-user) where /usr/local/bin is already part of the default PATH.
Architecture Selection
- amd64 (x86-64-based environments such as standard Intel or AMD servers)
- arm64 (ARM-based environments such as AWS Graviton)
The bootstrap script prompts you to select the node architecture interactively during deployment (step 1b).
Restoring the PPC
Ensure that the PCT is downloaded and extracted on the jump box. For more information, refer to Preparing for PPC deployment.
Run the following command to initiate restore using an existing backup:
./bootstrap.sh --cloud aws --restore
To deploy PPC on ARM-based infrastructure, specify the architecture explicitly:
The bootstrap script asks for variables to be set to complete the deployment. Follow the instructions on the screen.
The --restore command enables the restore mode for the installation. It initiates restoration of data from the configured AWS S3 backup bucket. This process must be followed on a fresh jump box.
The script prompts for the following variables.
Enter Cluster Name
- Ensure that the cluster name does not match the name of the source cluster. Reusing an existing cluster name during restore can lead to discrepancies during cluster installation.
- This same cluster name must already be updated in the KMS key policy. If this update is not performed, the restore process fails because the new cluster cannot decrypt the backup data.
- Ensure that the cluster name does not exceed 31 characters. Cluster names longer than this limit can cause the bootstrap script to fail in subsequent installation steps.
If the installation fails because the cluster name exceeds the 31-character limit, correct the name and re-run the script.- Correction: Choose a cluster name with 31 characters or fewer.
- Retry: Execute the installation command again with the updated name. The script will automatically handle the update and proceed with the bootstrap process.
- Correction: Choose a cluster name with 31 characters or fewer.
The following characters are allowed:
- Lowercase letters:
a-z - Numbers:
0-9 - Hyphens:
-
The following characters are not allowed:
- Uppercase letters:
A-Z - Underscores:
_ - Spaces
- Any special characters such as:
/ ? * + % ! @ # $ ^ & ( ) = [ ] { } : ; , . - Leading or trailing hyphens
- More than 31 characters
After the cluster name is accepted, the script prompts you to select the node architecture.
[1b/10] Select Node Architecture
[1b/10] Select Node Architecture 1) amd64 (x86_64) — e.g. t3, m5, c5 instances 2) arm64 (Graviton) — e.g. t4g, m7g, c7g instancesEnter
1to selectamd64or2to selectarm64.Note: Select
amd64for standard Intel/AMD 64-bit environments. Selectarm64for ARM-based environments such as AWS Graviton instances.- Ensure that the cluster name does not match the name of the source cluster. Reusing an existing cluster name during restore can lead to discrepancies during cluster installation.
Enter a VPC ID from the table
The script automatically retrieves the available VPCs. Enter the VPC ID where the cluster must be created.
Querying for subnets in VPC…
The script queries for the available VPC subnets and prompts to enter two private subnet IDs. Specify two private subnet IDs from different availability zones.
The script then automatically updates the VPC CIDR block based on the VPC details.Enter FQDN
This is the Fully Qualified Domain Name (FQDN) for the ingress.
While entering the FQDN, ensure that:
- contains at least two labels separated by a dot, such as, app.example.com
- does not start or end with a dot or hyphen
- does not contain consecutive dot character
- label length must be between 1–63 characters
- each label must start and end with a letter or digit
- the last label must start only with a letter
Warning: Ensure that the FQDN does not exceed 50 characters and only the following characters are used:
- Lowercase letters:
a-z - Numbers:
0-9 - Special characters:
- .
Restore mode requires an existing S3 bucket. Proceeding to existing bucket validation in Step 6.
Select S3 Backup Bucket Option
An AWS S3 bucket encrypted with SSE‑KMS containing backup artifacts used during the restore process.
Use a dedicated S3 bucket per cluster for backup and restore operations to ensure data and encryption isolation. Sharing a bucket across clusters increases the risk of cross-cluster data access or decryption due to IAM misconfiguration. Dedicated buckets with unique IAM policies eliminate this risk.
Select backup
During restore, the script prompts to manually select a backup from the available backups stored in the S3 bucket. User input is required to either restore from the latest backup or choose a specific backup from the list.
[restore] Backup selection — schedule: authnz-postgresql-schedule-backup Restore from latest backup? [Y/n]- Enter Y to restore from the most recent backup.
- Enter n to manually select a backup.
If
nis selected, then the script displays a list of available backups (latest first) and prompts to select one by number:Available backups (latest first): [1] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx [2] <timestamp> authnz-postgresql-schedule-backup-xxxxxxx Select a backup number (1-2):After entering the backup number, the chosen backup is used for the restore, and the installation continues.
Enter Insight backup
During restore, the script prompts to manually select the required insight backup from the available backups stored in the S3 bucket. User input is required to either restore from the latest backup or choose a specific backup from the list.
[restore] Insight backup selection — schedule: insight-configmaps-secrets Restore from latest backup? [Y/n]- Enter Y to restore from the most recent backup.
- Enter n to manually select a backup.
If
nis selected, then the script displays a list of available backups (latest first) and prompts to select one by number:Available backups (latest first): [1] <timestamp> insight-configmaps-secrets-xxxxxxxxx [2] <timestamp> insight-configmaps-secrets-xxxxxxxxx Select a backup number (1-2):After entering the backup number, the chosen backup is used for the restore, and the installation continues.
Enter Image Registry Endpoint
The image repository from where the container images are retrieved.
Expected format:
<hostname>[:port].Do not include ‘https://’Note: The container registry endpoint must be a FQDN (Fully Qualified Domain Name). Sub-paths like, my-registry.com/v2/path, are not supported by the OCI distribution specification.
Enter Registry Username
Enter the username for the registry mentioned in the previous step. Leave this entry blank if the registry does not require authentication.
Enter Registry Password or Access Token
Enter Password or Access Token for the registry. Input is masked with
*characters. Press Enter to keep the current value.Leave this entry blank if the registry does not require authentication.
Enter Owner Email (for cluster tagging)
Enter email address in the format name@domain.tld. This email is used for the cluster ‘Owner’ tag and the platform owner contact.
Note: The cluster creation process can take approximately 15-20 minutes.
If the session is terminated during restore due to network issues, power outage, and so on, then the restore stops. To restart the process, run the following commands:
# Navigate to deployment directory and run the following command
./bootstrap.sh --cloud aws destroy
# Restore the cluster
./bootstrap.sh --cloud aws --restore
Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform or OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior.
Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as./bootstrap.sh --cloud aws destroy.
To check the active kubectl context, run the following command:kubectl config current-context
After the restore to the new cluster is completed successfully and all required validation and migration activities are finished, the source cluster can be deleted.
Feedback
Was this page helpful?