Automated script-based upgrade

Before you begin

Before starting an upgrade, ensure the following conditions are met:

• Access to the jump box associated with the PPC v1.1.0 cluster is required. This must be the same jump box used during the PPC v1.1.0 installation.

• Create a Velero backup and an OpenSearch snapshot to ensure data recovery is possible, in case of an upgrade failure.

• Ensure that the context is set to the cluster where the upgrade operation is to be performed. Run the following command:

  kubectl config current-context
  

  The cluster name in the output must match the cluster installed with PPC v1.1.0.

• Before running the bootstrap scripts as the root user on RHEL, ensure that /usr/local/bin, and the Azure CLI binary path, if applicable, is included in the $PATH. Alternatively, run the script using a non-root user, such as azureuser, where /usr/local/bin is already part of the default $PATH.

• Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform or OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior.
  Use a dedicated directory, per cluster.

Required Permissions

The Azure principal performing the upgrade must have the following RBAC roles assigned.

Required container registry Secrets

Export the container registry credentials as environment variables.

export TF_VAR_username='<registry-username>'
export TF_VAR_password='<registry-password>'

Values to collect before starting

Gather the following values before running the upgrade command.

Upgrading PPC

The upgrade process downloads the latest images from the repository and deploys the updated containers to the target environment.

To upgrade the PPC from v1.1.0 to v1.2.0, perform the following steps:

1. Create a upgrade_120 directory on the jump box and navigate to it.

   mkdir upgrade_120 && cd upgrade_120
   

2. Download and extract the PCT to the upgrade_120 directory.

3. Authenticate the Azure CLI and select the target subscription. Terraform and the Azure CLI operations invoked by the upgrade script rely on an active az session.

   If the session has expired or you are not logged in, run the following commands to authenticate and confirm the active account:

   az login
   az account show
   

   Note: CLI login using Single Sign-On (SSO) is not supported. Use az login --use-device-code or authenticate as a Service Principal.

   Set the active subscription to the one that hosts the AKS cluster.

   az account set --subscription <subscription-ID>
   

   Replace the placeholder with the value collected in Values to collect before starting. Ensure the target subscription, resource group, managed identities, subnet, and private DNS zone are accessible before continuing.

4. Verify the active context points to the correct cluster using the following command:

    kubectl config current-context
   

5. Configure the container registry pull secret based on your registry type:

• If using a private registry with authentication: The upgrade must recreate the Harbor pull secret. Navigate to /iac/azure/cluster/modules/05_helm_releases/ under the deployment_120 directory. Edit the terraform.tfvars file to update the value of harbor_secret parameter with harbor-secret.

• If the registry supports anonymous image pulls: No action is required. Ensure the harbor_secret value in the terraform.tfvars file is empty.

Note: Ensure that the value of the harbor_secret parameter in the 05_helm_releases module is the same in both installation and upgrade terraform.tfvars files.

6. Initiate the upgrade from the upgrade_120 directory using the following command:

   ./upgrade.sh --cloud azure --resource-group <resource-group> --storage-account <storage-account-name> --subscription-id <subscription-ID> --cluster <cluster-name>
   

   Replace the placeholders with the environment-specific values.

After the upgrade is successful, a Upgrade complete for all stages message is displayed.

The upgrade typically takes 10–15 minutes approximately, but duration may vary.

Post Upgrade Steps

After the upgrade completes, perform the following verification steps to confirm the cluster is healthy.

1. Verify all nodes are Ready

To verify if all nodes are in the ready state, run the following command:

kubectl get nodes -o wide

2. Verify all pods are Running

To verify if all pods are in the running state, run the following command:

kubectl get pods -A

3. Verify Helm releases

To verify if the helm releases are in the deployed state, run the following command:

helm list -A

4. Directory Maintenance after Upgrade

During upgrade, both the previous and the new PPC installation directories must be retained.

• The original directory contains Terraform state and deployment metadata.
• The new directory contains the updated configuration and binaries.

Both directories are required for proper management of the deployment.

Reverting to a previous version of PPC

To revert to a previous version of PPC, restore the cluster using the steps in Restoring the PPC.