Manual component-based upgrade

Before you begin

Before starting an upgrade, ensure the following conditions are met:

• Access to the jump box associated with the PPC v1.1.0 cluster is required. This must be the same jump box used during the PPC v1.1.0 installation.

• Create a Velero backup and an OpenSearch snapshot to ensure data recovery is possible.

• Ensure that the context is set to the cluster where the upgrade operation is to be performed. Run the following command:

  kubectl config current-context
  

  The cluster name in the output must match the cluster installed with PPC v1.1.0.

• Before running the bootstrap as the root user on RHEL, ensure that /usr/local/bin, and the AWS CLI binary path, if applicable, is included in the $PATH. Alternatively, run the script using a non-root user, such as ec2-user where /usr/local/bin is already part of the default $PATH.

• Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform or OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior.
  Use a dedicated directory, per cluster.

Required permissions

The IAM user performing the upgrade must have the same permissions as those required for installation.

For the full list of required IAM, S3, KMS, EC2, and EKS permissions, refer to Prerequisites.

Jump box requirement

Ensure that the entire process is performed using the same jump box.

Required container registry secrets

Export the container registry credentials as environment variables.

export TF_VAR_username='<registry-username>'
export TF_VAR_password='<registry-password>'

Parameter values

The following values are required before starting the upgrade. Configuration such as VPC, subnets, FQDN, and so on, are already stored in the terraform.tfvars files in S3 and does not need to be re-entered.

Access to terraform.tfvars file from PPC v1.1.0

The terraform.tfvars files for each module are stored in the AWS S3 backup bucket from the PPC v1.1.0 installation.

In the AWS Management Console, navigate to Amazon S3 > Buckets.
Select the S3 bucket used for the installation of PPC v1.1.0, then navigate to infrastructure-manifest > cluster-name > tfvars.
For each module, open the corresponding folder and select the terraform.tfvars file. Copy the AWS S3 URI shown on the file’s detail page. This is the URI required for the aws s3 cp commands in Modifying and applying terraform.tfvars files.

Upgrading PPC using component-based approach

To upgrade the PPC from v1.1.0 to v1.2.0 using the component-based approach, perform the following steps:

1. Exporting AWS credentials

The installation process uses Terraform to provision and manage AWS resources, such as VPCs, EKS clusters, IAM roles, and S3 buckets. The AWS credentials must be exported as environment variables to authenticate these operations.

Note: While using temporary credentials, such as, using AWS STS, ensure the session token is included and that the credentials have not expired before proceeding.
CLI login using AWS SSO (IAM Identity Center) is not supported. Export IAM user credentials or temporary STS credentials as environment variables.

If the components are installed by different users, ensure that each user must export their AWS credentials. This ensures that all the required access and permissions are available.

To export the AWS credentials, run the following commands:

export AWS_ACCESS_KEY_ID=<your-access-key-id>
export AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
export AWS_SESSION_TOKEN=<your-session-token>

2. Downloading and extracting the PCT

a. Create a deployment_120 directory on the jump box and navigate to it.

mkdir deployment_120 && cd deployment_120

b. Download and extract the PPC-K8S-ALL_1.2.0.x.tar PCT to the deployment_120 directory.

Note: For more information about the PCT, refer to My.Protegrity or the release notes.

3. Configuring container registry secrets

Configure the container registry pull secret based on your registry type:

• If using a private registry with authentication: The upgrade must recreate the Harbor pull secret. Navigate to /iac/aws/cluster/modules/05_helm_releases/ under the deployment_120 directory. Edit the terraform.tfvars file to update the value of harbor_secret parameter with harbor-secret.

• If the registry supports anonymous image pulls: No action is required. Ensure the harbor_secret value in the terraform.tfvars file is empty.

Note: Ensure that the value of the harbor_secret parameter in the 05_helm_releases module is the same in both installation and upgrade terraform.tfvars files.

4. Modifying and applying terraform.tfvars files

For each module, copy the terraform.tfvars from AWS S3 and save to the upgrade directory, then apply the Terraform configuration before proceeding to the next module.

To navigate to the modules directory, run the following command:

cd deployment_120/iac/aws/cluster/modules

00_backup_bucket

This module reconciles the existing S3 backup bucket state with the new working directory.

a. Navigate to the 00_backup_bucket directory.

cd 00_backup_bucket

b. Copy the terraform.tfvars from S3 and save to the upgrade 00_backup_bucket/terraform.tfvars.

aws s3 cp s3://<s3-bucket>/infrastructure-manifest/<cluster-name>/tfvars/00_backup_bucket/terraform.tfvars terraform.tfvars

c. Copy the terraform.tfstate from S3 to preserve the existing resource state inside 00_backup_bucket.

aws s3 cp s3://<s3-bucket>/infrastructure-manifest/<cluster-name>/states/00_backup_bucket.tfstate terraform.tfstate

d. To apply the terraform configuration, run the following command:

tofu init
tofu plan   # review the plan that will be created
tofu apply  # when prompted with "Enter a value:", type "yes"

e. Navigate to the bootstrap-scripts directory.

cd deployment_120/bootstrap-scripts

f. Run the following script to copy the state files to the S3 bucket:

./sync-backup-bucket-state-to-s3.sh

01_iam_roles

This module applies IAM role configuration for the EKS cluster, node group, and service accounts.

a. Navigate to the 01_iam_roles directory.

cd 01_iam_roles

b. Copy the terraform.tfvars from S3 and save to the upgrade 01_iam_roles/terraform.tfvars.

aws s3 cp s3://<s3-bucket>/infrastructure-manifest/<cluster-name>/tfvars/01_iam_roles/terraform.tfvars terraform.tfvars

c. To apply the terraform configuration, run the following command:

tofu init
tofu plan   # review the plan that will be created
tofu apply  # when prompted with "Enter a value:", type "yes"

02_eks_cluster

This module applies the EKS cluster, VPC security group, and networking add-ons configuration.

a. Navigate to the 02_eks_cluster directory.

cd 02_eks_cluster

b. Copy the terraform.tfvars from S3 and save to the upgrade 02_eks_cluster/terraform.tfvars.

aws s3 cp s3://<s3-bucket>/infrastructure-manifest/<cluster-name>/tfvars/02_eks_cluster/terraform.tfvars terraform.tfvars

c. To apply the terraform configuration, run the following command:

tofu init
tofu plan   # review the plan that will be created
tofu apply  # when prompted with "Enter a value:", type "yes"

03_node_group

This module applies the EKS worker node group configuration.

a. Navigate to the 03_node_group directory.

cd 03_node_group

b. Copy the terraform.tfvars from S3 and save to the upgrade 03_node_group/terraform.tfvars.

aws s3 cp s3://<s3-bucket>/infrastructure-manifest/<cluster-name>/tfvars/03_node_group/terraform.tfvars terraform.tfvars

c. To apply the terraform configuration, run the following command:

tofu init
tofu plan   # review the plan that will be created
tofu apply  # when prompted with "Enter a value:", type "yes"

04_eks_addons

This module applies EKS add-ons configuration including CoreDNS, Metrics Server, EBS CSI driver, and snapshot controller.

a. Navigate to the 04_eks_addons directory.

cd 04_eks_addons

b. Copy the terraform.tfvars from S3 and save to the upgrade 04_eks_addons/terraform.tfvars.

aws s3 cp s3://<s3-bucket>/infrastructure-manifest/<cluster-name>/tfvars/04_eks_addons/terraform.tfvars terraform.tfvars

c. To apply the terraform configuration, run the following command:

tofu init
tofu plan   # review the plan that will be created
tofu apply  # when prompted with "Enter a value:", type "yes"

05_helm_releases

This module applies all PPC platform Helm chart configurations.

a. Navigate to the 05_helm_releases directory.

cd 05_helm_releases

b. Copy the terraform.tfvars from S3 and save to the upgrade 05_helm_releases/terraform.tfvars.

aws s3 cp s3://<s3-bucket>/infrastructure-manifest/<cluster-name>/tfvars/05_helm_releases/terraform.tfvars terraform.tfvars

c. To apply the terraform configuration, run the following command:

tofu init
tofu plan   # review the plan that will be created
tofu apply  # when prompted with "Enter a value:", type "yes"

Post Upgrade Steps

1. Verify all nodes are Ready

To verify if all nodes are in the ready state, run the following command:

kubectl get nodes -o wide

2. Verify all pods are Running

To verify if all pods are in the running state, run the following command:

kubectl get pods -A

3. Verify Helm releases

To verify if the helm releases are in the deployed state, run the following command:

helm list -A

4. Directory Maintenance after Upgrade

During upgrade, both the previous and the new PPC installation directories must be retained.

• The original directory contains Terraform state and deployment metadata.
• The new directory contains the updated configuration and binaries.

Both directories are required for proper management of the deployment.

Reverting to a previous version of PPC

To revert to a previous version of PPC, restore the cluster using the steps in Restoring the PPC.