Upgrading the PPC
Before you begin
Before starting an upgrade, ensure the following conditions are met:
Access to the jump box associated with the PPC v1.0.0 cluster is required. This must be the same jump box used during the PPC v1.0.0 installation.
Maintain both, previous and current installation directories. Do not delete the original cluster deployment directory, as it contains the Terraform state files required for managing the deployment.
Create a Velero backup and an OpenSearch snapshot to ensure data recovery is possible, in case of an upgrade failure.
Maintain a backup of the state file in the AWS S3 backup bucket.
If logs are forwarded to an external SIEM, then from the PPC CLI, run the
insight delete syslogor theinsight delete fluentdas per your configuration. After the upgrade is complete, re-configure the SIEM. For more information about SIEM configuration, refer to Sending logs to a SIEM.The
contextis set to the cluster where the upgrade operation is to be performed. Run the following command.kubectl config current-contextThe cluster name in the output must match the cluster installed with PPC v1.0.0.
Note: Before running the bootstrap or resiliency scripts as the root user on RHEL, ensure that
/usr/local/bin, and the AWS CLI binary path, if applicable, is included in the $PATH. Alternatively, run the script using a non-root user, such as ec2-user where/usr/local/binis already part of the default $PATH.
Required Permissions
The IAM user performing the upgrade must have the same permissions as those required for installation.
For the full list of required IAM, S3, KMS, EC2, and EKS permissions, refer to Prerequisites.
Required Harbor Secrets
If harbor secrets are used during upgrade, then the harbor_secret parameter must be updated in the 05_helm_releases module with the following values.
export TF_VAR_username='<registry-username>'
export TF_VAR_password='<registry-password>'
export TF_VAR_harbor_secret='harbor-secret'
Note: Ensure that the value of the
harbor_secretparameter in the05_helm_releasesmodule is the same in both installation and upgradeterraform.tfvarsfiles.
For more information, refer to Step 8 in Upgrading PPC.
Configuring PPC for Non-FIPS Deployment
To deploy a standard, non-FIPS cluster, the following Terraform configuration files must be modified to use the standard (non-FIPS) Bottlerocket AMI types and SSM parameter paths before performing the upgrade process.
For a non-FIPS PPC upgrade on AWS EKS, perform the following steps:
Update the node group AMI type map in the
/03_node_group/variables.tffile.a. Navigate to
/03_node_group directory, using the following command.cd iac/aws/cluster/modules/03_node_group/b. Edit the
variables.tffile to make the following changes.Change from (FIPS defaults):
hcl variable "node_group_ami_type_map" { description = "Fallback EKS AMI type per architecture, used when node_group_ami_type is empty." type = map(string) default = { amd64 = "BOTTLEROCKET_x86_64_FIPS" arm64 = "BOTTLEROCKET_ARM_64_FIPS" } }Change to (non-FIPS values):
```hcl variable "node_group_ami_type_map" { description = "Fallback EKS AMI type per architecture, used when node_group_ami_type is empty." type = map(string) default = { amd64 = "BOTTLEROCKET_x86_64" arm64 = "BOTTLEROCKET_ARM_64" } } ```Update the Karpenter AMI SSM parameter path in the
/05_helm_releases/main.tffile.
The karpenter_fips_ami_ssm_parameter value specifies the AWS Systems Manager (SSM) parameter path that Karpenter uses to resolve the correct Bottlerocket AMI for worker nodes. The default path targets the FIPS AMI. For a non-FIPS cluster, remove the -fips segment from the EKS version portion of the path.
a. Navigate to /05_helm_releases directory.
cd iac/aws/cluster/modules/05_helm_releases/
b. Edit the main.tf file to make the following changes.
**Change from (FIPS SSM path):**
```hcl
karpenter_fips_ami_ssm_parameter = "/aws/service/bottlerocket/aws-k8s-${data.aws_eks_cluster.this.version}-fips/${local.karpenter_ami_arch}/latest/image_id"
```
**Change to (non-FIPS SSM path):**
```hcl
karpenter_fips_ami_ssm_parameter = "/aws/service/bottlerocket/aws-k8s-${data.aws_eks_cluster.this.version}/${local.karpenter_ami_arch}/latest/image_id"
```
Upgrading PPC
To upgrade the PPC from v1.0.0 to v1.1.0, perform the following steps:
Create a
deployment_110directory on the jump box and navigate to it.mkdir deployment_110 && cd deployment_110Log in to the My.Protegrity portal.
Navigate to Product Management > Explore Products > AI Team Edition > Platform & Features.
Navigate to Platform Installation.
From the Actions column for Protegrity Provisioned Cluster, click the Download Product icon and download the PPC 1.1 archive to the
deployment_110directory.Extract the archive contents.
tar -xvf PPC-K8S-ALL_1.1.0.97.tarCopy the Terraform state file from the v1.0.0 cluster installation to the
deployment_110directory created for upgrade.cp /<v1.0.0-cluster-dir>/deployment/iac_setup/scripts/iac/terraform.tfstate /<v1.1.0-cluster-dir>/deployment_110/terraform.tfstatewhere
<v1.0-cluster-dir>is the path to the original PPC v1.0.0 cluster installation directory on the jump box, and<v1.1.0-cluster-dir>is the path to the new directory created for upgrade.This step is required only if a private container registry that requires authentication is configured on the base cluster to retrieve images. Then, the upgrade process must recreate the Harbor pull secret.
This step is not required if the registry supports anonymous image pulls. Then the
harbor_secret = "harbor-secret"value must be empty.If harbor secrets are used for upgrading from PPC v1.0.0 to PPC v1.1.0, then navigate to
/iac/aws/cluster/modules/05_helm_releases/. Edit theterraform.tfvarsfile to update theharbor_secret = "harbor-secret"parameter with the following values.export TF_VAR_username='<registry-username>' export TF_VAR_password='<registry-password>' export TF_VAR_harbor_secret='harbor-secret'Note: Ensure that the value of the
harbor_secretparameter in the05_helm_releasesmodule is the same in both installation and upgradeterraform.tfvarsfiles.Verify the active context points to the correct cluster using the following command.
kubectl config current-context
- Initiate the upgrade from the
deployment_110directory using the following command.
bash upgrade.sh --cloud aws --v1.0-state-file /<v1.1.0-cluster-dir>/deployment_110/terraform.tfstate
where /<v1.1.0-cluster-dir>/deployment/terraform.tfstate is the path to the Terraform state file copied in Step 6.
After the upgrade is successful, a PPC upgrade completed successfully. message is displayed.
The cluster upgrade process can take 10–15 minutes approximately and may differ accordingly.
Post Upgrade Steps
Verify all nodes are Ready
To verify if all nodes are in the ready state, run the following command.
kubectl get nodes -o wide
Verify all pods are Running
To verify if all pods are in the running state, run the following command.
kubectl get pods -A
Verify Helm releases
To verify if the helm releases are in the deployed state, run the following command.
helm list -A
Password Policy After Upgrade
PPC v1.1.0 upgrade enforces the following behavior.
If the minimum password length was set to a value less than 14 characters, or was not configured, the upgrade sets it to 14 characters.
If the minimum password length was already set to 14 characters or greater, that value is preserved and remains unchanged after the upgrade.
For more information about the password policy, refer to Password Policy APIs or CLI reference for get password_policy.
Configuring SIEM
If logs were forwarded to an external SIEM before the upgrade, then SIEM log forwarding was disabled as part of the pre-upgrade prerequisites. After completing the upgrade, update the settings to resume sending logs to the external SIEM.
For more information about configuring SIEM, refer to Sending logs to an external security information and event management (SIEM).
Directory Maintenance after Upgrade
During upgrade, both the previous and the new PPC installation directories must be retained.
- The original directory contains Terraform state and deployment metadata.
- The new directory contains the updated configuration and binaries.
Both directories are required for proper management of the deployment.
Recovering from a Failed Upgrade
If the session is terminated during upgrade due to network issues, power outage, or any such issues, then the upgrade process stops. The ./upgrade.sh script resumes the upgrade.
To restart the process, run the following command:
bash upgrade.sh --cloud aws --v1.0-state-file /<v1.1.0-cluster-dir>/deployment_110/terraform.tfstate
Warning: Do not install or manage multiple clusters from the same working directory. Each cluster deployment maintains its own Terraform or OpenTofu state, and reusing a directory can overwrite state files, causing loss of cluster tracking and unintended cleanup behavior.
Use a dedicated directory, and jump box, where possible, per cluster, and always verify the active kubectl context before running cleanup commands such as./bootstrap.sh --cloud aws destroy.
To check the active kubectl context, run the following command:kubectl config current-context
Reverting to a previous version of PPC
To revert to a previous version of PPC, restore the cluster using the steps in Restoring the PPC.
Feedback
Was this page helpful?