Upgrading the Protector from Version 10.x to 10.y
This section explains the steps and procedure for performing a rolling upgrade and roll back on a Kubernetes deployment consisting of pods. This method is useful for maintenance releases such as bug fixes and CVE updates. In this method, the protector is upgraded from version 10.x to version 10.y.
Before you begin
- Ensure that you have access to the Kubernetes cluster with appropriate permissions. For more information about the required permissions, refer to the section Software Requirements.
- Ensure that you have a separate directory structure for the 10.x and 10.y deployments.
- Ensure that your container logs are accessible. These can be used to verify the deployment.
- Ensure that the Container images for 10.y version are uploaded in the Container registry.
- Ensure that the protector pods for the 10.x version are running and are in a healthy state.
- Ensure that the required security policy is available on the 10.y ESA.
Rolling Upgrade Steps for Dynamic Deployment
This section explains how to perform a rolling upgrade for dynamic deployment.
Perform the following steps to upgrade the Log Forwarder.
i. Run the following command to check the 10.x Log Forwarder pods running on each node.
kubectl get pods
ii. Navigate to the 10.y directory and run the following command to upgrade the Log Forwarder pod.
helm -n v1 upgrade test-logforwarder-v1 logforwarder/ \
--atomic --timeout 2m \
--set imagePullSecrets[0].name="regcred" \
--set image.repository="<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set image.tag="LOGFORWARDER_RHUBI-9-64_x86-64_K8S_10.0.1.6.019e32.tgz" \
--set service.port=15780 \
--set opensearch[0].name="node-1" \
--set opensearch[0].host="10.49.7.212" \
--set opensearch[0].port="9200"
Ensure that the fields image.tag and image.repository are assigned appropriate values.
iii. Run the following command to get the daemonset value.
kubectl get daemonset -n v1
The following output appears.
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
test-logforwarder-v1 2 2 2 2 2 <none> 5h27m
iv. Run the following command to verify the rollout status.
kubectl rollout status daemonset test-logforwarder-v1 -n v1
The following output appears.
daemon set "test-logforwarder-v1" successfully rolled out
v. Run the following command to validate the pod status.
kubectl get pods -n <namespace>
The following output appears.
NAME READY STATUS RESTARTS AGE
test-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-logforwarder-v1-pms6f 1/1 Running 0 8h
Additionally, you can run kubectl describe pod to check the version from the latest image. After the upgrade is completed, validate that the logs are appearing on the Audit Store in the ESA.
Perform the following steps to upgrade the RPP pod.
i. Run the following command to upgrade the RPP pod.
helm -n v1 upgrade test-rpp-v1 rpproxy/ \
--atomic --timeout 2m \
--set imagePullSecrets[0].name="regcred" \
--set image.repository="<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set image.tag="RPPROXY_RHUBI-9-64_x86-64_K8S_1.8.1.8.0bba4b.tgz" \
--set commonCertSecrets="common-certs-v1" \
--set rpp.upstream.host="10.49.7.212" \
--set rpp.upstream.port="25400" \
--set rpp.logging.logLevel="DEBUG" \
--set rpp.logging.logHost="test-rpp-logforwarder-v1.v1.svc" \
--set rpp.logging.logPort="15780" \
--set rpp.service.cacheTTL="60"
Ensure that the fields image.tag and image.repository are assigned appropriate values.
ii. Run the following command to get the deployment value.
For example:
kubectl get deployment -n v1
The following output appears.
NAME READY UP-TO-DATE AVAILABLE AGE
test-rpp-v1-rpproxy 1/1 1 1 25h\
iii. Run the following command to verify the rollout status.
kubectl rollout status deployment test-rest-dynamic-10-v1-iap-rest-dynamic -n v1
The following output appears.
deployment "test-rest-dynamic-10-v1-iap-rest-dynamic" successfully rolled out
iv. Run the following command to get the pod details.
kubectl get pods -n <namespace>
The following output appears.
NAME READY STATUS RESTARTS AGE
test-rpp-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-rpp-logforwarder-v1-pms6f 1/1 Running 0 8h
test-rpp-v1-rpproxy-5f78d4f9f4-dnndb 1/1 Running 0 8h
v. Run the following command to validate the RPP pod details on the ESA after the upgrade procedure.
a. Log in to the ESA and navigate to Audit Store > Dashboard.
b. Navigate to Logs > Eventexplorer.
c. Change the logs search to DQL and change the filter to pty_insights_analytics*troubleshooting_*.
d. Search for <RPP Pod name>.
The origin IP mentioned should be updated to the latest pod after pod upgrade.
e. To get the pod IP, run the following command.
kubectl get pods -n <namespace> -o wide
Perform the following steps to upgrade the Protector pod.
i. Run the following command to upgrade the Protector pod.
helm -n v1 upgrade test-dynamic-10-v1 iap-rest-dynamic/ \
--atomic --timeout 2m \
--set iaprestImage.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set iaprestImage.tag="REST_RHUBI-9-64_x86-64_Generic.K8S.JRE-1.8_10.1.0+4.2e1243.tgz" \
--set protector.policy.cadence="60" \
--set protector.policy.host="test-rpp-v1-rpproxy.v1.svc" \
--set protector.policy.certificates="common-certs-v1" \
--set protector.logs.mode="error" \
--set protector.logs.host="test-rpp-logforwarder-v1.svc" \
--set service.type="LoadBalancer" \
--set iaprestService.type="LoadBalancer"
--set iaprestService.annotations."service\.beta\.kubernetes\.io\/aws-load-balancer-internal"=\"true\"
ii. Run the following command to get the deployment details.
kubectl get deployment -n v1
The following output appears.
NAME READY UP-TO-DATE AVAILABLE AGE
test-dynamic-10-v1-iap-rest-dynamic 1/1 1 1 25h
test-rpp-v1-rpproxy 1/1 1 1 25h
iii. Run the following command to get the rollout status.
kubectl rollout status deployment test-dynamic-10-v1-iap-rest-dynamic -n v1
The following output appears.
deployment "test-dynamic-10-v1-iap-rest-dynamic" successfully rolled out
iv. Run the following command to get the pod details.
kubectl get pods -n <namespace>
The following output appears.
NAME READY STATUS RESTARTS AGE
test-dynamic-10-v1-iap-rest-dynamic-6dcfd46c8d-dgqfv 2/2 Running 0 8h
test-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-logforwarder-v1-pms6f 1/1 Running 0 8h
test-v1-rpproxy-5f78d4f9f4-dnndb 1/1 Running 0 8h
Perform the following steps to verify the rollout upgrade.
i. Run the following command to verify that all the pods are running the new version.
kubectl get pods -n <namespace>
The following output appears.
NAME READY STATUS RESTARTS AGE
test-dynamic-10-v1-iap-rest-dynamic-6dcfd46c8d-dgqfv 2/2 Running 0 8h
test-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-logforwarder-v1-pms6f 1/1 Running 0 8h
test-v1-rpproxy-5f78d4f9f4-dnndb 1/1 Running 0 8h
ii. Run the following command to verify the updated image tag.
kubectl describe pod <pod name> -n <namespace>
The following output appears.
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 46m default-scheduler Successfully assigned rpp-v1/test-rpp-v1-rpproxy-5f78d4f9f4-cphhh to ip-10-49-5-188.ec2.internal
Normal Pulling 46m kubelet Pulling image "<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container:RPPROXY_RHUBI-9-64_x86-64_K8S_1.9.3.8.ec81ce.tgz"
Normal Pulled 46m kubelet Successfully pulled image "<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container:RPPROXY_RHUBI-9-64_x86-64_K8S_1.9.3.8.ec81ce.tgz" in 3.612s (3.612s including waiting). Image size: 40588092 bytes.
Normal Created 46m kubelet Created container: pty-rpproxy
Normal Started 46m kubelet Started container pty-rpproxy
iii. Run the following command to view the rollout history.
helm history <deploymentname> -n <namespace>
The following output appears.
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Mon Dec 1 09:58:30 2025 superseded rpproxy-1.0.0 1.9.3.8.xxxxxx Install complete
2 Mon Dec 1 10:15:01 2025 deployed rpproxy-1.0.0 1.9.3.8.xxxxxx Upgrade complete
iv. Run the following command to get the service details.
kubectl get svc -n <Namespace>
For example:
kubectl get svc -n iap-rest
The following output appears.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
logforwarder ClusterIP 172.20.14.88 <none> 15780/TCP 2m37s
rpproxy ClusterIP 172.20.181.92 <none> 443/TCP 113s
test-rest-10-v1-iap-rest LoadBalancer 172.20.60.61 internal-a70jkfsdf98908.us-east-1.elb.amazonaws.com 8080:30746/TCP 24s
v. Run the following command to validate the service of the pod.
kubectl get endpoints <service-name> -n <namespace>
For example:
kubectl get endpoints test-rest-10-v1-iap-rest -n 10-v2
The following output appears.
NAME ENDPOINTS AGE
test-rest-10-v1-iap-rest 10.49.10.xxx:9080 22h
Rolling Upgrade Steps for Static Deployment
This section explains how to perform rolling upgrade for static deployment.
Perform the following steps to upgrade the Log Forwarder.
i. Run the following command to check the Log Forwarder pods running on each node.
kubectl get pods
ii. Run the following command to upgrade the Log Forwarder pod.
helm -n v1 upgrade test-logforwarder-v1 logforwarder/ \
--atomic --timeout 2m \
--set imagePullSecrets[0].name="regcred" \
--set image.repository="<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set image.tag="LOGFORWARDER_RHUBI-9-64_x86-64_K8S_10.0.1.6.019e32.tgz" \
--set service.port=15780 \
--set opensearch[0].name="node-1" \
--set opensearch[0].host="10.49.7.212" \
--set opensearch[0].port="9200"
Ensure that the fields image.tag and image.repository are assigned appropriate values.
iii. Run the following command to get the daemonset value.
kubectl get daemonset -n v1
The following output appears.
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
test-logforwarder-v1 2 2 2 2 2 <none> 5h27m
iv. Run the following command to verify the rollout status.
kubectl rollout status daemonset test-logforwarder-v1 -n v1
The following output appears.
daemon set "test-logforwarder-v1" successfully rolled out
v. Run the following command to validate the pod status.
kubectl get pods -n <namespace>
The following output appears.
NAME READY STATUS RESTARTS AGE
test-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-logforwarder-v1-pms6f 1/1 Running 0 8h
Additionally, you can run kubectl describe pod to check the version from the latest image. After the upgrade is completed, validate that the logs are appearing on the Audit Store in the ESA.
Perform the following steps to upgrade the KMS-Proxy pod.
i. Run the following command to upgrade the KMS-Proxy pod.
helm -n devops-10-v2 upgrade test-kms-10-v1 kms-proxy/ \
--atomic --timeout 2m \
--set imagePullSecrets[0].name="regcred" \
--set image.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set image.tag="KMSPROXY_RHUBI-9-64_x86-64_K8S_1.0.0.11.31d6f0.tgz" \
--set serviceAccount.name="kms-v1-sa" \
--set kms.vendor="AWS" \
--set kms.keyid="arn:aws:kms:us-east-1:<AWS_ID>:key/c4be5e1a-fbdd-4a8e-aed6-0202d806274f" \
--set kms.ttl="1200" \
--set application.logLevel="INFO" \
--set service.certificates="pty-certs-secret"
Ensure that the fields image.tag and image.repository are assigned appropriate values.
ii. Run the following command to get the deployment details.
kubectl get deployment -n 10-v1
The following output appears.
NAME READY UP-TO-DATE AVAILABLE AGE
test-kms-10-v1-kms-proxy 1/1 1 1 18d
iii. Run the following command to check the rollout status.
kubectl rollout status deployment test-kms-10-v1-kms-proxy -n 10-v1
The following output appears.
deployment "test-kms-10-v1-kms-proxy" successfully rolled out
iv. Run the following command to validate the pod status.
kubectl get pods -n <namespace>
Perform the following steps to upgrade the Protector pod.
i. Run the following command to upgrade the Protector pod.
helm -n v1 upgrade test-static-10-v1 iap-rest-static/ \
--atomic --timeout 2m \
--set iaprestImage.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set iaprestImage.tag="REST_RHUBI-9-64_x86-64_Generic.K8S.JRE-1.8_10.1.0+4.2e1243.tgz" \
--set protector.policy.cadence="60" \
--set protector.policy.host="test-kms-v1-kmsproxy.v1.svc" \
--set protector.policy.certificates="common-certs-v1" \
--set protector.logs.mode="error" \
--set protector.logs.host="test-kms-logforwarder-v1.v1.svc" \
--set service.type="LoadBalancer" \
--set iaprestService.type="LoadBalancer"
--set iaprestService.annotations."service\.beta\.kubernetes\.io\/aws-load-balancer-internal"=\"true\"
ii. Run the following command to get the deployment details.
kubectl get deployment -n v1
The following output appears.
NAME READY UP-TO-DATE AVAILABLE AGE
test-static-10-v1-iap-rest-static 1/1 1 1 25h
test-kms-v1-kmsproxy 1/1 1 1 25h
iii. Run the following command to get the rollout status.
kubectl rollout status deployment test-dynamic-10-v1-iap-rest-dynamic -n v1
The following output appears.
deployment "test-static-10-v1-iap-rest-static" successfully rolled out
iv. Run the following command to get the pod details.
kubectl get pods -n <namespace>
The following output appears.
NAME READY STATUS RESTARTS AGE
test-static-10-v1-iap-rest-static-6dcfd46c8d-dgqfv 2/2 Running 0 8h
test-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-logforwarder-v1-pms6f 1/1 Running 0 8h
test-v1-kmsproxy-5f78d4f9f4-dnndb 1/1 Running 0 8h
Perform the following steps to verify the rollout upgrade.
i. Run the following command to verify that all the pods are running the new version.
kubectl get pods -n <namespace>
The following output appears.
NAME READY STATUS RESTARTS AGE
test-static-10-v1-iap-rest-static-6dcfd46c8d-dgqfv 2/2 Running 0 8h
test-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-logforwarder-v1-pms6f 1/1 Running 0 8h
test-v1-kmsproxy-5f78d4f9f4-dnndb 1/1 Running 0 8h
ii. Run the following command to verify the updated image tag.
kubectl describe pod <pod name> -n <namespace>
The following output appears.
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 46m default-scheduler Successfully assigned v1/test-kms-v1-kmsproxy-5f78d4f9f4-cphhh to ip-10-49-5-188.ec2.internal
Normal Pulling 46m kubelet Pulling image "<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container:KMSPROXY_RHUBI-9-64_x86-64_K8S_1.9.3.8.ec81ce.tgz"
Normal Pulled 46m kubelet Successfully pulled image "<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container:KMSPROXY_RHUBI-9-64_x86-64_K8S_1.9.3.8.ec81ce.tgz" in 3.612s (3.612s including waiting). Image size: 40588092 bytes.
Normal Created 46m kubelet Created container: pty-kmsproxy
Normal Started 46m kubelet Started container pty-kmsproxy
iii. Run the following command to view the rollout history.
helm history <deploymentname> -n <namespace>
The following output appears.
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Mon Dec 1 09:58:30 2025 superseded kmsproxy-1.0.0 1.9.3.8.xxxxxx Install complete
2 Mon Dec 1 10:15:01 2025 deployed kmsproxy-1.0.0 1.9.3.8.xxxxxx Upgrade complete
iv. Run the following command to get the service details.
kubectl get svc -n <Namespace>
For example:
kubectl get svc -n iap-rest
The following output appears.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
logforwarder ClusterIP 172.20.14.88 <none> 15780/TCP 2m37s
kmsproxy ClusterIP 172.20.181.92 <none> 443/TCP 113s
test-static-10-v1-iap-rest-static LoadBalancer 172.20.60.61 internal-a70jkfsdf98908.us-east-1.elb.amazonaws.com 8080:30746/TCP 24s
v. Run the following command to validate the service of the pod.
kubectl get endpoints <service-name> -n <namespace>
For example:
kubectl get endpoints test-static-10-v1-iap-rest-static -n 10-v2
The following output appears.
NAME ENDPOINTS AGE
test-static-10-v1-iap-rest-static 10.49.10.xxx:9080 22h
Rollback Steps
This section explains how to roll back the upgrade.
Order of Rollback
This section explains the order of rolling back an upgrade in case of dynamic and static deployments.
Roll back the Dynamic Deployment
Perform the following steps to roll back a dynamic deployment.
Roll back the Protector deployment.
Roll back the RPP deployment.
Roll back the Log Forwarder deployment.
Roll back the Static Deployment
Perform the following steps to roll back a static deployment.
Roll back the Protector deployment.
Roll back the KMS-Proxy deployment.
Roll back the Log Forwarder deployment.
Rolling Back a Deployment
If any deployment fails during the upgrade process, then the --atomic flag ensures that the deployment is automatically rolled back to the previous deployment.
If the deployment is successful but you still want to rollback, then perform the following steps to roll back the deployment. You can use these steps to roll back the Protector, RPP, KMS-Proxy, and Log Forwarder deployments.
- Run the following command to obtain the revision number of the deployment to which you want to roll back your current deployment.
helm history <deployment name> -n <namespace>
The following output appears.
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Mon Dec 1 09:58:30 2025 superseded rpproxy-1.0.0 1.9.3.8.xxxxxx Install complete
2 Mon Dec 1 10:15:01 2025 deployed rpproxy-1.0.0 1.9.3.8.xxxxxx Upgrade complete
Note down the revision number of the deployment to which you want to roll back.
- Run the following command to roll back to the specific revision number.
helm rollback <deployment name> <revision-number> -n <namespace>
- Run the following command to verify that the deployment has been rolled back to the specified revision number.
helm history <deploymentname> -n <namespace>
The following output appears.
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Mon Dec 1 09:58:30 2025 superseded rpproxy-1.0.0 1.9.3.8.xxxxxx Install complete
2 Mon Dec 1 10:15:01 2025 superseded rpproxy-1.0.0 1.9.3.8.xxxxxx Upgrade complete
3 Tue Dec 2 12:04:43 2025 deployed rpproxy-1.0.0 1.9.3.8.xxxxxx Rollback to 1
Perform the following steps to verify the deployment after rollback.
i. Run the following command to ensure that all the pods are running the previous stable version.
kubectl get pods -n <namespace>
The following output appears for the dynamic deployment.
NAME READY STATUS RESTARTS AGE
test-dynamic-10-v1-iap-rest-dynamic-6dcfd46c8d-dgqfv 2/2 Running 0 8h
test-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-logforwarder-v1-pms6f 1/1 Running 0 8h
test-v1-rpproxy-5f78d4f9f4-dnndb 1/1 Running 0 8h
The following output appears for the static deployment.
NAME READY STATUS RESTARTS AGE
test-static-10-v1-iap-rest-static-6dcfd46c8d-dgqfv 2/2 Running 0 8h
test-logforwarder-v1-6nc8m 1/1 Running 0 8h
test-logforwarder-v1-pms6f 1/1 Running 0 8h
test-v1-kmsproxy-5f78d4f9f4-dnndb 1/1 Running 0 8h
ii. Run the following command to verify the pod details.
kubectl describe pod <pod name> -n <namespace>
The following output appears if you run the kubectl describe pod command for dynamic deployment.
iaprest-dynamic:
Container ID: containerd://37855b0e6dc0387215b03d3aeac6676479225cbb1b5a84556c41e160743145eb
Image: <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/container:REST_RHUBI-10-v10-1-5
The following output appears if you run the kubectl describe pod command for static deployment.
iaprest-devops:
Container ID: containerd://37855b0e6dc0387215b03d3aeac6676479225cbb1b5a84556c41e160743145eb
Image: <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/container:REST_RHUBI-10-v10-1-5
Feedback
Was this page helpful?