Upgrading the Protector from Version 9.x to 10.x
This section explains the steps and procedure to upgrade the REST Container protector from version 9.x to 10.x. This method is used for a major release upgrade. For example, this upgrade procedure is used in case of architectural changes.
Upgrade Approach
The 9.x and 10.x versions include different components and resource requirements as part of the deployment. As a result, the approach uses the following steps:
- Create a 10.x setup in a different namespace.
- Run test traffic to the 10.x setup to verify that the security operations are working.
- Stop the traffic to the 9.x setup and make changes to point the traffic to the 10.x setup.
- Switch the production traffic from the 9.x deployment to the 10.x deployment.
Before you begin
- Ensure that you have access to the Kubernetes cluster with appropriate permissions. For more information about the required permissions, refer to the section Software Requirements.
- Ensure that you have a separate directory structure for the 9.x and 10.x deployments.
- Ensure that your container logs are accessible. These can be used to verify the deployment.
- Ensure that the Container images for 10.x version are uploaded in the Container registry.
- Ensure that the protector pods for the 9.x version are running and are in a healthy state.
- Ensure that the required security policy is available on the 10.x ESA.
Upgrading the Protector in Dynamic Mode
Perform the following steps to upgrade the protector from 9.x to 10.x in dynamic mode.
- Install 10.x Log Forwarder.
helm -n test-v1 install test-rpp-logforwarder-v1 logforwarder/ \
--set imagePullSecrets[0].name="regcred" \
--set image.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set image.tag="LOGFORWARDER_RHUBI-9-64_x86-64_K8S_10.0.1.6.019e32.tgz" \
--set service.port=15780 \
--set opensearch[0].name="node-1" \
--set opensearch[0].host="10.49.7.212" \
--set opensearch[0].port="9200"
Ensure that the set image.tag and set image.repository fields are assigned the appropriate values.
For more information about installing Log Forwarder, refer to the section Installing the Protector.
- Install 10.x RPP.
helm -n rpp-v1 install test-rpp-v1 rpproxy/ \
--set imagePullSecrets[0].name="regcred" \
--set image.repository="<AWS_ID >.dkr.ecr.us-east-1.amazonaws.com/container" \
--set image.tag="RPPROXY_RHUBI-9-64_x86-64_K8S_1.8.1.8.0bba4b.tgz" \
--set commonCertSecrets="common-certs-v1" \
--set rpp.upstream.host="10.49.7.212" \
--set rpp.upstream.port="25400" \
--set rpp.logging.logLevel="DEBUG" \
--set rpp.logging.logHost="test-rpp-logforwarder-v1.rpp-v1.svc" \
--set rpp.logging.logPort="15780" \
--set rpp.service.cacheTTL="60"
Ensure that the set image.tag and set image.repository fields are assigned the appropriate values.
For more information about installing RPP, refer to the section Installing the Protector.
Validate the RPP pod details on the ESA after installation.
a. Log in to the ESA and navigate to Audit Store > Dashboard.
b. Navigate to Logs > Eventexplorer.
c. Change the logs search to
DQLand change the filter topty_insights_analytics*troubleshooting_*.d. Search for <RPP pod name>.
The origin IP mentioned should be updated to the latest pod after the pod upgrade.
e. To get the pod IP , run the following command.
kubectl get pods -n <namespace> -o wideInstall 10.x Protector using the following command.
helm -n rpp-v1 install test-dynamic-10-v1 iap-rest-dynamic/ \
--set iaprestImage.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set iaprestImage.tag="REST_RHUBI-9-64_x86-64_K8S_10.0.0.34.22f868.tgz" \
--set nginxImage.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set nginxImage.tag="nginx-unprivileged-1.28" \
--set protector.policy.cadence="60" \
--set protector.policy.host="test-rpp-v1-rpproxy.rpp-v1.svc" \
--set protector.policy.certificates="common-certs-v1" \
--set protector.logs.mode="error" \
--set protector.logs.host="test-rpp-logforwarder-v1.rpp-v1.svc" \
--set service.certificates="pty-secret" \
--set service.type="LoadBalancer" \
--set service.port="443" \
--set service.annotations."service\.beta\.kubernetes\.io\/aws-load-balancer-internal"=\"true\"
- Run the following command to check the status of the pods.
kubectl get pods -n <Namespace>
For example:
kubectl get pods -n iap-rest
The following output appears.
NAME READY STATUS RESTARTS AGE
iap-rest-dynamic-7b97d5dff7-grqph 2/2 Running 0 11h
log1-logforwarder-f6gvj 1/1 Running 0 11h
log1-logforwarder-ls4hn 1/1 Running 0 11h
log1-logforwarder-phk4t 1/1 Running 0 11h
log1-logforwarder-z2mz7 1/1 Running 0 11h
rpp-rpproxy-5fd7d859b6-p9544 1/1 Running 0 11h
- Run the following command to obtain the service details.
kubectl get svc -n <Namespace>
For example:
kubectl get svc -n iap-rest
The following output appears.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
logforwarder ClusterIP 172.20.14.88 <none> 15780/TCP 2m37s
rpproxy ClusterIP 172.20.181.92 <none> 25400/TCP 113s
iap-rest-dynamic LoadBalancer 172.20.60.61 internal-a70jkfsdf98908.us-east-1.elb.amazonaws.com 8080:30746/TCP 24s
Use the DNS name of the load balancer that appears in the EXTERNAL-IP column while running the security operations.
For more information about running security operations, refer to the section Application Protector API on REST.
Run the following command to obtain the IP address of the Load Balancer.
ping <DNS of Load Balancer>For example:
ping internal-b70jkfs23423jg8.us-east-1.elb.amazonaws.comThe following output appears that displays the IP address of the Load Balancer.
PING internal-b70jkfs23423jg8.us-east-1.elb.amazonaws.com (10.49.5.152) 56(84) bytes of data. 64 bytes from ip-10-49-5-152.ec2.internal (10.49.5.152): icmp_seq=1 ttl=255 time=0.831 ms 64 bytes from ip-10-49-5-152.ec2.internal (10.49.5.152): icmp_seq=2 ttl=255 time=0.262 msUse this IP address while running the security operations.
Validate the service of pod as mentioned below
kubectl get endpoints <service-name> -n <namespace>
For example:
kubectl get endpoints test-sampleapp-10-v1-iap-rest -n 10-v2
Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
NAME ENDPOINTS AGE
test-sampleapp-10-v1-iap-rest 10.49.6.229:8443 9m7s
Verify that the IP address mentioned in the output is the same one that you get after running the kubectl get pods command.
- Run test protect and unprotect operations and verify functionality.
For more information about running security operations, refer to the section Application Protector API on REST.
- Validate the Audit logs on the ESA.
a. Login to ESA and navigate to Audit Store > Dashboard.
b. Navigate to Logs > Eventexplorer.
c. Change the logs search to DQL.
d. Refresh the page to sync up the logs.
e. Verify that the logs for the security operations performed in step 10 are displayed.
- If the 10.x deployment is working, then switch the production traffic to 10.x and monitor the traffic and scaling pods. If everything is working, then bring down the 9.x deployment.
Upgrading the Protector in Static Mode
Perform the following steps to upgrade the protector from 9.x to 10.x in static mode.
- Install 10.x Log Forwarder.
helm -n test-v1 install test-rpp-logforwarder-v1 logforwarder/ \
--set imagePullSecrets[0].name="regcred" \
--set image.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set image.tag="LOGFORWARDER_RHUBI-9-64_x86-64_K8S_10.0.1.6.019e32.tgz" \
--set service.port=15780 \
--set opensearch[0].name="node-1" \
--set opensearch[0].host="10.49.7.212" \
--set opensearch[0].port="9200"
Ensure that the set image.tag and set image.repository fields are assigned the appropriate values.
For more information about installing Log Forwarder, refer to the section Installing the Protector.
- Install the KMS Pod using the following command.
helm -n devops-10-v2 install test-kms-10-v1 kms-proxy/ \
--set imagePullSecrets[0].name="regcred" \
--set image.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set image.tag="KMSPROXY_RHUBI-9-64_x86-64_K8S_1.0.0.11.31d6f0.tgz" \
--set serviceAccount.name="kms-v1-sa" \
--set kms.vendor="AWS" \
--set kms.keyid="arn:aws:kms:us-east-1:<AWS_ID>:key/c4be5e1a-fbdd-4a8e-aed6-0202d806274f" \
--set kms.ttl="1200" \
--set application.logLevel="INFO" \
--set service.certificates="pty-certs-secret
For more information about installing the KMS Proxy Container, refer to the section Installing the Protector.
- Install 10.x Protector using the following command.
helm -n v1 install test-dynamic-10-v1 iap-rest-dynamic/ \
--set iaprestImage.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set iaprestImage.tag="REST_RHUBI-9-64_x86-64_K8S_10.0.0.34.22f868.tgz" \
--set nginxImage.repository="<AWS_ID>.dkr.ecr.us-east-1.amazonaws.com/container" \
--set nginxImage.tag="nginx-unprivileged-1.28" \
--set protector.policy.cadence="60" \
--set protector.policy.host="test-rpp-v1-rpproxy.rpp-v1.svc" \
--set protector.policy.certificates="common-certs-v1" \
--set protector.logs.mode="error" \
--set protector.logs.host="test-rpp-logforwarder-v1.rpp-v1.svc" \
--set service.certificates="pty-secret" \
--set service.type="LoadBalancer" \
--set service.port="443" \
--set service.annotations."service\.beta\.kubernetes\.io\/aws-load-balancer-internal"=\"true\"
- Run the following command to check the status of the pods.
kubectl get pods -n <Namespace>
For example:
kubectl get pods -n iap-rest
NAME READY STATUS RESTARTS AGE
iap-rest-static-7b97d5dff7-grqph 2/2 Running 0 11h
log1-logforwarder-f6gvj 1/1 Running 0 11h
log1-logforwarder-ls4hn 1/1 Running 0 11h
log1-logforwarder-phk4t 1/1 Running 0 11h
log1-logforwarder-z2mz7 1/1 Running 0 11h
kms-proxy-5fd7d859b6-p9544 1/1 Running 0 11h
- Run the following command to obtain the service details.
kubectl get svc -n <Namespace>
For example:
kubectl get svc -n iap-rest
The following output appears.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
logforwarder ClusterIP 172.20.14.88 <none> 15780/TCP 2m37s
kms-proxy ClusterIP 172.20.181.92 <none> 443/TCP 113s
iap-rest-dynamic LoadBalancer 172.20.60.61 internal-a70jkfsdf98908.us-east-1.elb.amazonaws.com 8080:30746/TCP 24s
Use the DNS name of the load balancer that appears in the EXTERNAL-IP column while running the security operations.
For more information about running security operations, refer to the section Application Protector API on REST.
Run the following command to obtain the IP address of the Load Balancer.
ping <DNS of Load Balancer>For example:
ping internal-b70jkfs23423jg8.us-east-1.elb.amazonaws.comThe following output appears and displays the IP address of the Load Balancer.
PING internal-b70jkfs23423jg8.us-east-1.elb.amazonaws.com (10.49.5.152) 56(84) bytes of data. 64 bytes from ip-10-49-5-152.ec2.internal (10.49.5.152): icmp_seq=1 ttl=255 time=0.831 ms 64 bytes from ip-10-49-5-152.ec2.internal (10.49.5.152): icmp_seq=2 ttl=255 time=0.262 msUse this IP address while running the security operations.
Run the following command to validate the service of the pod.
kubectl get endpoints <service-name> -n <namespace>
For example:
kubectl get endpoints test-rest-10-v1-iap-rest -n 10-v2
The following output appears.
Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
NAME ENDPOINTS AGE
test-rest-10-v1-iap-rest 10.49.6.229:8443 9m7s
Verify that the IP address mentioned in the output is the same one that you get after running the kubectl get pods command.
- Run test protect and unprotect operations and verify functionality.
For more information about running security operations, refer to the section Application Protector API on REST.
- Validate the Audit logs on the ESA.
a. Login to ESA and navigate to Audit Store > Dashboard.
b. Navigate to Logs > Eventexplorer.
c. Change the logs search to DQL.
d. Refresh the page to sync up the logs.
e. Verify that the logs for the security operations performed in step 10 are displayed.
- If the 10.x deployment is working, then switch the production traffic to 10.x and monitor the traffic and scaling pods. If everything is working, then bring down the 9.x deployment.
Rolling Back the Upgrade Procedure
Perform the following steps to roll back any failed upgrade procedure:
Ensure the 9.x deployment is running succesfully.
Ensure that the IP address of the 9.x service is updated in the hosts file or the Client configuration and switch the traffic back to 9.x.
Delete the failing 10.x deployment.
Feedback
Was this page helpful?