Troubleshoot

This section helps you diagnose and resolve common issues with an AI Gateway deployment. Use the following guidance to inspect pods, services, policy configuration, and scanner connectivity.

Common Issues

The following sections describe frequently encountered problems and how to resolve them.

One or more pods not reaching the Running state

Inspect the affected pod’s events and logs to identify why it failed to start, such as scheduling or image pull errors.

kubectl get pods -n federated-guardrails
kubectl describe pod -n federated-guardrails <POD_NAME>
kubectl logs -n federated-guardrails <POD_NAME> --all-containers --tail=100

Check that Karpenter provisioned the required nodes:

kubectl get nodes -l federated-guardrails.protegrity.com/deployment=true

API returns 401 Unauthorized

The gateway could not verify the caller’s identity token. Confirm that:

• The request includes a JWT in the x-user-auth header (or the header configured by FG_PRINCIPAL_TOKEN_HEADER).
• The token’s iss claim exactly matches an issuer in the configured IdP registry.
• The token is not expired and its signature and audience match the IdP registry.

For setup instructions, refer to External IdP Setup.

API returns 403 Forbidden

The configured policy may have blocked the request, or the caller may not have the required PPC permission. Check the response for gateway policy details, confirm that x-team-id selects the expected profile, and inspect gateway logs:

kubectl logs -n federated-guardrails deploy/federated-guardrails --all-containers --tail=100

If PPC rejects the request before it reaches the gateway, verify that the PPC user’s role includes ai_gateway_access.

Scanner or classification service is not reachable

Check that chart-managed scanner pods are running:

kubectl get pods -n federated-guardrails
kubectl get pods -n semantic-guardrails-ns
kubectl get svc -n semantic-guardrails-ns

The Classification Service is an external deployment prerequisite. Verify its availability in the data-discovery namespace:

kubectl get pods -n data-discovery
kubectl get svc -n data-discovery

Check the gateway logs for connection errors and confirm that the scanner endpoints in the Helm-managed ConfigMap match the services available in the cluster.

OpenTofu reports AccessDenied

The AWS credentials may not have permission to describe the EKS cluster or provision the configured Karpenter resources. Verify the required AWS permissions with your platform administrator.

OpenTofu reports that EC2NodeClass or NodePool CRDs are missing

Karpenter may not be installed or its custom resource definitions may not be registered. Verify the Karpenter installation and CRDs in the target cluster before rerunning tofu apply.

Helm upgrade fails with an immutable field error

Review the Helm error and compare the requested change with the existing release. Do not uninstall the release as a first response; uninstalling can remove deployed components or data. Contact your platform administrator or Protegrity Support if the update cannot be applied safely.

For additional troubleshooting guidance, refer to Protegrity Support.