Protegrity AI Gateway

• 1: Introduction
• 2: AI Gateway Architecture
• 3: Installation
• 3.1: Prerequisites
• 3.2: Install
• 4: Gateway Configuration
• 5: Uninstallation
• 6: API Reference
• 6.1: Health
• 6.2: Administration
• 6.3: Policy
• 7: Usage
• 7.1: User and Identity Setup
• 7.2: Connecting to the Gateway
• 8: Troubleshoot

1 - Introduction

Organizations are rapidly adopting large language models (LLMs) to power applications, automate workflows, and extract insights from unstructured data. Sending sensitive prompts to external LLMs or exposing raw LLM responses to end users introduces data privacy, compliance, and security risks.

AI Gateway addresses these risks by acting as an AI security gateway between client applications and supported LLM APIs. Every outbound prompt and inbound response passes through configurable scanners before reaching its destination. Scanners can detect and block PII leakage, identify prompt injection attacks, enforce topic restrictions, and apply graduated responses from logging to redaction to blocking based on the Cedar policy managed by Policy.

The gateway supports OpenAI-compatible and Anthropic APIs. OpenAI-compatible client libraries and frameworks can be configured to use the gateway URL. AWS Bedrock protocol support has not been verified for this release and is not covered by this guide.

The AI Gateway umbrella chart installs the gateway and its chart-managed runtime components as one release. Sensitive Data Guardrails, Semantic Guardrails, and Vault-Based Tokenization are supporting components of that release, not separate AI Gateway products or standalone installation paths. The Classification Service is a separate deployment prerequisite.

2 - AI Gateway Architecture

Overview of AI Gateway architecture, processing flow, and deployment components on AWS EKS.

AI Gateway is deployed as an umbrella Helm chart that bundles the gateway and its chart-managed runtime components into a single release. Sensitive Data Guardrails and Semantic Guardrails are enabled by default as chart sub-charts. The gateway processes LLM requests by scanning outbound prompts before they reach the LLM and inbound responses before they are returned to the client.

AI Gateway uses Kubernetes for deployment at scale. This deployment guide covers AWS EKS.

Communication Ports

The following table lists the ports used for communication between the components.

Components

The umbrella chart deploys the following components into the Helm release namespace. The standard namespace is federated-guardrails.

• AI Gateway (federated-guardrails): Forms the core AI security gateway. Receives supported LLM API requests, runs configured outbound scanners (including PII tokenization and Semantic Guardrails checks), forwards policy-compliant requests to the upstream LLM, scans the inbound response, and returns the processed result to the client. Helm renders the scanner and routing configuration from policyConfig.policyYaml into the gateway ConfigMap. The gateway retrieves Cedar policy from Policy.

• Sensitive Data Guardrail (sensitive-data-guardrails): Performs reversible PII tokenization on outbound prompts and restores original values from LLM responses using the Vault-Based Tokenizer. Depends on the Classification Service for PII detection.

• Vault-Based Tokenization (vault-based-tokenization): Stores PII tokens in a Valkey cluster. Tokens are keyed by session and expire automatically via TTL.

• Valkey Cluster: High-availability key-value store for token persistence. Deployed as a sharded cluster (3 primaries by default).

• Semantic Guardrails: Provides PII and topic-level policy enforcement. It is deployed by the chart into the semantic-guardrails-ns namespace and accessed by the gateway.

• Classification Service (Data Discovery): Identifies PII spans in text. Deployed into the data-discovery namespace and accessed by both Sensitive Data Guardrails and Semantic Guardrails.

• PPC API Gateway: Provides the Envoy-based API gateway included in the Protegrity Provisioned Cluster (PPC). It terminates TLS and routes client requests to the AI Gateway API prefix, /pty/federated-guardrails/v1.

Policy Configuration

Helm renders the scanner and routing configuration into the gateway ConfigMap. The gateway retrieves the authoritative Cedar policy from Policy.

The gateway retrieves the authoritative Cedar policy from Policy during startup and, when configured, refreshes it on a background interval. The chart sets FG_POLICY_POLL_INTERVAL_S to 60 seconds.

• FG_POLICY_SYNC_URL: Policy endpoint exposed by the current Helm chart.
• FG_POLICY_POLL_INTERVAL_S: Background synchronization interval in seconds.

3 - Installation

The installation process includes preparing the cluster, provisioning the required infrastructure with OpenTofu, deploying the umbrella Helm chart, and verifying the gateway and its bundled dependencies.

3.1 - Prerequisites

Required tools, cloud permissions, and Kubernetes cluster configuration for deployment on Amazon EKS or Azure Kubernetes Service (AKS).

Ensure the following prerequisites are met:

Tools

Install and configure the following tools on the host from which you run the installation:

Install the CLI for your cloud provider:

• AWS
• Azure

Cloud Setup

Ensure that the following requirements are met:

• A Protegrity Provisioned Cluster (PPC) is available. For more information about PPC, refer to Protegrity Provisioned Cluster.
• The host from which you run the installation can connect to the required registry. If you are not already authenticated, log in to it.

Configure cloud credentials for your provider:

• AWS
• Azure

To authenticate to the Protegrity Container Registry, use the following command with credentials from the My.Protegrity portal during account creation:

echo "<password>" | helm registry login <Container_Registry_Path> --username "<username>" --password-stdin

Note: If deploying from the Protegrity Container Registry, use registry.protegrity.com:9443. Specify the local registry endpoint if a local registry is used.

Note: helm registry login authenticates Helm only. The OpenTofu module is pulled from the OCI registry and additionally requires container registry credentials in ~/.config/containers/auth.json. You configure these during the Install procedure before running tofu init.

Cluster Requirements

A standard Protegrity Provisioned Cluster (PPC) already satisfies the following requirements, so no additional setup is needed in that case. Use the checklist below to verify the cluster only if you are not deploying onto a standard PPC cluster.

• AWS
• Azure

External Service Prerequisites

The umbrella chart deploys Sensitive Data Guardrails, Vault-Based Tokenization, and Semantic Guardrails as part of the release. The Classification Service must already be available in the cluster:

Note: AI Gateway does not require cloud object storage, such as an S3 bucket or Azure Storage Account, or dedicated cloud data-access roles. The OpenTofu module provisions only the node isolation resources for workload scheduling.

Cloud Permissions for Installation

• AWS
• Azure

3.2 - Install

This section deploys the AI Gateway umbrella chart on a supported Kubernetes cluster, either Amazon EKS or Azure Kubernetes Service (AKS). OpenTofu provisions node isolation resources and writes a federated-guardrails-values.yaml file with scheduling overrides. Helm then deploys the full stack using that file.

The chart renders the scanner and routing configuration from policyConfig.policyYaml. The gateway retrieves the authoritative Cedar policy from Policy during startup and, when configured, refreshes it on a background interval.

Deployment Steps

Complete the prerequisites before starting. Then complete the following steps in order, selecting the tab for your cloud provider.

1. Provision Cloud Infrastructure with OpenTofu

The OpenTofu module sets up workload scheduling for each cloud and creates federated-guardrails-values.yaml with Helm settings. For AWS, it also creates the Karpenter EC2NodeClass and NodePool. For AKS, use the Azure module instead of the AWS configuration.

Configure OCI Registry Credentials

The OpenTofu module is pulled from the OCI registry, which requires container registry credentials in ~/.config/containers/auth.json. helm registry login authenticates Helm only and does not create this file, so configure these credentials before running tofu init.

1. Run the following command to create the configuration directory.

   mkdir -p ~/.config/containers
   

2. Obtain the username and access token from the My.Protegrity portal. For more information about obtaining the credentials from the My.Protegrity portal, refer to the section Configuring Authentication for Protegrity AI Team Edition.

3. Generate base64 encoded string with padding for username:accesstoken obtained from the My.Protegrity portal.

   Ensure that you specify the username and access token within single quotes when generating the base64 encoded value. For example, 'username:accesstoken'.

   Note: For the Protegrity Container Registry, set <Container_Registry_Path> to registry.protegrity.com:9443. For a local registry, set it to the local registry endpoint.

4. Create a file named ~/.config/containers/auth.json with the following content.

   {
       "auths": {
           "<Container_Registry_Path>": {
               "auth": "<base64 generated string from step-3>"
           }
       }
   }
   

Configure cloud access, create a directory with a single main.tf for your cloud provider, and then run the OpenTofu commands. Run the OpenTofu and Helm commands from that directory, keeping the same shell session so that your cloud environment variables remain available.

• AWS
• Azure

aws sts get-caller-identity

terraform {
  required_version = "~> 1.10"

  required_providers {
    aws = {
      source  = "registry.opentofu.org/hashicorp/aws"
      version = "~> 5.0"
    }
    kubernetes = {
      source  = "registry.opentofu.org/hashicorp/kubernetes"
      version = "~> 2.35"
    }
  }
}

variable "cluster_name" {
  type        = string
  description = "EKS cluster name."
}

variable "environment" {
  type    = string
  default = "production"
}

data "aws_eks_cluster" "this" {
  name = var.cluster_name
}

data "aws_eks_cluster_auth" "this" {
  name = var.cluster_name
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.this.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.this.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.this.token
}

module "federated_guardrails" {
  source = "oci://<Container_Registry_Path>/federated-guardrails/federated-guardrails/opentofu/federated-guardrails?tag=aws-1.0.0"

  cluster_name  = var.cluster_name
  oci_host      = "<Container_Registry_Path>"
  environment   = var.environment
  chart_version = "1.0.0"
}

output "helm_install" {
  value = module.federated_guardrails.helm_install
}

tofu init
tofu plan -var="cluster_name=<CLUSTER_NAME>"
tofu apply -var="cluster_name=<CLUSTER_NAME>"

az login

az account set --subscription "{{SUBSCRIPTION_ID}}"

export ARM_SUBSCRIPTION_ID="{{SUBSCRIPTION_ID}}"

az account show --query '{subscriptionId:id, subscriptionName:name}' --output table
az aks show \
  --resource-group "{{RESOURCE_GROUP}}" \
  --name "{{CLUSTER_NAME}}" \
  --query '{name:name, resourceGroup:resourceGroup, provisioningState:provisioningState}' \
  --output table

terraform {
  required_version = "~> 1.10"

  required_providers {
    azurerm = {
      source  = "registry.opentofu.org/hashicorp/azurerm"
      version = "~> 5.7"
    }
    kubernetes = {
      source  = "registry.opentofu.org/hashicorp/kubernetes"
      version = "~> 2.35"
    }
  }
}

variable "cluster_name" {
  type        = string
  description = "AKS cluster name."
}

variable "resource_group_name" {
  type        = string
  description = "Azure resource group that contains the AKS cluster."
}

variable "environment" {
  type    = string
  default = "production"
}

provider "azurerm" {
  features {}
  resource_provider_registrations = "none"
}

data "azurerm_kubernetes_cluster" "this" {
  name                = var.cluster_name
  resource_group_name = var.resource_group_name
}

provider "kubernetes" {
  host                   = data.azurerm_kubernetes_cluster.this.kube_config[0].host
  client_certificate     = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].client_certificate)
  client_key             = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].client_key)
  cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].cluster_ca_certificate)
}

module "federated_guardrails" {
  source = "oci://<Container_Registry_Path>/federated-guardrails/federated-guardrails/opentofu/federated-guardrails?tag=azure-1.0.0"

  cluster_name        = var.cluster_name
  resource_group_name = var.resource_group_name
  oci_host            = "<Container_Registry_Path>"
  environment         = var.environment
  chart_version       = "1.0.0"
}

output "helm_install" {
  value = module.federated_guardrails.helm_install
}

tofu init
tofu plan \
  -var="cluster_name={{CLUSTER_NAME}}" \
  -var="resource_group_name={{RESOURCE_GROUP}}"
tofu apply \
  -var="cluster_name={{CLUSTER_NAME}}" \
  -var="resource_group_name={{RESOURCE_GROUP}}"

Verify that federated-guardrails-values.yaml exists in the working directory. Display the generated Helm command:

tofu output -raw helm_install

Confirm that the command uses the intended chart version and namespace. Use the generated values file in the Helm deployment that follows.

Module variables

• AWS
• Azure

Note: If deploying from the Protegrity Container Registry, use registry.protegrity.com:9443. Specify the local registry endpoint if a local registry is used.

2. Deploy with Helm

The OpenTofu module writes a federated-guardrails-values.yaml file in the working directory. The chart defaults to an OpenAI upstream. Use an OpenAI API key for that default, or configure a different upstream in the policy. If you do not have a key during installation, use a placeholder and replace it before sending requests to the upstream LLM. The value is required only on the first install and is preserved automatically on upgrades.

The Helm commands are the same for AWS and Azure. Use the values file generated by the module for your cloud provider.

1. Configure the kubectl context for the target cluster:

• AWS
• Azure

aws eks update-kubeconfig \
  --region {{AWS_REGION}} \
  --name {{CLUSTER_NAME}}

az aks get-credentials \
  --resource-group "{{RESOURCE_GROUP}}" \
  --name "{{CLUSTER_NAME}}"

Verify that the current context points to the target cluster:

kubectl config current-context
kubectl get nodes

Confirm that the context identifies the intended cluster and that its nodes are Ready before continuing.

2. Authenticate to the container registry before running any Helm command:

helm registry login <Container_Registry_Path>

Note: If deploying from the Protegrity Container Registry, use registry.protegrity.com:9443. Specify the local registry endpoint if a local registry is used.

3. Configure the Policy endpoint and synchronization interval in the federated-guardrails-values.yaml file that the OpenTofu run generated in the working directory. Add the following block at the top level of the file. If a federated-guardrails: key already exists, merge the environment entries into it instead of adding a second key:

federated-guardrails:
  environment:
    FG_POLICY_SYNC_URL: "http://policy-api-service.pty-policy.svc.cluster.local:8000/pty/policy/v1.0/product-policies/federated-guardrails"
    FG_POLICY_POLL_INTERVAL_S: "60"

Policy must publish the Cedar policy for the federated-guardrails product. The gateway validates and caches the active policy in memory.

Note: FG_POLICY_SYNC_URL and FG_POLICY_POLL_INTERVAL_S control the synchronization for Policy.

4. Install the chart. On the first install, provide the upstream API key with --set:

export FG_UPSTREAM_API_KEY="placeholder-update-before-use"

helm upgrade --install federated-guardrails \
  oci://<Container_Registry_Path>/federated-guardrails/federated-guardrails/helm/federated-guardrails \
  --version 1.0.0 \
  --namespace federated-guardrails --create-namespace \
  --values federated-guardrails-values.yaml \
  --set federated-guardrails.auth.apiKey="$FG_UPSTREAM_API_KEY"

For upgrades, omit the --set flag because the API key is already stored in the cluster secret:

helm upgrade federated-guardrails \
  oci://<Container_Registry_Path>/federated-guardrails/federated-guardrails/helm/federated-guardrails \
  --version 1.0.0 \
  --namespace federated-guardrails \
  --values federated-guardrails-values.yaml

Note: If deploying from the Protegrity Container Registry, use registry.protegrity.com:9443. Specify the local registry endpoint if a local registry is used.

3. Monitor

1. Monitor the deployment process using the following command:

kubectl get pods -n federated-guardrails

Verify that all pods are in the Running state. The following is a sample output:

NAME                                                   READY   STATUS    RESTARTS   AGE
federated-guardrails-<hash>                            1/1     Running   0          3m10s
federated-guardrails-sensitive-data-guardrails-<hash>  1/1     Running   0          3m10s
federated-guardrails-vault-based-tokenization-<hash>   1/1     Running   0          3m10s
federated-guardrails-valkey-0                          1/1     Running   0          3m10s
federated-guardrails-valkey-1                          1/1     Running   0          3m10s
federated-guardrails-valkey-2                          1/1     Running   0          3m10s

2. Verify that all services are deployed.

kubectl get svc -n federated-guardrails

3. Run health checks for each component.

# AI Gateway
kubectl exec -n federated-guardrails deploy/federated-guardrails -- \
  curl -s http://localhost:8080/pty/federated-guardrails/v1/health

# Sensitive Data Guardrail
kubectl exec -n federated-guardrails deploy/federated-guardrails-sensitive-data-guardrails -- \
  curl -s http://localhost:8080/pty/sensitive-data-guardrails/v1/health

# Vault-Based Tokenization
kubectl exec -n federated-guardrails deploy/federated-guardrails-vault-based-tokenization -- \
  curl -s http://localhost:8000/api/v1/version

For more information about connecting client applications to the gateway, refer to Connecting to the Gateway.

4 - Gateway Configuration

AI Gateway uses a YAML policy to configure the upstream LLM, scanners, profiles, and request routing. In a Helm deployment, manage this policy through the chart values and gateway ConfigMap. Cedar authorization policy is retrieved from Policy.

The examples below are complete minimal policies. They use a local regex scanner so they do not require a remote scanner service. Choose one upstream configuration for the deployed policy.

OpenAI-Compatible Upstream

Set FG_UPSTREAM_API_KEY in the gateway environment, then use this policy as the YAML policy value:

version: "1.0"
settings:
  default_action: block
  timeout_ms: 5000
  fail_open: false
upstream:
  url: "https://api.openai.com/v1"
  protocol: openai
  headers:
    Authorization: "Bearer ${FG_UPSTREAM_API_KEY}"
scanners:
  prompt-injection:
    type: regex
    patterns:
      - "ignore previous instructions"
      - "disregard your system prompt"
profiles:
  default-protection:
    inbound:
      scanners: [prompt-injection]
      on_reject: block
    outbound:
      scanners: []
      on_reject: block
routing:
  - default: default-protection

The OpenAI SDK client key is not used as the upstream credential by this configuration; the gateway supplies the key from FG_UPSTREAM_API_KEY.

Anthropic Upstream

For an Anthropic upstream, use this complete policy and set FG_UPSTREAM_API_KEY to the Anthropic key:

version: "1.0"
settings:
  default_action: block
  timeout_ms: 5000
  fail_open: false
upstream:
  url: "https://api.anthropic.com"
  protocol: anthropic
  headers:
    x-api-key: "${FG_UPSTREAM_API_KEY}"
scanners:
  prompt-injection:
    type: regex
    patterns:
      - "ignore previous instructions"
      - "disregard your system prompt"
profiles:
  default-protection:
    inbound:
      scanners: [prompt-injection]
      on_reject: block
    outbound:
      scanners: []
      on_reject: block
routing:
  - default: default-protection

Important: In the current gateway image, the Anthropic protocol forwards the client’s x-api-key upstream. Provide a valid Anthropic API key in the Anthropic client. This differs from the OpenAI configuration above, where the gateway supplies the upstream key.

External Identity Provider

Configure trusted JWT issuers in the file identified by FG_IDP_CONFIG_PATH. For an example issuer registry and setup steps, refer to External IdP Setup. Requests carry the caller’s JWT in the header specified by FG_PRINCIPAL_TOKEN_HEADER, which defaults to x-user-auth.

Environment Variables

All gateway environment variables use the FG_ prefix. The following reference lists the runtime settings used by the gateway and the upstream-key variable used by the policy and chart. The Helm chart can provide deployment-specific values that differ from the application defaults.

Do not place secrets in policy files committed to source control or in command history. Supply secret values through the deployment’s approved secret-management mechanism. For the OpenAI-compatible default upstream, use an OpenAI key; if one is not available at installation time, use a placeholder and replace it before sending requests to the upstream service.

5 - Uninstallation

1. Configure kubectl and uninstall the Helm release

Configure kubectl for the cluster where AI Gateway is installed:

• AWS
• Azure

aws eks update-kubeconfig \
  --region {{AWS_REGION}} \
  --name {{CLUSTER_NAME}}

az aks get-credentials \
  --resource-group "{{RESOURCE_GROUP}}" \
  --name "{{CLUSTER_NAME}}"

Verify that the current context points to the intended cluster:

kubectl config current-context
kubectl get nodes

Remove the AI Gateway Helm release and all bundled sub-charts from the cluster:

helm uninstall federated-guardrails -n federated-guardrails

Optionally, delete the namespace after the Helm release is removed:

kubectl delete namespace federated-guardrails

2. Wait for AWS Karpenter to remove provisioned nodes

This step applies only to AWS deployments. Uninstalling the Helm release removes the gateway workloads, but the nodes that Karpenter provisioned for them still exist as NodeClaim resources. Karpenter must terminate these nodes and delete their NodeClaim resources before you destroy the NodePool and EC2NodeClass with OpenTofu. Azure deployments do not use this AWS Karpenter cleanup step.

Important: If you run tofu destroy while NodeClaim resources still exist, Karpenter can no longer reconcile them. This can orphan the backing cloud instances and leave NodeClaim resources stuck on finalizers, requiring manual cleanup.

1. Delete the NodeClaim resources provisioned for AI Gateway so Karpenter drains and terminates the backing nodes:

kubectl delete nodeclaims -l federated-guardrails.protegrity.com/deployment=true

2. Wait until no matching NodeClaim resources remain:

kubectl wait --for=delete nodeclaims -l federated-guardrails.protegrity.com/deployment=true --timeout=10m

Alternatively, confirm manually that the following command returns No resources found:

kubectl get nodeclaims -l federated-guardrails.protegrity.com/deployment=true

Proceed to the next step only after all matching NodeClaim resources are deleted.

3. Destroy OpenTofu infrastructure

Run tofu destroy from the same directory that contains the main.tf and OpenTofu state used during installation. Use the same variable values as the installation.

• AWS
• Azure

tofu destroy \
  -var="cluster_name={{CLUSTER_NAME}}" \
  -var="environment=production" \

az login
az account set --subscription "{{SUBSCRIPTION_ID}}"
export ARM_SUBSCRIPTION_ID="{{SUBSCRIPTION_ID}}"

tofu destroy \
  -var="cluster_name={{CLUSTER_NAME}}" \
  -var="resource_group_name={{RESOURCE_GROUP}}"

Review the destroy plan before approving the operation. By default, tofu destroy prompts for confirmation before deleting resources. Add -auto-approve to skip the prompt.

6 - API Reference

The AI Gateway API is exposed under /pty/federated-guardrails/v1.

Each API page lists the method, URL, input fields, and output fields for its endpoints. When an admin token is configured, send it as Authorization: Bearer <admin-token>. Mutating admin requests must also include X-Requested-With: XMLHttpRequest for CSRF protection. Cedar policy is managed by Policy; the Cedar endpoints expose or synchronize the active policy and are not local policy-file workflows.

The machine-readable contract is available here: Download the AI Gateway OpenAPI specification.

6.1 - Health

The health and readiness endpoints report the status of the gateway instance that handles the request. They are not cluster-wide checks; a request through a load-balanced service may reach any replica.

Health

Return the current health status of the gateway process.

Input Fields

None.

Output Fields

Ready

Return the current readiness status of the gateway process, including verification of the YAML policy, scanners, and policy state.

Input Fields

None.

Output Fields

Readiness status indicating whether the YAML policy, scanners, and policy state are acceptable.

Response Codes

The health and readiness endpoints return the following HTTP status codes to indicate the outcome of a request.

6.2 - Administration

Administration endpoints operate on one gateway instance at a time. A request routed through a load-balanced service may reach any replica; these endpoints do not provide cluster-wide state. When an admin token is configured, send Authorization: Bearer <admin-token>. POST, PUT, PATCH, and DELETE requests also require X-Requested-With: XMLHttpRequest.

Reload Policy

Reload the YAML policy on the gateway instance that receives the request.

Input Fields

None.

Output Fields

Policy reload status.

Circuit Breakers

Return the circuit-breaker status for each configured scanner on this instance.

Input Fields

None.

Output Fields

Circuit-breaker status for configured scanners on this instance.

SLA

Report SLA metrics for this instance only, including service availability and latency adherence.

Input Fields

None.

Output Fields

Per-instance SLA metrics, including availability and latency adherence.

Statistics

Return summary metrics from this instance only.

Input Fields

None.

Output Fields

Per-instance summary metrics.

Streaming Latency Estimate

Return per-profile estimated scan latency percentiles for streaming requests handled by this instance.

Input Fields

None.

Output Fields

Per-profile estimated P50, P95, and P99 scan latency for this instance.

Response Codes

6.3 - Policy

Policy endpoints operate on one gateway instance at a time. A request routed through a load-balanced service may reach any replica. Manage YAML policy through Helm values and the gateway ConfigMap; Cedar policy comes from Policy. When an admin token is configured, send Authorization: Bearer <admin-token>. Mutating requests also require X-Requested-With: XMLHttpRequest.

Get YAML Policy

Retrieve the active YAML policy from the gateway instance handling the request. Use the deployment’s Helm values and ConfigMap to manage policy changes.

Input Fields

None. Requires the admin token when configured.

Output Fields

The active YAML policy as JSON, including scanner definitions, profiles, routing, and upstream configuration.

Validate Scanners

Probe each configured remote scanner on the instance handling the request to verify connectivity and response-format compatibility.

Input Fields

None. The gateway sends a probe request to each remote scanner.

Output Fields

Scanner connectivity and response-format validation results for this instance.

List Scanners

Return the scanners configured on the instance handling the request, along with their types.

Input Fields

None.

Output Fields

Configured scanners and their types.

List Profiles

Return the policy profiles configured on the instance handling the request.

Input Fields

None.

Output Fields

Configured policy profiles and their scanner configuration.

Response Codes

7 - Usage

After installation is complete, you can begin using the AI Gateway. The following guides walk you through configuring the gateway, creating users, and establishing a connection: Gateway Configuration, User Creation, and Connecting to the Gateway.

7.1 - User and Identity Setup

AI Gateway accepts a verified JWT as the caller’s identity. Choose one of these identity setups:

• Use a PPC user when your deployment uses PPC authentication.
• Configure an external identity provider (IdP) when callers authenticate through an issuer such as Keycloak or Microsoft Entra ID.

Regardless of issuer, send the identity JWT to the gateway in the x-user-auth request header. The gateway validates it against the configured IdP trust registry.

PPC User Setup

Before client applications can call the gateway with a PPC-issued identity, the PPC role must grant gateway access and the user must be assigned that role.

1. Set the PPC URL and obtain an authentication token

1. Set the PPC URL using the following command:

export PPC_URL="https://<PPC_FQDN>"

2. Obtain an authentication token.

2. Add the gateway permission to the PPC role, preserving any permissions the role already requires.

curl -sS -X PUT "${PPC_URL}/pty/v1/auth/roles" \
  -H 'Accept: application/json' \
  -H "Authorization: Bearer ${TOKEN}" \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "<PPC_ROLE>",
    "description": "Role for AI Gateway access",
    "permissions": ["can_create_token", "ai_gateway_access"]
  }'

3. Create a PPC user

curl -sS -X POST "${PPC_URL}/pty/v1/auth/users" \
  -H 'Accept: application/json' \
  -H "Authorization: Bearer ${TOKEN}" \
  -H 'Content-Type: application/json' \
  -d '{
    "username": "<USERNAME>",
    "email": "<USER_EMAIL>",
    "firstName": "<FIRST_NAME>",
    "lastName": "<LAST_NAME>",
    "password": "<INITIAL_PASSWORD>",
    "roles": ["<PPC_ROLE>"]
  }'

Replace all placeholders with values for your PPC deployment. Use your organization’s password policy and secure credential-handling process.

Verify in the PPC administration interface that the role includes ai_gateway_access and that the new user is assigned that role. Then obtain the user’s identity JWT through your configured identity flow and test a request using the x-user-auth header as shown in Connecting to the Gateway.

External IdP Setup

The gateway trusts external issuers through an IdP registry file, idp.yaml. For each request, it reads the JWT from the header named by FG_PRINCIPAL_TOKEN_HEADER (default: x-user-auth), selects the issuer by the token’s iss claim, verifies the signature, and checks the configured issuer, audience, and expiration before mapping claims to the principal context.

Set the following gateway environment variables:

Example idp.yaml for an issuer with a JWKS endpoint:

issuers:
  - iss: "<IDP_ISSUER_URL>"
    idp: keycloak
    jwks_url: "<IDP_JWKS_URL>"
    audience: "<EXPECTED_AUDIENCE>"
    # Optional overrides for non-standard claim names:
    # user_id_claim: sub
    # roles_claim: realm_access.roles

Set iss to the exact issuer value in the JWT. Prefer jwks_url, which supports signing-key rotation. The gateway has claim-mapping defaults for keycloak, entra, and generic; override them only when the issuer uses non-standard claims. Supply a valid issuer-specific token to verify the configuration using the Connecting to the Gateway instructions.

7.2 - Connecting to the Gateway

Configure client applications to send requests through the AI Gateway. The gateway API base URL is <GATEWAY_HOST>/pty/federated-guardrails/v1; the client library appends its endpoint path.

Every request must include the caller’s verified identity JWT in x-user-auth. The optional x-team-id header selects a profile according to the deployment’s routing configuration. For example, the sample policy routes cs-bot to simple-protection, dps-bot to dps-protection, dpsl-bot to dpsl-protection, sdc-bot to sdc-protection, and gai-bot to gai-protection; requests without a matching header use the configured default profile.

1. Set the Gateway URL and Identity Token

Obtain an access token from the identity provider configured for your gateway. Set the gateway host and token for the examples:

export GATEWAY_BASE_URL="https://<GATEWAY_HOST>/pty/federated-guardrails/v1"
export IDENTITY_TOKEN="<IDP_ACCESS_TOKEN>"
export TEAM_ID="cs-bot"

Do not use a PPC token response header as the identity header value unless that token is the JWT issued for the caller and trusted by the gateway’s IdP registry.

2. Send an OpenAI-Compatible Request

The OpenAI SDK sends the identity and profile headers with each request. The client api_key is not the upstream credential for the OpenAI configuration shown here; the gateway uses its configured upstream key.

import os

import httpx
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["GATEWAY_BASE_URL"],
    api_key="unused",
    default_headers={
        "x-user-auth": os.environ["IDENTITY_TOKEN"],
        "x-team-id": os.environ["TEAM_ID"],
    },
    http_client=httpx.Client(),
)

reply = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello from the gateway."}],
)
print(reply.choices[0].message.content)

The OpenAI SDK appends /chat/completions to the configured base URL. To test without the SDK:

curl -X POST "${GATEWAY_BASE_URL}/chat/completions" \
  -H "Content-Type: application/json" \
  -H "x-user-auth: ${IDENTITY_TOKEN}" \
  -H "x-team-id: ${TEAM_ID}" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello from the gateway."}]
  }'

3. Send an Anthropic Request

Configure the gateway’s YAML policy with upstream.protocol: anthropic and the Anthropic upstream URL. The Anthropic SDK appends /v1/messages to the same gateway base URL.

Important: In the current gateway image, the Anthropic protocol forwards the client’s x-api-key upstream. Supply a valid Anthropic API key in the SDK; unlike the OpenAI example, a placeholder client key will not work.

import os

import httpx
from anthropic import Anthropic

client = Anthropic(
    base_url=os.environ["GATEWAY_BASE_URL"],
    api_key=os.environ["ANTHROPIC_API_KEY"],
    default_headers={
        "x-user-auth": os.environ["IDENTITY_TOKEN"],
        "x-team-id": os.environ["TEAM_ID"],
    },
    max_retries=0,
    http_client=httpx.Client(),
)

message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=512,
    messages=[{"role": "user", "content": "Hello from the gateway."}],
)
print(message.content[0].text)

To verify the connection, confirm that the SDK returns a response and that the gateway logs show the request being routed to the expected profile. Use a trusted CA certificate in production; do not disable TLS certificate verification.

8 - Troubleshoot

This section helps you diagnose and resolve common issues with an AI Gateway deployment. Use the following guidance to inspect pods, services, policy configuration, and scanner connectivity.

Common Issues

The following sections describe frequently encountered problems and how to resolve them.

One or more pods not reaching the Running state

Inspect the affected pod’s events and logs to identify why it failed to start, such as scheduling or image pull errors.

kubectl get pods -n federated-guardrails
kubectl describe pod -n federated-guardrails <POD_NAME>
kubectl logs -n federated-guardrails <POD_NAME> --all-containers --tail=100

Check that Karpenter provisioned the required nodes:

kubectl get nodes -l federated-guardrails.protegrity.com/deployment=true

API returns 401 Unauthorized

The gateway could not verify the caller’s identity token. Confirm that:

• The request includes a JWT in the x-user-auth header (or the header configured by FG_PRINCIPAL_TOKEN_HEADER).
• The token’s iss claim exactly matches an issuer in the configured IdP registry.
• The token is not expired and its signature and audience match the IdP registry.

For setup instructions, refer to External IdP Setup.

API returns 403 Forbidden

The configured policy may have blocked the request, or the caller may not have the required PPC permission. Check the response for gateway policy details, confirm that x-team-id selects the expected profile, and inspect gateway logs:

kubectl logs -n federated-guardrails deploy/federated-guardrails --all-containers --tail=100

If PPC rejects the request before it reaches the gateway, verify that the PPC user’s role includes ai_gateway_access.

Scanner or classification service is not reachable

Check that chart-managed scanner pods are running:

kubectl get pods -n federated-guardrails
kubectl get pods -n semantic-guardrails-ns
kubectl get svc -n semantic-guardrails-ns

The Classification Service is an external deployment prerequisite. Verify its availability in the data-discovery namespace:

kubectl get pods -n data-discovery
kubectl get svc -n data-discovery

Check the gateway logs for connection errors and confirm that the scanner endpoints in the Helm-managed ConfigMap match the services available in the cluster.

OpenTofu reports AccessDenied

The AWS credentials may not have permission to describe the EKS cluster or provision the configured Karpenter resources. Verify the required AWS permissions with your platform administrator.

OpenTofu reports that EC2NodeClass or NodePool CRDs are missing

Karpenter may not be installed or its custom resource definitions may not be registered. Verify the Karpenter installation and CRDs in the target cluster before rerunning tofu apply.

Helm upgrade fails with an immutable field error

Review the Helm error and compare the requested change with the existing release. Do not uninstall the release as a first response; uninstalling can remove deployed components or data. Contact your platform administrator or Protegrity Support if the update cannot be applied safely.

For additional troubleshooting guidance, refer to Protegrity Support.