Introduction

Organizations are rapidly adopting large language models (LLMs) to power applications, automate workflows, and extract insights from unstructured data. Sending sensitive prompts to external LLMs or exposing raw LLM responses to end users introduces data privacy, compliance, and security risks.

AI Gateway addresses these risks by acting as an AI security gateway between client applications and supported LLM APIs. Every outbound prompt and inbound response passes through configurable scanners before reaching its destination. Scanners can detect and block PII leakage, identify prompt injection attacks, enforce topic restrictions, and apply graduated responses from logging to redaction to blocking based on the Cedar policy managed by Policy.

The gateway supports OpenAI-compatible and Anthropic APIs. OpenAI-compatible client libraries and frameworks can be configured to use the gateway URL. AWS Bedrock protocol support has not been verified for this release and is not covered by this guide.

The AI Gateway umbrella chart installs the gateway and its chart-managed runtime components as one release. Sensitive Data Guardrails, Semantic Guardrails, and Vault-Based Tokenization are supporting components of that release, not separate AI Gateway products or standalone installation paths. The Classification Service is a separate deployment prerequisite.