Advanced Guardrails
Prompt protection, data-loss protection, context integrity and Semantic Cache must run after Prompt Template, Prompt Decorator and RAG Injection. Inspection has to happen on the prompt exactly as it will reach the model; a guardrail that runs earlier cannot see content added afterwards.
Personal-data masking is deliberately exempt. Whether masking runs before or after RAG is your own data-boundary decision, and Apinizer never moves that policy. Placing masking before RAG keeps the caller's personal data from ever reaching the embedding provider. Placing it after RAG causes it to also scan and mask personal data inside the injected context before that context reaches the model.
Apinizer does not move Off-Topic Protection, Retry-Storm (Loop) Protection or Token Rate Limit either.
If you save the policy list in an order that breaks this rule, Apinizer corrects the order when you save and tells you what was moved and why. This keeps the order you see on the Develop screen identical to the real execution order shown in tracing.
Prompt protection and context integrity always scan the request body. They do not wait for a routing plan.
Guardrail Types
Detects and masks personal data — such as Turkish national ID numbers, IBANs, and phone numbers — in both regular requests and streamed responses.
Detects jailbreak and injection attempts in incoming prompts before they reach the model. Each rule carries its own Block or Flag action — a Block rule wins over any Flag rule regardless of order, and an external judge verdict of unsafe always blocks regardless of rule actions. An empty rule list makes the policy inactive (no-op). A blocked request returns HTTP 403 (ERR-264). Can be paired with an external LLM judge — see External Provider Guardrails.
Detects and masks secrets — such as API keys, certificates, and JWTs — that could otherwise leak through a request or response.
Flags or blocks requests that fall outside the topics you define for a gateway, such as a support bot being used for unrelated chit-chat. Can be paired with an external LLM judge — see External Provider Guardrails.
Detects the same request being sent repeatedly within a short time window, such as a client or agent stuck retrying.
Detects structural injection attempts — fabricated conversation turns or fake system/assistant markers embedded in user content to manipulate the model.
Checks whether an LLM's response is actually supported by the RAG context this proxy injected into the same request, using an external LLM judge — see Groundedness Protection.
MCP and A2A Gateways
Five of the seven guardrails above — Personal Data Masking, Prompt Protection, Data-Loss Protection (DLP), Off-Topic Protection, and Retry-Storm (Loop) Protection — aren't limited to LLM Gateways: you can attach them to an MCP Gateway or an A2A Gateway as well, and they run through the exact same policy pipeline. What they inspect differs by surface, since neither protocol carries an OpenAI-format chat body:
- On an MCP Gateway, a guardrail scans the JSON-RPC tool-call arguments (
params.arguments) on the request side, and the tool result content on the response side. - On an A2A Gateway, a guardrail scans the JSON-RPC message parts (
params.message.parts[]— text and other part types) on the request side, and the task/message result envelope on the response side.
Context-Integrity Protection, Groundedness Protection, and every other AI policy type — Semantic Cache, Token Rate Limit Policy, Prompt Decorator, Prompt Templates, RAG Injection, and the AI routing policy itself — stay AI-Gateway only: they depend on an actual LLM invocation or on an OpenAI-format chat message array, neither of which an MCP or A2A Gateway has. Attaching one of these to an MCP or A2A Gateway is rejected when you save.
Decoding Encoded Content
Sending a prompt as base64, hex, percent-encoding (%41) or \u0041 changes the text a guardrail sees without changing the text the model understands. Models learned to decode these during pre-training, so the payload still reaches its target, while a pattern-based guardrail sees nothing but a meaningless character run on the wire. OWASP does not treat this as a separate threat: it classifies encoding as a delivery method that carries prompt injection (LLM01) past input filters.
For that reason Personal Data Masking, Prompt Protection, Data-Loss Protection and Off-Topic Protection each offer a Decode encoded content option. When it is on, the guardrail decodes encoded segments of the prompt before scanning and inspects the decoded text as well.
With the option off, behaviour is byte-for-byte what it is today. Nothing about your existing proxies changes unless you turn it on.
Decoded content is inspected, never rewritten
The decoded text is scanned but never written back into the request body. That has one practical consequence: when personal data or a secret is found inside encoded content, the request is rejected rather than masked.
The reason is a technical constraint. Writing a masked value back into a base64 blob corrupts the payload the caller sent, and with nested or partial encodings there is no reliable way to map a byte back to its position. That leaves two options: forward the encoded personal data unmasked — which is exactly the leak masking exists to prevent — or reject the request. Apinizer rejects.
Consistently with that, a Data-Loss Protection rule whose action is mask is escalated to block when it matches decoded content. A flag action is left alone, because flagging never claimed to modify anything in the first place.
Prompt Protection and Off-Topic Protection have no such escalation: both only ever detect, so whatever your rule specifies is what happens.
Guarding against false matches
Hashes, UUIDs and API keys also look like base64 or hex. Decoding them produces byte soup, and reporting that as "hidden content" would flag nearly every request. Apinizer checks how much of the decoded output is readable text and silently discards anything below the threshold, so an ordinary prompt containing a SHA-256 digest or a random key is unaffected.
A JWT payload that decodes to readable JSON, by contrast, is inspected — that is not a false match but the intended behaviour: personal data inside the claims is in scope too.
Limits
- Compressed content (gzip, deflate) is not decoded. This is deliberate: every supported encoding shrinks when decoded, so there is no risk of exponential growth; compression would remove that guarantee.
- Nested encodings are decoded up to three levels, and there are ceilings on input size, candidate count and total decoded text. When a ceiling is reached the scan continues with a partial result and records the fact in the trace — it is never truncated silently.
- Streaming responses are not decoded chunk by chunk; the buffered scan path is covered.
- A failed decode never rejects the request: the scan of the original text has already run, and decoding only widens coverage. Sending deliberately malformed base64 cannot be used to force a rejection.
Turning It On From the Screen
All four guardrails (Personal Data Masking, Prompt Protection, Data-Loss Protection, Off-Topic Protection) carry a Decode Encoded Content switch on their policy screen:
| Guardrail | Where the switch lives |
|---|---|
| Personal Data Masking | Mask Settings section |
| Data-Loss Protection (DLP) | General Information section |
| Prompt Protection / Off-Topic Protection | Content Scope section |
Turn it on, save the policy, and click Deploy. The same setting is also reachable through the APIops REST API (API Reference: AI Gateway) or by setting decodeEncodedContent to true in the policy JSON.
Message Role Scope
Prompt Protection and Off-Topic Protection can each choose which message roles (system, user, assistant, tool) get scanned or compared, through three switches: include system prompts, include assistant prompts, include tool prompts.
These three switches share the same names on both guardrails, but their default direction is reversed — don't configure one from habit formed on the other:
| Prompt Protection | Off-Topic Protection | |
|---|---|---|
| Default (all three unset) | Every role is scanned — today's full-scan behavior | Only the last user message is compared — today's behavior |
| What the switch means | An exclude switch: turning it off narrows the scan | An include switch: turning it on widens the comparison |
| Turning a switch off | Removes that role from the scan | (already the default — no effect) |
| Turning a switch on | (already the default — no effect) | Adds that role to the comparison and triggers the mode switch below |
Prompt Protection: Exclude Switches
Prompt Protection scans every message in the request today, regardless of role — these three switches exist to narrow that scan, not widen it. includeSystemPrompts covers more than role: "system" messages: it also gates the Anthropic top-level system field, since both are the same system prompt arriving through two different surfaces, and one switch governs both.
- An unrecognized role is never excluded. A
developerrole, afunctionrole, a provider-specific role, or a message with norolefield at all is always scanned — these three switches only ever gatesystem/assistant/tool. A guardrail must never drop attacker-controlled text just because it doesn't recognize the role label. - An empty scan falls back to the raw body. If role filtering leaves nothing to scan (for example, a request that contains only a
systemmessage withincludeSystemPromptsoff), the guardrail does not silently skip the scan — it scans the raw request body instead. The net effect: these switches can only narrow what part of the request is scanned via the structured path, never make the guardrail scan nothing (fail-closed).
Off-Topic Protection: Include Switches + Mode Switch
Off-Topic Protection compares only the conversation's last user message against the topic embeddings today. Turning any one of the three switches on moves the guardrail from "last message" mode to "whole conversation" mode — the text fed into the comparison is no longer a single message, but the entire conversation including the roles you selected.
The mode switch changes not just what's in scope but the size of the text going into the embedding call, which shifts the similarity distribution — a whole conversation produces a more "diluted" embedding than a single message. After turning on any role switch, verify your existing Similarity Threshold still draws the right line, and account for the larger embedding call when you do.
Scope: AI/OpenAI-Anthropic Format Only
These three switches apply only to the LLM Gateway arm that reads the OpenAI/Anthropic messages array (and, for Prompt Protection, the Anthropic system field). The same policy attached to an MCP Gateway or A2A Gateway scans that protocol's own content shape (JSON-RPC arguments / parts[]) instead, and these switches have no effect there.
Turning It On From the Screen
Prompt Protection and Off-Topic Protection carry three switches in the same Content Scope section as the Decode Encoded Content switch above: Include System Messages, Include Assistant Messages, Include Tool Messages. The same settings are also reachable through the APIops REST API (API Reference: AI Gateway) or by setting includeSystemPrompts / includeAssistantPrompts / includeToolPrompts in the policy JSON.
Personal-Data Types
Personal-data masking recognizes the following built-in types out of the box, and you can add your own custom patterns for anything installation-specific:
| Type | Example | Validation |
|---|---|---|
| Turkish National ID (TCKN) | 12345678950 | Checksum |
| IBAN (Turkey) | TR330006100519786457841326 | Checksum (mod 97) |
| Phone (Turkey) | 05321234567 | Structural |
user@example.com | Structural | |
| Credit Card | 4111111111111111 | Checksum (Luhn) |
| IP Address | 203.0.113.5 | Structural (IPv4/IPv6) |
| URL | https://example.com/path | Structural |
| Passport Number | P1234567 | Structural only — a Turkish serial (1–2 letters + 6–7 digits), or a 9-character value that contains at least one letter and one digit |
| Social Security Number (US) | 123-45-6789 | Structural |
| Driver License Number (Turkey) | A12345678 | Structural only |
| IBAN (Generic, any country) | DE89370400440532013000 | Checksum (mod 97) |
| Crypto Wallet Address | 0x71C7656EC7ab88b098defB751B7401B5f6d8976 | Checksum (Bitcoin / Ethereum) |
The Turkish and checksum-validated types produce far fewer false positives than a plain regular expression, since a random string of digits has to actually satisfy the check-digit formula to match.
Unlike most of the other built-in types, passport numbers and Turkish driver's license numbers have no reliable check digit to validate against — these two match on structure only (a letter followed by digits, or a generic alphanumeric code), so they carry a materially higher false-positive rate. If you enable either broadly, pair it with a custom pattern that also requires a matching field name, rather than scanning free-form text for it.
The passport pattern does not mask a 9-character value made only of letters (such as assistant) or only of digits. The 9-character form must contain at least one letter and one digit. A policy or preset that keeps a copy of the earlier pattern is updated when you upgrade, and every deployment that uses it is marked redeploy required. Redeploy the proxy to start using the new pattern.
Managing the Rule List
A new personal-data masking policy opens with an empty rule list — the same behaviour as every other Apinizer form: you add only the rules you want. The bulk actions above the list are:
| Action | What it does |
|---|---|
| Add Default Set | Adds one masking rule for every built-in type listed above, in a single click. Rules that are already defined are skipped, so pressing it twice never duplicates the list. |
| Add from Preset | Adds the patterns you pick from the PII Patterns catalog. The box in the modal header selects the whole catalog at once. |
| Add Definition | Creates a single rule from scratch. |
| Clear All | Removes every rule in one step, after a confirmation. |
Rule-list changes are not persisted until you save the policy. A policy left with an empty rule list masks nothing — the screen flags this as a warning.
In JSON Bodies: Rules Apply to Values Only
When the request or response body is JSON, masking does not scan it as flat text. It walks the JSON tree and applies the rules to string values only. Field names (keys) and JSON syntax — quotes, braces, commas — are never rewritten by a rule.
The practical consequence: a broad or greedily written custom regex cannot corrupt the body's structure, not even inside a very long prompt. The masked body always reaches the provider as valid JSON, and when no rule matches, the body is passed through byte for byte. For non-JSON bodies (plain text, form data) the scan still runs over the whole text as before.
When a masked body cannot be produced, the request is rejected. After masking, the rebuilt JSON is validated before it replaces the original body. In the unlikely case that validation fails, Apinizer does not fall back to forwarding the original — that would send the very personal data the rule matched to the provider unmasked — and it does not forward a broken body either. The request is rejected with error ERR-303 (AI PII Mask: request rejected — a masked request body could not be produced safely, so the original body was not forwarded, HTTP 500). The message and status code can be customized on the Error Message tab like any other policy error. No request content is written to the application log in this case.
To mask by field name, use the element-name rule type rather than a regex — that rule matches the key and masks the corresponding value.
On a streamed response, masking uses a sliding-window carry mechanism — chunks are released to the client as they arrive, keeping only a small overlap buffer (Carry Buffer Size, default 48 bytes) across chunk boundaries so that a match split across two chunks is never missed. Because only this small buffer is held back, time-to-first-token is not affected. The carry buffer size is configurable on the policy screen.
The answer text is masked even when the provider sends it in a chunk that carries usage or finish_reason (for example, Google AI Studio sends usage in every chunk). When a policy changes or holds the text of a chunk in a chat completions or text completions stream, that chunk is rebuilt. The finish_reason, usage and tool call fields of the chunk stay. The logprobs field and every choice after the first (when n is larger than 1) are removed. In a chat chunk, the refusal and audio fields are removed too.
Tool Results in the Tool-Call Loop
When a model calls an MCP or A2A tool through the tool-call loop, Apinizer masks the tool result before it goes back to the model as the next turn. The masking uses the union of the rules of every active response-direction Personal Data Masking policy on the proxy. A policy inside a Policy Group counts only when the group is active. If one policy masks email addresses and another masks phone numbers, the tool result gets both.
Apinizer collects these masking rules before the response-direction policies run; in the tool-call loop the model's answer does not exist yet at that point, and a condition can change by the time its policy runs. For this reason Apinizer does not evaluate the condition of a Personal Data Masking policy for tool results. A conditional policy masks the tool result even when its condition is not met for the request. Masking of the response body is unchanged: the policy follows its own condition there.
Tool Calls and Reasoning in Streams
On a streamed response, Personal Data Masking and Data-Loss Protection mask the arguments of a tool call and the reasoning text of the model, in addition to the answer text. This works when the proxy has one of these active policies: Personal Data Masking with at least one rule that does more than detect, or Data-Loss Protection with a Mask or Block rule (or with file signature detection turned on). The policy can sit in the request direction or in the response direction, on the method, the proxy or the proxy group. A policy inside an active Policy Group counts too. A policy whose condition is not met for the request does not mask. Without such a policy, or when no rule matches, the stream reaches the client unchanged.
| API | What is masked |
|---|---|
| Chat completions | The arguments of a tool call (also the older function call format), the input of a custom tool, and the reasoning text. A tool call that arrives in the same chunk as answer text is masked too. A stream that Apinizer translates from Anthropic or Gemini to the chat format is included. |
| Anthropic Messages | The input of a tool call and of an MCP tool call, and the thinking text |
| Responses | The arguments of a function call, a custom tool call and an MCP call, the code of a code interpreter call, the reasoning text and the reasoning summary. Events that carry the complete value are masked too. |
- Apinizer holds the events of one value until the value is complete. It masks the whole value and sends it in the first event. The next events of the same value are empty. The number and the order of events do not change. This means that the client sees a tool call or a reasoning text when it is complete, not piece by piece.
- Keep-alive comments and ping events are sent at once. An empty line between held events stays in place.
- A match that is split across two parts of a value is found, because the whole value is masked.
- If a value is larger than 16 MB, Apinizer masks it in two parts at that point. A match that crosses the cut can be missed.
- If masking fails, the value is sent empty.
- An event type that Apinizer does not know is sent unchanged. When a provider adds a new event with a value, the list is extended.
Anthropic signs the thinking text. When a rule matches and the thinking text changes, the signature is no longer valid. A client that sends the thinking blocks back in the next turn, for example in a multi-turn conversation with thinking, can get HTTP 400 for that turn. This happens only when a rule matches.
Off-Topic Protection and Groundedness Protection judge text and do not change it. They do not work on tool calls and reasoning text.
While Live Trace is on, Apinizer keeps the raw lines of the provider for the traced requests. These lines contain the text before masking. Live Trace is a tool for debugging. Turn it on only for a short time.
Data-Loss Protection Patterns
Data-loss protection ships with ready-made patterns for common secret formats, and you can add your own custom patterns for your organization. A pattern match with a Block action returns HTTP 403 (ERR-269).
| Pattern | Example |
|---|---|
| AWS access key | AKIAIOSFODNN7EXAMPLE |
| OpenAI API key | sk-... |
| GitHub personal access token | ghp_... |
| Slack token | xoxb-... |
| Google API key | AIza... |
| PEM private key block | -----BEGIN PRIVATE KEY----- |
| JWT (JSON Web Token) | eyJhbGci... |
| Generic password/secret/API-key assignment | password: ..., api_key=... |
When a Block or Mask rule is active, this policy buffers the entire streamed response before scanning — the complete text is scanned once the stream ends, which removes time-to-first-token for that policy's streamed responses. Flag-only rules (with no File Signature Detection active) use the per-chunk path and preserve streaming behavior; a sliding-window carry buffer (Streaming Carry Buffer Size (chars), default 512 characters) keeps matches split across chunk boundaries from being missed.
Built-in Pattern Versioning
The built-in patterns above ship as a versioned, checksummed package rather than being fixed in the product's code — in the same spirit as Prompt Protection's own signature pack described further down this page, applied here to secret-detection patterns instead. The package's integrity is verified automatically as part of an Apinizer upgrade: if a pattern in the package doesn't match its recorded checksum — a sign the package was corrupted or tampered with — none of its patterns are applied, and a warning is written to the application logs rather than the mismatch failing silently or blocking the upgrade. Patterns already in effect from a previously verified package are left exactly as they are.
A verified package only ever touches the built-in patterns Apinizer itself ships — a custom pattern you added under your own name is never overwritten or removed by it, no matter what changes in a later package.
Editing a built-in pattern protects it from the next package refresh. Built-in patterns are no longer read-only — an admin can edit or delete any of them from the DLP preset screen (or via APIops) the same as a custom pattern. The first time you edit one, it is flagged internally as overridden; from that point on, package verification treats that row exactly like a custom pattern and never overwrites your edit again, even if a later Apinizer upgrade changes the same pattern in the shipped package. Deleting a built-in pattern is permanent — it does not come back on the next upgrade unless a future release ships an explicit re-seed.
Unlike Prompt Protection's signature-pack banner (installed version, checksum, an "up to date" badge), the Data-Loss Protection pattern list does not show a package version on screen today — verification happens automatically in the background during an upgrade, and a failure is visible only in the application logs.