Prompt injection causes damaging tool execution and data leakage when an application places untrusted retrieved or web content into the same privilege layer as developer system instructions. Because current large language models process system prompts, user inputs, retrieved documents, and tool outputs inside a single token stream without a built-in boundary between instructions and data, two deterministic engineering gates must sit outside the model: untrusted text must never escalate tool permissions, and every outbound destination—across backend HTTP tools and browser-rendered Markdown—must match an explicit allowlist before invocation.
For engineering teams reviewing architectures across our AI Security coverage, this guide maps the boundary controls, outbound host and IP verification rules, pre-deployment smoke tests, and the exact allowlist inputs to collect before auditing an agent integration.
Why Shared Context Turns Retrieved Text into Privileged Instructions
Under OWASP LLM01:2025, a prompt injection vulnerability occurs whenever prompts alter an LLM's behavior or output in unintended ways. Direct prompt injection originates from a user's immediate prompt input altering model behavior intentionally or unintentionally. Indirect prompt injection occurs when an LLM accepts input from external sources—such as websites, files, emails, or retrieved documents—containing data that alters model behavior when interpreted.
Injected instructions do not need to be human-visible or human-readable to take effect, so long as the content is parsed by the model. While Retrieval-Augmented Generation (RAG) and fine-tuning aim to make outputs more relevant and accurate, research shows that they do not fully mitigate prompt injection vulnerabilities.
Empirical evaluations demonstrate why natural-language prompt tuning cannot serve as an execution gate:
- Public multi-model agent red-teaming (NIST CAISI, UK AISI, and Gray Swan, March 2026): Across more than 250,000 attack attempts from over 400 participants evaluating 13 frontier models across tool use, coding, and computer use agentic scenarios, at least one successful indirect prompt injection (agent hijacking) attack was found against every one of the 13 target frontier models.
- Best-of-N augmented prompting (Hughes et al., 2024, cited by OWASP): Using up to 10,000 Best-of-N augmented prompts per request, researchers reported 89% attack success on GPT-4o and 78% attack success on Claude 3.5 Sonnet in their 2024 evaluation.
OWASP LLM07:2025 (System Prompt Leakage) explicitly states that a system prompt should not be considered a secret, nor should it be used as a security control, and credentials or connection strings must never be placed inside it. Critical controls such as privilege separation and authorization bounds checks must not be delegated to the LLM through the system prompt or otherwise, because they must execute in a deterministic, auditable layer outside the model.
When external documents and developer instructions share a single token stream, formatting delimiters and input filters assist with telemetry and structure without creating a privilege boundary.
| Layer or Control | What It Actually Does | Why It Cannot Replace a Tool or Egress Boundary | Required Deterministic Counterpart |
|---|---|---|---|
| Layer or Control System prompt role instructions & XML/text delimiters | What It Actually Does Orientes the model on expected task structure and output formatting. | Why It Cannot Replace a Tool or Egress Boundary LLMs do not enforce a privilege boundary between instructions and data in a token stream. | Required Deterministic Counterpart Enforce authorization and privilege separation in application code. |
| Layer or ControlKeyword / regex input screening | What It Actually Does Flags known attack phrasing for logging, review, or rate limiting. | Why It Cannot Replace a Tool or Egress Boundary Input screening is not a security boundary because rephrased or encoded instructions still reach the model. | Required Deterministic Counterpart Validate permissions and parameters at the tool execution boundary. |
Layer or ControlUnicode NFKC normalization | What It Actually Does Folds compatibility characters prior to inspection. | Why It Cannot Replace a Tool or Egress Boundary
| Required Deterministic Counterpart Normalize and strip or flag invisible tag characters ( |
| Layer or Control Secondary guardrail LLM (LLM-as-judge) | What It Actually Does Screens inputs or outputs as one layer in a defense-in-depth design. | Why It Cannot Replace a Tool or Egress Boundary A guardrail LLM is itself an LLM susceptible to prompt injection. | Required Deterministic Counterpart Keep least-privilege tool scopes, parameter validation, and human approval on destructive actions. |
Enforcing the Tool Boundary So Untrusted Text Cannot Escalate Permissions
In OWASP LLM06:2025, Excessive Agency is defined as the vulnerability enabling damaging actions to be performed in response to unexpected, ambiguous, or manipulated outputs from an LLM. The root cause of Excessive Agency is one or more of three factors: excessive functionality, excessive permissions, or excessive autonomy. When an agent combines three specific capabilities—what practitioner Simon Willison defines as the lethal trifecta: (1) access to private data, (2) exposure to untrusted content, and (3) the ability to externally communicate—untrusted content can trick the model into accessing private data and sending it to an attacker.
Complete mediation requires that every tool invocation be authorized by deterministic downstream code bound to the caller's session and least-privilege scope.
To guarantee that untrusted text cannot escalate tool permissions, enforce four architectural controls outside the model:
- Minimize functionality and eliminate open-ended tools: Avoid open-ended extensions such as running a shell command or fetching an arbitrary URL where possible, and replace them with narrow extensions that expose granular functionality.
- Hold tokens in application code and scope permissions to the caller: Provide the application with its own API tokens for extensible functionality and handle those functions in code rather than providing tokens to the model. Enforce the complete mediation principle by implementing authorization in downstream systems rather than relying on an LLM to decide if an action is allowed.
- Separate untrusted text parsing from privileged tool calling (Dual-LLM and CaMeL): In Simon Willison's Dual LLM pattern, a Quarantined LLM processes untrusted content with zero tool access, while a Privileged LLM with tool access never sees untrusted content or raw tainted summaries—operating only on symbolic variable references such as
$VAR1and$VAR2managed by a deterministic software Controller. Google DeepMind's CaMeL extends this separation with a custom interpreter tracking data flow and capability metadata, though OWASP notes that CaMeL assumes a trusted user prompt, does not prevent misleading text summaries inside an allowed flow, and its released code is a research artifact rather than a supported security component. - Treat tool annotations as untrusted metadata: Under the Model Context Protocol (MCP) specification revision 2026-07-28, clients MUST consider tool annotations (optional properties describing tool behavior, such as behavioral hints) to be untrusted unless they come from trusted servers, and servers MUST validate all tool inputs, implement proper access controls, rate limit tool invocations, and sanitize tool outputs. For JSON Schema 2020-12
inputSchemavalidation, OAuth resource-server audience verification (RFC 8707 / RFC 9068), and protocol versusisError: trueerror separation, see our companion guide on Model Context Protocol Tool Schema and Auth. Where a state-changing or outbound tool requires explicit human sign-off bound to the exact proposed arguments, pair the tool gate with the pause-and-resume controls in Human-in-the-Loop Agent Authorization Checks.
Locking Down Outbound Destinations Across Backend Tools and Browser Rendering
Even when write tools are restricted, indirect prompt injection causes LLM data exfiltration whenever sensitive context can leave the environment through an unverified outbound channel. Auditing egress requires locking down both backend HTTP/webhook tools and client-rendered Markdown output.
Surface A: Backend HTTP, Webhook, and API Egress
Under OWASP LLM05:2025 (Improper Output Handling), insufficient validation, sanitization, and handling of LLM-generated outputs before they are passed downstream can result in server-side request forgery (SSRF), privilege escalation, or remote code execution on backend systems. Applications must treat the model as any other untrusted user under a zero-trust approach and apply strict input validation on responses coming from the model to backend functions.
When a backend tool or MCP server makes an outbound network call, every destination must pass five deterministic checks before a socket opens:
- Parse the URI authority component per IETF RFC 3986 §3.2 and match an explicit domain allowlist: RFC 3986 §3.2 defines the authority component (preceded by
//and terminated by/,?,#, or the end of the URI) as[ userinfo "@" ] host [ ":" port ], separating the destination host from path (§3.3) and query (§3.4) components. Extract the host, compare it strictly against the service's approved outbound domain allowlist, and construct the outbound request from the fixed allowlist target rather than forwarding a raw model-supplied URL string. - Reject URL parser disagreements across service boundaries: Where a URL crosses a service boundary as a string and is parsed again downstream, different implementations can read different hosts from the same bytes. For example, on an authority string containing a backslash-escaped commercial at-sign (
example.com\@evil.com), a WHATWG URL Standard parser reads the host asexample.comwhile CPython'surllib.parse.urlsplitreads the hostname asevil.com. Applications must reject any URL whose host is not read identically by every parser in play, rather than attempting to reconcile conflicting readings. - Disable automatic HTTP redirection: To prevent input-validation bypass via URL redirection, disable support for redirection in the web client, and ensure MCP clients do not blindly follow redirects to internal resources, applying HTTPS and IP range restrictions to redirect destinations or disabling automatic redirect following and validating each hop.
- Pin validated DNS resolutions to prevent DNS rebinding: Domain allowlisting alone does not prevent DNS rebinding. The application must validate the resolved destination IP addresses against permitted networks and ensure the HTTP client connects only to the validated IP address while preserving the
Hostheader and TLSSNI. - Block private, loopback, and link-local cloud metadata IP ranges: Outbound IP validation must block private IPv4 ranges
10.0.0.0/8,172.16.0.0/12, and192.168.0.0/16; loopback addresses127.0.0.0/8and::1(::1/128); IPv6 Unique Localfc00::/7and Link-Localfe80::/10; and the IPv4 link-local range169.254.0.0/16, including cloud instance metadata service (IMDS) endpoints at169.254.169.254across AWS, GCP, and Azure.
Validating domain names without pinning the resolved IP address or blocking 169.254.0.0/16 and RFC 1918 ranges leaves backend fetch tools exposed to DNS rebinding and cloud metadata SSRF.
Surface B: Client-Rendered Markdown Images and Hyperlinks
Even when an AI agent has zero outbound HTTP tools on the backend, client-side rendering creates a direct exfiltration channel.
Under OWASP LLM01:2025 Scenario #2, when a user employs an LLM to summarize a webpage containing hidden instructions, those instructions can cause the LLM to insert an image linking to an external URL—such as —exfiltrating the private conversation as soon as the browser requests the image.
Similarly, an injected instruction can tell the model to base64-encode private context into a clickable hyperlink and persuade the user to click it, or exploit improper output handling to trigger cross-site scripting (XSS) and cross-site request forgery (CSRF) in web browsers.
Lock down the browser rendering boundary with three controls:
- Strip or block external Markdown and HTML images in untrusted outputs: Do not render external image tags (
![...]()or<img>) generated during sessions that ingest untrusted content. - Restrict clickable links to an approved domain allowlist: Allow rendered hyperlinks only when their parsed target host matches a pre-approved allowlist of trusted domains that cannot be used by an attacker to harvest query parameters or referral logs.
- Enforce browser Content Security Policy: OWASP LLM05:2025 recommends employing strict Content Security Policies (CSP) and context-aware output encoding when rendering LLM-generated content. Under W3C Content Security Policy Level 3, the
img-srcdirective (§6.1.6) restricts the URLs from which image resources may be loaded, and theconnect-srcdirective (§6.1.2) restricts the URLs that can be loaded using script interfaces (fetch(),XMLHttpRequest,EventSource,Beacon,WebSocket, and anchorping), returning network errors whenever a URL does not match the policy's source list.
Pre-Deployment Verification Matrix and Smoke-Test Grading
OWASP LLM01:2025 recommends performing regular penetration testing and breach simulations that treat the model as an untrusted user to test the effectiveness of trust boundaries and access controls. When grading prompt-injection smoke tests, do not rely on whether the model's final chat message sounds like a refusal—a polite refusal in text does not undo a tool call or outbound request that already fired during the turn.
Instead, grade every smoke-test run against three separate observables defined in the OWASP LLM Prompt Injection Prevention Cheat Sheet:
- System-prompt marker disclosure: Whether a harmless dummy canary marker placed in the test system prompt appears in the model's output.
- Instrumented unauthorized tool calls: Whether the agent runtime dispatches any tool call or state mutation outside the caller's authorized scope.
- External disclosure at an instrumented test sink: Whether any sensitive context reaches a local or sandboxed listener sink via backend HTTP parameters or rendered Markdown URLs.
Always report false-positive rates on a defined benign workload alongside violation counts, and include a confidence interval rather than a bare ratio: as OWASP notes, observing 0 false positives across 7 independent binary trials gives a 95% Wilson confidence interval of approximately 0% to 35.4%, not proof of a 0% false-positive rate.
| Failure Pathway | Where Untrusted Text Enters | What Fails if Unchecked | Deterministic Boundary Control | Verification Test & Observable |
|---|---|---|---|---|
| Failure Pathway 1. Privilege escalation via shared context | Where Untrusted Text Enters Retrieved RAG chunk, webpage, or email concatenated with system prompt. | What Fails if Unchecked Model treats injected prose as instructions and invokes write or admin tools. | Deterministic Boundary Control Enforce per-tool authorization in application code using caller-scoped least privilege. | Verification Test & Observable Place an instruction requesting a write tool inside a sandboxed retrieval document; assert 0 dispatches at the instrumented write handler. |
| Failure Pathway2. Tainted data-to-argument flow | Where Untrusted Text Enters Untrusted summary or extracted field passed back into a tool-calling LLM. | What Fails if Unchecked Injected text overwrites target recipient, file path, or query parameters in a subsequent tool call. | Deterministic Boundary Control Isolate untrusted content in a Quarantined LLM with zero tools and pass
| Verification Test & Observable Feed a mocked untrusted message containing an alternate recipient; assert the controller blocks or gates any recipient change on explicit human approval. |
| Failure Pathway 3. Open-ended tool or untrusted MCP annotation | Where Untrusted Text Enters Plugin or MCP server exposing open-ended extensions (running a shell
command or fetching a URL) or untrusted tool | What Fails if Unchecked Client trusts unverified tool behavior | Deterministic Boundary Control Replace open-ended tools with granular extensions, treat MCP | Verification Test & Observable Attempt to invoke an open-ended tool or rely on an unverified tool annotation; assert pre-dispatch refusal by server access controls. |
| Failure Pathway 4. Backend HTTP / webhook egress & SSRF | Where Untrusted Text Enters Model supplies an unlisted URL, redirect chain, or parser-ambiguous
authority ( | What Fails if Unchecked Private context is sent to an external host, or the client fetches
internal/IMDS endpoints ( | Deterministic Boundary Control Parse host per RFC 3986, match the outbound domain allowlist, reject parser mismatches, disable redirects, pin DNS, and block private/IMDS CIDRs. | Verification Test & Observable Pass an unlisted host, a redirect to an internal resource,
|
| Failure Pathway 5. Client-rendered Markdown image or link exfiltration | Where Untrusted Text Enters Model response includes | What Fails if Unchecked Browser automatically requests the external image URL on render or user clicks a link carrying base64-encoded private data. | Deterministic Boundary Control Strip external Markdown/HTML images, restrict link domains to an
allowlist, and enforce browser CSP3 | Verification Test & Observable Render a synthetic response containing an unlisted image URL under production CSP; assert sanitizer strips the tag and CSP3 blocks the request. |
Pre-Review Intake Checklist: Tool Allowlist and Outbound Domain Allowlist
Before conducting an architecture or security review on an AI agent or RAG workflow, assemble two concrete configuration artifacts for the target deployment: the registered tool allowlist and the outbound domain allowlist.
Part A: Registered Tool Allowlist Inventory
Record every tool callable by the agent runtime:
| Field to Collect | Required Engineering Specification | Boundary Failure if Omitted |
|---|---|---|
| Field to CollectTool identifier & narrow purpose | Required Engineering Specification Exact function name and single-purpose action; confirm no open-ended shell or arbitrary URL-fetch tools exist. | Boundary Failure if Omitted Excessive functionality allows injected instructions to repurpose generic tools. |
| Field to Collect Caller authentication & OAuth scope | Required Engineering Specification Application-held token policy and per-user least-privilege scope enforced in downstream code. | Boundary Failure if Omitted Shared service accounts or over-scoped tokens enable confused-deputy privilege escalation. |
| Field to Collect Untrusted content exposure (Lethal Trifecta leg 2) | Required Engineering Specification Whether the tool ingests external web pages, emails, files, issues, or unverified RAG chunks. | Boundary Failure if Omitted Tainted tool outputs enter the privileged planning context without
Quarantined LLM / |
| Field to Collect State mutation & human approval gate | Required Engineering Specification Whether the tool modifies external state or dispatches messages, requiring deterministic parameter validation and human approval. | Boundary Failure if Omitted High-impact actions execute autonomously from manipulated model outputs. |
Part B: Outbound Domain and Browser CSP Allowlist Inventory
Record every external destination reachable by backend tools or browser-rendered responses:
| Field to Collect | Required Engineering Specification | Boundary Failure if Omitted |
|---|---|---|
| Field to Collect Exact RFC 3986 host, scheme, and port | Required Engineering Specification Case-sensitive allowlisted domain names, fixed scheme, and permitted port used to reconstruct outbound requests. | Boundary Failure if Omitted Substring or regex host checks allow attacker subdomains or
parser-mismatch bypasses ( |
| Field to CollectParser-agreement & redirect policy | Required Engineering Specification Confirmation that URLs with parser disagreements are rejected, support for redirection in the web client is disabled, and redirects to internal resources are blocked. | Boundary Failure if Omitted URL redirection bypasses input validation or follows redirects to internal resources. |
| Field to CollectDNS pinning & blocked CIDR list | Required Engineering Specification Socket binding to the validated public IP (preserving | Boundary Failure if Omitted DNS rebinding or direct IP literals expose internal services and cloud IMDS credentials. |
| Field to CollectBrowser Markdown & CSP3 directives | Required Engineering Specification Sanitizer rule stripping external Markdown/HTML images, clickable link
domain allowlist, and | Boundary Failure if Omitted Browser automatically fires |
Once your team has compiled both allowlist tables, cross-check your MCP server schema and token-audience rules against Model Context Protocol Tool Schema and Auth, verify your high-consequence approval state machines against Human-in-the-Loop Agent Authorization Checks, and review related summit sessions under AI Security.
Sources
- OWASP GenAI Security Project, Top 10 for LLM Applications 2025: LLM01:2025 Prompt Injection (including its MITRE ATLAS AML.T0051.000 and AML.T0051.001 mappings), LLM05:2025 Improper Output Handling, LLM06:2025 Excessive Agency, and LLM07:2025 System Prompt Leakage.
- OWASP Cheat Sheet Series, LLM Prompt Injection Prevention Cheat Sheet (also the source cited for the Hughes et al. 2024 Best-of-N figures and the CaMeL boundaries).
- OWASP Cheat Sheet Series, Server-Side Request Forgery Prevention Cheat Sheet.
- Model Context Protocol specification, revision 2026-07-28: Server Tools.
- Model Context Protocol specification, revision 2026-07-28: Security Best Practices.
- U.S. NIST Center for AI Standards and Innovation (CAISI), UK AISI, and Gray Swan, public AI agent red-teaming competition analysis, March 2026 (arXiv:2603.15714).
- Greshake et al., Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (arXiv:2302.12173v2).
- IETF RFC 3986 (STD 66), Uniform Resource Identifier (URI): Generic Syntax.
- W3C, Content Security Policy Level 3.
- Simon Willison, The Dual LLM pattern for building AI assistants that can resist prompt injection.
- Simon Willison, The lethal trifecta for AI agents: private data, untrusted content, and external communication.