diff --git a/docs/threat-models/mcp-trust-boundaries.md b/docs/threat-models/mcp-trust-boundaries.md index a3490e5..841be8d 100644 --- a/docs/threat-models/mcp-trust-boundaries.md +++ b/docs/threat-models/mcp-trust-boundaries.md @@ -5,6 +5,10 @@ identity-and-access-management perspective. Companion to this repository's thesi [paved-org](../../README.md) prevents, [CloudCanary](https://github.com/ChrisInvictus/CloudCanary) detects — this document extends the prevention lens to the newest workload class.* +**Version 1.1.** Changelog: +- *v1.1* — extended B4 to multi-agent (agent↔agent) architectures; added the retrieval-corpus (RAG) surface. +- *v1.0* — initial five-boundary model (B1–B5), STRIDE per boundary, OWASP LLM Top 10 2025 mapping, IAM lens, mitigations. + OWASP references use the **Top 10 for LLM Applications, 2025 edition**. Where the 2023 v1.1 numbering differs, it is noted — one of the 2025 revisions is itself part of this document's premise: v1.1's *Insecure Plugin Design* (LLM07:v1.1) was @@ -38,7 +42,7 @@ security — together they are the whole problem. | B1 | User ↔ Host | Intent, rendered results | The user's intent is interpreted, not executed — the model may act on more or less than what was meant | | B2 | Host ↔ MCP server | Tool calls, delegated authority | **The load-bearing boundary.** The server is separate, often third-party, code; the host cannot observe what it does with a call — only what it returns | | B3 | Server ↔ Downstream | Credentials, data, side effects | The server holds standing credentials; its compromise is the credential's compromise | -| B4 | Server ↔ Server | Shared conversational context | Tools don't talk to each other, but their outputs coexist in one context window — output from one server becomes input influencing calls to another | +| B4 | Server ↔ Server | Shared conversational context | Tools don't talk to each other, but their outputs coexist in one context window — output from one server becomes input influencing calls to another. **Extended to agent↔agent in §4.1.** | | B5 | Model ↔ Tool output | Untrusted content re-entering the reasoning loop | Tool results are *data* that the model treats as *context* — the channel for indirect prompt injection and instruction smuggling | ## 3. Threats @@ -85,6 +89,51 @@ tool output back into the component that decides the next call.** That last clause is where genuinely new work is required. Everything else is discipline we already know how to run. +### 4.1 B4 extended: agent ↔ agent + +B4 as stated above is the single-model case — multiple tools whose outputs share +one context window. As architectures graduate from *one model with tools* to +*multiple agents delegating to each other*, B4 stops being a side effect of +shared context and becomes a first-class boundary: an instruction now crosses +from one autonomous decision-maker to another. Where this is heading, three +controls follow directly from the boundaries already established. + +**(a) Per-agent identity.** The confused-deputy and over-scoped-token problems +(B2→B3) do not improve when the caller is another agent — they compound, because +the calling agent's authority is now exercised on behalf of a chain the +downstream agent cannot see. The direction the field is taking is per-agent +identity: OAuth 2.1 token exchange so each agent presents its own scoped, +short-lived credential rather than borrowing a shared one, with SPIFFE/SPIRE as +the workload-identity substrate this converges on at fleet scale. The rule is the +one already in the IAM-lens table, restated for a new caller: **no shared service +accounts and no static keys between agents** — each agent is an identity, scoped +to its verbs. + +**(b) Signed inter-agent messaging.** An instruction arriving from another agent +is untrusted input with a provenance question attached: *which* agent issued it, +and was it altered in transit. This is the same reason audit logs and CloudTrail +exist — a non-repudiable record of who did what — applied to delegation chains +rather than API calls. Signed messages give the receiving agent a basis to verify +origin before acting, and give incident response a reconstructable chain +afterward. Without it, a compromised or impersonated agent injects instructions +that look authoritative because nothing establishes that they aren't. + +**(c) Circuit breakers.** Runaway agency (LLM06) at single-agent scale is a loop +taking side-effectful actions without a gate. At chain scale it is worse: an +action can propagate through several agents, each of which believed the previous +step was authorized. The controls are bounded delegation depth (a chain may not +recurse indefinitely) and spend/action budgets enforced *across* the chain rather +than per agent — so the aggregate blast radius is capped even when no single hop +looks unreasonable. This is LLM06 relocated from the loop to the delegation graph. + +The single-agent ancestor of all three is already shipped in this repository: the +**fail-closed exec-approval gate** documented in +[the runbook, Part 2](../runbooks/securing-mcp-setup.md) puts a human in the loop +before a self-hosted agent executes a side-effectful command. Multi-agent circuit +breakers are that same control — approval required before consequential action — +generalized from one human gate to a bounded, budgeted, identity-aware delegation +chain. The gate is the primitive; the chain controls are the primitive at scale. + ## 5. Mitigations | Threat | Control | Implemented by | @@ -105,6 +154,29 @@ allowlisting, credential custody, and audit. A deployment is as strong as the weakest of the three — which is the same sentence every vendor-risk program already has on a slide. +### 5.1 Retrieval (RAG) as a surface + +Retrieval-augmented generation adds a surface the boundaries above only imply: the +**retrieval corpus** — the document store and its vector/embedding index that +ground the model's answers. In practice this is where OWASP **LLM04 (data and +model poisoning)** and **LLM08 (vector and embedding weaknesses)** live: whoever +can *write* to the retrieval store injects into every future grounded answer that +surfaces the poisoned content, silently and durably. The embedding store is a +sensitive data store wearing a search index's clothes — and the classic +data-security instinct is inverted here: **write access is the dangerous +privilege, not read.** Reading the corpus leaks; writing to it *corrupts the +model's future outputs*, which is the higher-impact compromise. + +Three controls follow. **Write-access governance on the corpus** — who and what +may add or modify retrieval content is the primary control, governed more tightly +than read (the inversion above). **Ingestion provenance** — documents entering +retrieval get vendor-onboarding-grade scrutiny: source, integrity, and +authorization checked before indexing, because an indexed document is a standing +instruction to every future query. And the **model-side rule, unchanged from +B5**: retrieved content is tool output — untrusted, demarcated, and never +auto-executed as instruction. Retrieval does not get a trust exemption for being +"our own" data; the corpus is exactly as trustworthy as its weakest write path. + ## 6. Scope and limitations - This models a **generic** MCP deployment, not any specific product's @@ -116,7 +188,21 @@ already has on a slide. - B5 mitigations are the least mature section by nature — "treat tool output as untrusted" is necessary but not yet sufficient, and honest practice today pairs it with the agency limits in LLM06 rather than claiming injection is solved. +- The multi-agent controls in §4.1 describe where the architecture is heading; + they are a design position, not a claim of a running multi-agent deployment in + this repository. The single-agent primitive (the exec-approval gate) is shipped; + the chain-scale generalization is design. - Alignment with NIST AI RMF is at the *practice* level, not certification: this - exercise instantiates the **Map** function (context and risk identification) - and feeds **Manage** (risk treatment via the mitigations table). No broader - claim is made. + exercise instantiates the **Map** function (context and risk identification) and + feeds **Manage** (risk treatment via the mitigations table). No broader claim is + made. + +## 7. Future work + +Deliberately deferred, in rough order of value: a SPIFFE/SPIRE lab demonstrating +per-agent workload identity end to end; a minimal multi-agent demo exercising the +§4.1 circuit-breaker controls in code rather than prose; and a treatment mapped to +**NIST SP 800-218A** (the Secure Software Development Practices profile for +generative AI and dual-use foundation models), cited here as adjacent guidance the +mitigations above are consistent with — fuller alignment is future scope, not a +claim made today.