Skip to content

datasets(traces): add adversarial memory_read_write trace — crafted queries extract restricted derived items from shared memory - #104

Open
sangaraju1988 wants to merge 1 commit into
GenAI-Security-Project:mainfrom
sangaraju1988:adversarial-memory-trace
Open

sangaraju1988 wants to merge 1 commit into
GenAI-Security-Project:mainfrom
sangaraju1988:adversarial-memory-trace

Conversation

@sangaraju1988

Copy link
Copy Markdown
Contributor

Summary

Adds the adversarial companion to #102, as suggested in that review. Where #102 models the unintentional path (a derived summary reaches a non-entitled user by accident), this trace models a workspace member who deliberately words a series of questions to pull restricted derived items out of a shared memory namespace.

  • Category: memory_read_write
  • Disposition: adversarial
  • Provenance tier: derived_from_public_documentation
  • DSGAI: DSGAI11 (primary), DSGAI01, DSGAI13, DSGAI18, DSGAI15, DSGAI07
  • OWASP LLM: LLM02:2025, LLM08:2025, LLM02:2026, LLM09:2026 · Agentic: ASI03:2026 · NIST AML: NISTAML.038, NISTAML.036

What the trace demonstrates

The adversary is an authenticated workspace member without the compensation entitlement. They use only the agent's normal query interface: no prompt injection into other users' sessions, no direct store access, no stolen credentials.

  1. Questions avoid every restricted term and ask for prior insights verbatim. A keyword input filter allows them (s2, DSGAI15).
  2. The agent's memory search is checked on namespace membership only (s4, DSGAI11).
  3. The store returns its nearest items by similarity. Some are summaries derived from the restricted column, with no lineage to distinguish them (s5, DSGAI13).
  4. The answer restates the figures, and a keyword output filter allows it because the figures are there but the words are not (s8 → s9, DSGAI01).
  5. Over twelve differently worded rounds, coverage of the shared namespace grows (38 distinct items, 6 restricted). Small-group averages combined with headcount the member may read reconstruct individual values (s12, DSGAI18).
  6. The audit shows only allowed reads, no filter matches and no alerts (s13, DSGAI07).

The pair with #102 makes one point: the read-time authorization against item lineage that fixes the unintentional path also stops this one, and content filtering stops neither.

Evidence

  • LangGraph store docs: search returns items under a namespace_prefix (prefix match) up to limit (default 10), ranked by semantic similarity. The documented search contract has no step that evaluates the reader's entitlements to an item's sources.
  • Wang et al., "Unveiling Privacy Risks in LLM Agent Memory" (MEXTRA), arXiv:2502.13172: a black-box, query-only attacker induces agents to retrieve and output stored memory records, and extraction grows with retrieval depth and number of attacking prompts. The notes field states that this study's memory design differs from the one modeled here, and what the citation is and isn't used for.
  • Zeng et al., "The Good and The Bad: Exploring Privacy Issues in RAG", arXiv:2402.16893: RAG systems leak their private retrieval database through attack queries.

No contributor-controlled source is cited.

Checks

Run locally on this branch:

  • validate.py: OK (5 entries + example.json)
  • dataset tests: 18 passed
  • run_all_checks.py: 0 failed
  • build_croissant.py --check: OK (dateModified 2026-10-16, index.csv sha256 updated)
  • index.csv regenerated with build_index.py and saved with LF endings per .gitattributes

Disclosure

As with #102: the contributor has published on derivation-aware access control for agent memory. That work is not cited here and is not used as provenance evidence.

…ueries extract restricted derived items from shared memory

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant