Skip to content

datasets(traces): add memory_read_write trace — derived summary outlives column entitlement - #102

Merged
emmanuelgjr merged 2 commits into
GenAI-Security-Project:mainfrom
sangaraju1988:add-memory-trace-derived-entitlement
Oct 14, 2026
Merged

emmanuelgjr merged 2 commits into
GenAI-Security-Project:mainfrom
sangaraju1988:add-memory-trace-derived-entitlement

Conversation

@sangaraju1988

Copy link
Copy Markdown
Contributor

Summary

Adds the first entry in the memory_read_write category of datasets/agentdataflow_toolexchange_traces:
entries/DSGAI-TRACE-derived-summary-memory-outlives-column-entitlement.json

An analytics agent answers an entitled user's question through a governed warehouse tool that enforces column-level access. The agent then saves its summary to a workspace-shared long-term memory namespace. The memory item records namespace, key and value but drops the source lineage the tool returned. Four days later, a workspace member without the column entitlement asks a related question. Semantic search returns the stored summary, and the only check is namespace membership, so the restricted figures reach that user. Some of them are small-group aggregates that come close to individual values.

Every guard passes. The failure only exists across the derivation step.

  • Workstream: GitHub repo, community datasets
  • Disposition: unintentional_failure
  • Provenance tier: derived_from_public_documentation, backed by two independently checkable sources:
    1. LangGraph Stores docs. The memory item model is value, key, namespace and timestamps. Namespaces are application-defined, and retrieval supports semantic search. Nothing in the model carries lineage or entitlements.
    2. PostgreSQL CREATE VIEW notes. A derived relation's base-table access is checked against the view owner by default, and invoker-side checking (security_invoker) is opt-in. This is the established precedent that derived artifacts aren't automatically bound to the reader's rights on their sources.
  • Disclosure: I've published on derivation-aware access control. Per the dataset's rule on contributor-controlled sources, that work is not used as evidence. It is mentioned once, in notes.

Not covered here: the adversarial variant, where a user deliberately phrases questions to pull high-sensitivity items from a shared namespace. That should be a separate entry, and I'm happy to follow up with it.

DSGAI mapping

DSGAI01 (primary), DSGAI11, DSGAI07, DSGAI15, DSGAI18.
Also LLM02:2025 / LLM02:2026 and ASI03:2026.

Checklist

  • Data is anonymized (no PII, credentials, or org names). Fully synthetic, with shape-only payloads.
  • Entries reference at least one DSGAI ID where applicable
  • Validation scripts pass (python data_validation/run_all_checks.py). The dataset's validate.py also passes.
  • One entry per file for dataset contributions

Note: I regenerated index.csv with build_index.py and updated the index.csv sha256 in croissant.json by hand, using the same LF-normalized hashing as build_croissant.py. I couldn't install mlcroissant locally to run build_croissant.py --check. If CI flags drift, I'll rerun it.

…ing column entitlement

First entry in the memory_read_write category. Models an agent writing a
summary derived from a restricted column to a workspace-shared memory
namespace without source lineage, and a non-entitled user later receiving
it via semantic retrieval. Maps to DSGAI01 (primary), DSGAI11, DSGAI07,
DSGAI15, DSGAI18. Tier: derived_from_public_documentation.

Regenerated index.csv and updated the index.csv sha256 in croissant.json.

@emmanuelgjr emmanuelgjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sangaraju1988, this is a strong first entry for memory_read_write. That category is in the README but had no entries until now, and this pattern is new to the dataset: every access check passes, and the leak happens only at the step where the summary is derived and stored. The span-level findings are well placed. The mitigations are concrete. Both evidence sources check out, and the PostgreSQL quote matches the docs word for word.

I ran the checks locally on your branch. run_all_checks.py passes, validate.py passes, and index.csv matches what build_index.py generates. Two changes are needed before merge:

1. Croissant metadata drift (CI will fail)
The index.csv sha256 you updated by hand is correct. But python data_validation/croissant/build_croissant.py --check still fails because dateModified wasn't updated:

-  "dateModified": "2026-09-15",
+  "dateModified": "2026-10-12",

You can edit that line in datasets/agentdataflow_toolexchange_traces/croissant.json directly or rerun the script. It's the only difference.

2. Remove the self-reference from notes
Thanks for disclosing your related work, and for not citing it as evidence. We'd like the disclosure to stay in the PR description and out of the merged file. Our rule is that merged dataset files credit the contributor only through the contributor field and don't point to the contributor's own work. Please drop the final sentence of notes, starting with "Mitigation 2 follows the general derivation-aware access-control pattern; the contributor has published…". The scope and limitations text before it is good and should stay.

Once those two changes are in, I'll merge. We'd also welcome the adversarial variant you mentioned, where a user deliberately phrases questions to pull restricted items from a shared namespace, as a follow-up PR. It would pair well with this one.

@sangaraju1988
sangaraju1988 force-pushed the add-memory-trace-derived-entitlement branch from e861fb2 to c794798 Compare October 13, 2026 21:20
@sangaraju1988

Copy link
Copy Markdown
Contributor Author

Thanks @emmanuelgjr for running the checks locally and for the clear review. Both changes are in (c794798): I removed the final sentence of notes (the disclosure stays in the PR description) and regenerated croissant.json, so dateModified is now 2026-10-12. Locally, build_croissant.py --check, run_all_checks.py and validate.py all pass. The workflows are waiting on maintainer approval since this comes from a fork. I’ll follow up with the adversarial shared-namespace variant as a separate PR.

@emmanuelgjr emmanuelgjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both changes verified, and CI is green. Thanks, merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants