test(properties): cover sensitivity enforcement (#755 item 6)

[dgenio]

Summary

Adds property coverage for item 6 of #755's phase-2 list — sensitivity enforcement — over context.sensitivity.apply_sensitivity_filter. With #887 (items 1, 2, 7) this leaves item 6 done.

Part of #755.

Changes

Four properties plus a guard-the-guard, appended to tests/test_properties.py:

• the floor is INCLUSIVE — an item whose sensitivity equals the floor is enforced, not passed through;
• everything below the floor survives untouched and in input order;
• the text of an at-or-above item reaches model-visible output under NEITHER action — the security contract stated directly rather than as a count;
• redact keeps the slot, masks the payload, and clears artifact_ref (Prevent redacted items from exposing original content via artifact handles and drilldown #451);
• a find()-based guard that the corpus still generates artifact_refs at all.

Two deliberate choices

The severity ladder is restated in the test rather than imported from _SENSITIVITY_ORDER. Importing it would make every property agree with the implementation by construction — reorder the ladder in the source and the tests would reorder with it and still pass.

The second property is the counter-property. Without it, an implementation that returned [] for every input would satisfy "nothing disallowed survives" perfectly. Pinning the survivors is what makes the first property mean anything.

The issue warns specifically against "an inverted floor assumption". The semantics are if item_level >= floor_level: → enforced, so the assertion on survivors is rank < floor, never rank <= floor.

Notes for reviewers

Mutation results — each mutation read back before trusting it

Mutation	Result
>= floor → > floor (the inverted floor)	4 failed
drop the else that keeps permitted items	2 failed
redact appends the original, not the mask	2 failed
MaskRedactionHook stops clearing artifact_ref	1 failed
stop counting drops	1 failed
mask leaks item.text into the placeholder	1 failed
permitted items reordered (insert(0, item))	2 failed

The artifact_ref mutation initially PASSED. ContextItem.artifact_ref defaults to None, so assert result.artifact_ref is None was vacuous — it held whether or not the hook cleared anything. The strategy now generates refs, and find() pins that it still reaches that state so the assertion cannot quietly go vacuous again. Every mutation was re-run after that strategy change, per the lesson from #887 where tightening one strategy silently un-caught a different mutation.

Honest limitation — please weigh this

The existing example suite catches all seven mutations too, usually more loudly. Measured:

Mutation	test_sensitivity*.py	these properties
inverted floor	14 failed	4 failed
artifact_ref not cleared	2 failed	1 failed
drop count	21 failed	1 failed
reordering	1 failed	2 failed

sensitivity.py was already at 100% line coverage and stays there. So these properties add input-space breadth (arbitrary ladder positions × actions × Unicode text × ref presence, against six fixed fixture files) and state the contract as a universally quantified claim — but I could not demonstrate a defect they catch that the examples miss. That is the honest marginal value, and if you judge it insufficient for the CI seconds, closing this is a reasonable call; the mutation table above is the useful artifact either way.

CI budget (#755 requires this recorded)

tests/test_properties.py: 7.11s → 7.59s. The five new tests alone: 0.92s. No new dependency — Hypothesis arrived with #887.

Two stale things noticed, not fixed here

1. This PR template says "Every modified module stays ≤ 300 lines". The enforced limit is 500 (scripts/check_module_size.py, raised by refactor: reconcile invariants and module-size policy #853); docs: state the module-size limit #853 actually enforces #888 corrected the Makefile comment and workflows.md but missed the template. Worth a one-line follow-up.
2. make ci cannot go green in this environment for reasons unrelated to this change — see below.

Checklist

• Tests added or updated for every new/changed public function — no public function changed; this is test-only

• make ci passes locally — it does not, for two pre-existing environmental reasons, both reproduced on unmodified origin/main in a separate worktree:

  • tests/test_mcp_serve_cli.py::test_serve_dry_run_writes_catalog_diagnostic_event and tests/test_benchmark.py::test_matrix_cell_skip_reason_is_none_for_default_backends both fail on bc10596 with no changes applied. The proxy blocks openaipublic.blob.core.windows.net, so tiktoken falls back to the heuristic estimator.
  • Coverage reports 85.95% against an 86.0% floor — and it is 85.95% with and without my tests (identical 19146 / 2285 / 5630; only branch-partials move, 597 → 595). The floor failure is a consequence of those two skipped code paths, not of this change.

  Every other gate passes: fmt, lint, type, drift-check, module-size-check, doc-snippets-check, readme-version-check, security-policy-check, version-metadata-check, schema-hosting-check — all OK. CI is the authority on the two environmental failures.

• CHANGELOG.md updated under ## [Unreleased]

• Docstrings added for all new public APIs — n/a, no new public API

• Public-API change? — none; api/public_api.txt untouched

• Every modified module stays within the enforced size limit — make module-size-check passes (the gate is 500; see note above)

• Related issue linked — [Testing, phase 2] Complete property-based coverage of core invariants #755

• Agent-facing docs updated if pipeline, API, or conventions changed — n/a, no pipeline/API/convention change

Reproducibility

Not applicable — this touches no routing, scoring, tokenisation or context-pipeline source. The diff is tests/test_properties.py and CHANGELOG.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XhJp1Rnr6mF4vdSLkj14jU

---

Generated by Claude Code

[github-actions]

Benchmark delta (vs main)

Soft regression feedback only — this comment never blocks the PR.
Latency budget: ⚠️ when head > base × 1.3. Accuracy budget: ⚠️ when head < base - 1pp.

Routing summary (single backend × catalog sizes)

size	recall@k (head Δ vs base)	MRR (head Δ vs base)	p99 (ms)
50	✅ 0.5649 (+0.0000)	✅ 0.4978 (+0.0000)	✅ 0.439 (base 0.650)
83	✅ 0.3825 (+0.0000)	✅ 0.3242 (+0.0000)	✅ 0.390 (base 1.205)
1000	✅ 0.1475 (+0.0000)	✅ 0.1456 (+0.0000)	✅ 19.644 (base 47.855)

Per-backend × per-size matrix

backend	size	recall@k (Δ)	MRR (Δ)	p99 (ms)
bm25	100	✅ 0.3825 (+0.0000)	✅ 0.3399 (+0.0000)	✅ 3.528 (base 8.399)
bm25	500	✅ 0.2250 (+0.0000)	✅ 0.2165 (+0.0000)	✅ 16.300 (base 41.865)
bm25	1000	✅ 0.1575 (+0.0000)	✅ 0.1525 (+0.0000)	✅ 46.464 (base 112.387)
embedding_hashing	100	✅ 0.5175 (+0.0000)	✅ 0.4360 (+0.0000)	✅ 3.970 (base 7.218)
embedding_hashing	500	✅ 0.2700 (+0.0000)	✅ 0.2674 (+0.0000)	✅ 21.878 (base 43.171)
embedding_hashing	1000	✅ 0.2000 (+0.0000)	✅ 0.1931 (+0.0000)	✅ 52.524 (base 102.979)
embedding_st	100	skipped (skipped: missing sentence-transformers)	—	—
embedding_st	500	skipped (skipped: missing sentence-transformers)	—	—
embedding_st	1000	skipped (skipped: missing sentence-transformers)	—	—
fuzzy	100	skipped (skipped: missing rapidfuzz)	—	—
fuzzy	500	skipped (skipped: missing rapidfuzz)	—	—
fuzzy	1000	skipped (skipped: missing rapidfuzz)	—	—
tfidf	100	✅ 0.3825 (+0.0000)	✅ 0.3220 (+0.0000)	✅ 0.599 (base 1.193)
tfidf	500	✅ 0.2325 (+0.0000)	✅ 0.2314 (+0.0000)	✅ 5.192 (base 10.803)
tfidf	1000	✅ 0.1475 (+0.0000)	✅ 0.1456 (+0.0000)	✅ 18.595 (base 42.986)

Context pipeline (per scenario)

scenario	tokens	dropped	dedup
large_catalog	1480 (base 1480, Δ+0)	0 (base 0, Δ+0)	0 (base 0, Δ+0)
long_conversation	2500 (base 2500, Δ+0)	0 (base 0, Δ+0)	0 (base 0, Δ+0)
mixed_payload	488 (base 488, Δ+0)	0 (base 0, Δ+0)	0 (base 0, Δ+0)
short_conversation	487 (base 487, Δ+0)	0 (base 0, Δ+0)	0 (base 0, Δ+0)
stress_conversation	6590 (base 6590, Δ+0)	11 (base 11, Δ+0)	4 (base 4, Δ+0)
tiny_payload	256 (base 256, Δ+0)	0 (base 0, Δ+0)	0 (base 0, Δ+0)

---

Numbers come from make benchmark / make benchmark-matrix.
Latency is hardware-dependent — treat the markers as a rough guide.
See benchmarks/scorecard.md for the full picture.