[Security] Version benchmark evidence identity against Unicode default-ignorable profile

[seonghobae]

Gap

Active benchmark PR #322 now fails closed on C0/C1 controls, U+2028/U+2029, and the exact Unicode Bidi_Control set for ControlledBenchmarkRunContext. That closes the reviewed line/paragraph and directional-rendering cases, but it is not yet a complete invisible-format profile.

Fresh source inspection of #322 exact a4c8ceaf67a075ef483334802aacfc54cf502068 shows validate_run_context_field admits any scalar that is not char::is_control, U+2028/U+2029, or one of the 12 bidi controls. Other Default_Ignorable_Code_Point values therefore remain admissible, including classes represented by zero-width/default-ignorable formatters and variation selectors. These benchmark-owned strings are later rendered in logs/reports and bound into evidence identity, so a byte-distinct value that is visually absent or presentation-dependent can create review/audit ambiguity.

This is narrower than general Unicode text policy. It applies only to OriginWeave-owned benchmark reproducibility identifiers. Browser-issued protocol addresses such as WebDriver BiDi browser.UserContext remain lossless external protocol identity and must not inherit this grammar.

Standards basis

Use the exact adopted Unicode profile, not a hand-maintained folklore list:

• Unicode Standard Annex feat(evidence): add purpose-bound sensitive access receipts #31, Unicode Identifiers and Syntax, Unicode 18.0.0, Revision 45, 2026-09-01, §7.3 Default-Ignorable Exclusion Profile: exclude code points with Default_Ignorable_Code_Point from identifier Start/Continue; the annex notes that these characters often have no visible effect and can enable spoofing.
• Unicode Technical Standard test(mv3): require repeated compatibility evidence #39, Unicode Security Mechanisms, Unicode 18.0.0, Revision 34, 2026-08-27: the General Security Profile classifies Identifier_Type=Default_Ignorable as Restricted. ZWJ/ZWNJ are explicitly Default_Ignorable/Restricted unless a documented tailored context is adopted.

References:

• https://www.unicode.org/reports/tr31/tr31-45.html
• https://www.unicode.org/reports/tr39/tr39-34.html

Ownership and compatibility boundary

• [Product Gap] Establish a release-grade web-agent benchmark and commercial acceptance gate #203 remains the release-grade benchmark/security-adversarial owner and already requires hostile Unicode/control/bidi/invisible-character evidence.
• fix(core): reject bidi controls in benchmark evidence identity #322 remains the bounded current repair for bidi plus line/paragraph rendering controls; do not turn it into an unversioned catch-all.
• fix(evidence): reject provenance locator control injection #162/fix(evidence): reject ambiguous network metadata names #165 contain analogous evidence-presentation hardening in another bounded context. They are comparison evidence only; do not copy mutable source or make originweave-core depend on an unrelated evidence aggregate merely to reuse a helper.
• If a reusable Unicode evidence-identity profile is warranted, introduce it only as a small versioned Shared Kernel/value policy with explicit consumers and tests. Otherwise keep the benchmark profile local and versioned.

RED → repair acceptance

1. Add test-first hostile cases proving the current exact predecessor admits representative default-ignorables outside fix(core): reject bidi controls in benchmark evidence identity #322's existing set. At minimum exercise U+200B ZERO WIDTH SPACE, U+200C ZWNJ, U+200D ZWJ, U+2060 WORD JOINER, U+FEFF ZERO WIDTH NO-BREAK SPACE/BOM, and representative variation-selector behavior such as U+FE0F.
2. Do not stop at those examples. The production decision must be tied to an exact Unicode-versioned property/profile so later Unicode property changes are reviewable rather than silently changing the accepted grammar.
3. Decide and document whether the benchmark identifier grammar rejects every Default_Ignorable_Code_Point or uses an explicit tailored exception profile. If any exception is permitted, record the exact scalar/sequence context and security rationale. Emoji/join-control usability guidance in UAX feat(evidence): add purpose-bound sensitive access receipts #31 / UTS test(mv3): require repeated compatibility evidence #39 is not an implicit exception.
4. Preserve visible Korean/Japanese/Chinese/Vietnamese/Spanish/German/French text and ordinary Arabic/Hebrew text that does not require an excluded formatting scalar. Do not widen this repair into ASCII-only admission.
5. Keep byte identity semantics. Do not silently add NFC/NFKC/case folding/confusable skeletonization; those are separate compatibility/security decisions with their own migration requirements.
6. Validation must run for both expected and observed contexts before equality can influence benchmark evidence.
7. Public diagnostics/rustdoc, doctoring, TRACEABILITY, CHANGELOG, and the product-technical gap baseline must name the exact adopted Unicode version/profile and reversal/migration path.
8. Owned production docstring/rustdoc, test and edge-case coverage remain complete; function/line/region/branch coverage remains 100%.
9. Exact-head repository contracts, rustfmt, locked tests, strict Clippy, rustdoc/API docs, security/review workflows, and applicable real benchmark/browser evidence must execute. Skipped, queued-only, predecessor, status-only, or synthetic evidence is non-passing.

Non-goals

• normalizing or restricting browser-issued WebDriver BiDi/Chromium identifiers;
• claiming complete Unicode spoofing resistance from default-ignorable exclusion alone;
• introducing confusable/mixed-script restrictions without a separate threat/compatibility decision;
• copying fix(evidence): reject provenance locator control injection #162/fix(evidence): reject ambiguous network metadata names #165 source across bounded contexts;
• altering benchmark thresholds, release/signing authority, workflows, provider/model routing, or browser policy.

[seonghobae]

Implementation successor opened as #324, stacked directly on #322 exact a4c8ceaf67a075ef483334802aacfc54cf502068. Test-first 592a1a3bc49a922df86285eab332308770b77474 covers the representative default-ignorables named here; production 19a37a667baeffe824c4fb025942eecb40bc67c8 pins an explicit unicode-18.0.0-default-ignorable-exclusion profile using the 4,174-code-point Unicode 18.0.0 DerivedCoreProperties.txt set with no tailored exceptions; 02edb5b2cb5ff9e1237afbc83d9a0f3e7aa48d6a extends range-boundary coverage; 641c1eee5d2a2184f4e04d5630a8839db35d407f updates doctoring and migration/reversal rationale. #324 remains Draft: hosted Rust RED/GREEN is not claimed, and lossless CHANGELOG / product-gap-baseline updates plus exact-head gates remain outstanding.

[seonghobae]

Current implementation/provenance status update: #324 remains the valid bounded implementation successor at exact 728c3a8b81ee6bcb1a7e8aa2ed2667e7a88fb050, and fresh CodeRabbit review 619ef893-d263-4b3f-bb0d-05c41eaec0a7 completed that exact head with no actionable comments and 100% touched-diff docstring coverage. A Ready probe then reproduced organization runner starvation: CI 34925029740 jobs 104241152237 (Rust contracts) and 104241152529 (Production coverage) had steps=[] and no runner assignment; MV3 34925029797 was likewise queued. Exact evidence was handed to .github#712.

A separate release/provenance finding is now #325. The 4,174-scalar table matches the DerivedCoreProperties-18.0.0.txt snapshot dated 2026-08-07, but on 2026-09-15 the versioned Unicode UCD URL still redirects to mutable /Public/draft/ucd/DerivedCoreProperties.txt; formal Unicode 18.0.0 release is still pending. #324 was returned to Draft, so those probe runs became cancelled and subsequent Draft-policy runs are skipped. Preserve the current source/test delta, but treat it as a version-pinned pre-release UCD snapshot until #325 reconciles it against the final versioned Unicode 18.0.0 DICP dataset. #323 stays open; no GREEN/release claim.

[seonghobae]

Current authority supersedes the 2026-09-15 pre-release note above. Unicode 18.0.0 publication/provenance is no longer open: #324 exact accdd2d194f21ae1444ccca5297ce6590bc5384e is Ready / mergeable and pins unicode-18.0.0-default-ignorable-exclusion against the stable versioned DICP receipt (1,159,889 decoded bytes, SHA-256 09c928886a178fcafd93c29e4bd59073a058e5a100b716d425cb563ab50f68c9; 27 source entries / 4,174 scalars; normalized 29,218 bytes, SHA-256 673264e62183e35f6055a2ad4940403e706669e0750fcc5d56a99f158fb3bb93). The independent full-property fixture at 506b315d3beb2059389ab8b66f7f3399b4f9c1d4 drives all 4,174 scalars through the public benchmark admission boundary; current-head review found no blocking issue in scope.

Acceptance item 7 is now also source-backed by the independent product-gap-baseline owner: #238 exact f47ed08685fca01bf4217b0476f9babcaac3e0af ordinary-forward currentizes docs/product-technical-gap-baseline.md, its navigation contract, and TRACEABILITY with the stable Unicode artifact/profile, migration rule, and current #324 authority. #238 remains Draft / mergeable and its exact required workflows are nonterminal, so this is not protected-main/release truth yet.

#323 therefore stays open for executable acceptance rather than source/provenance design: #324 CI 35531334971 (Rust contracts + Production coverage) and MV3 35531334972 are still queued with no runner assignment; #238 current workflows are likewise nonterminal. Queued/skipped/predecessor evidence is not GREEN. Do not reintroduce the superseded “formal Unicode 18 pending / mutable draft UCD” interpretation.