build_ack answers letter-delimited inbounds with a readable ACK, engine and tee (BACKLOG #2875)

[wshallwshall]

Batch 265 (transports), BACKLOG #2875. This is its own PR because it also retires the DAST ingress sweep's one known_defects entry and amends ADR 0155. The fix turned that entry's strict xfail into an XPASS, so the two had to move together.

Merge danger

Reversible: yes. A hand revert of 8e514fc undoes all of it, including the policy JSON and the ADR amendment.

Blast radius:

• build_ack in messagefoundry/mllpcodec.py, which builds every MLLP ACK and NAK the engine sends.
• The vendored copy in tee/mllp.py.
• The DAST register: scripts/security/dast-ingress-policy.json (known_defects is now empty), scripts/security/dast_ingress_sweep.py, and ADR 0155 (an amendment).
• Behaviour for punctuation delimiters is unchanged (MSH#@*!%# still echoes).

Evidence

• Before: against origin/main, 33 of the 34 new engine tests fail and 10 of the new tee tests fail. The punctuation control passes on both. MSH-2 ^AB, for example, is accepted, stored and answered AA, but the ACK does not read back.
• After: all of them pass. The ACK and the NAK read back AA/AE/AR down to the subcomponent for MSH-2 ^AB, -, +, ., a duplicate character, the field separator in MSH-2, and an F field separator. The strict xfail in tests/test_harness_fuzz.py is now a plain test. A parity test pins the tee to the engine.

Builder report: lane A, BACKLOG #2875

Source branch b265-2875, head 8e514fcaadc608d9aff31ab2de58f7797375a524. That is 1 commit on origin/main 8ba57ee.

• The rule. _ack_echo switches to |^~\& when MSH-1 or MSH-2 holds a character of the ACK's own fixed text, or when two delimiters share a character. The fixed-text characters are the segment ids, ACK, P, the MSA-1 codes, the E and F escapes, digits, +, - and .. The shared-character test counts the parser's default fill of a short MSH-2.
• Echoed values under the fallback are blanked, not escaped. A value is kept only if every default delimiter in it has the same role in the inbound and it holds neither escape character. Escaping would flatten the value's structure, and an escaped control id would no longer match the stored one (verify_ack_control_id).
• The tee. The tee is a stdlib-only parallel copy. Its MSH and MSA split also changed, because a field separator of M, S, H or A used to split the segment id.
• Checks:
  • ruff is clean, and all three mypy legs plus --strict tee/mllp.py are clean.
  • The full pytest run with -n 6 gave 47673 passed, 1166 skipped and 151 xfailed. That run started before the last small edit. The affected files were re-run green afterwards: test_tee_mllp, test_harness_fuzz, test_dast_ingress_sweep, test_mllp_nak_detail_bound, test_transports and test_hl7_core_features.
  • Two earlier failures did not reproduce in isolation: a uvicorn port-in-use error, and an xdist worker crash on an existing xfail.
• Where the brief and the tree disagree:
  • The brief did not list the DAST register files; they had to move with the fix.
  • The brief said to fall back on any alphanumeric. The trigger was narrowed to the ACK's own characters plus shared characters, so MSH-1 D is now echoed and reads back with its control id intact.
  • The brief cited a tee MSH-7 fix as precedent. No such fix exists; the tee's comment cites BACKLOG EXPERIMENT (unread result): run the windows-2025 harness tier at -n 3 -- BACKLOG #1109, against #1304. Do not merge on one green run. #1196.
  • The module docstring of messagefoundry/transports/mllp.py still says the ACK echoes the inbound's encoding characters. That file was another lane's, so it was not touched.
• Open findings, not fixed:
  1. MSA-3 still escapes only the escape character and the field separator. A component or repetition delimiter outside the fallback set (for example D~\& or :~\&) still splits a NAK text at component level. A full fix runs into the redactor and the event-loop budget tests.
  2. A caller-supplied control_id or timestamp is written verbatim, even after a silent fallback. No caller passes control_id today.
  3. When the inbound version holds an unsafe character, the fallback writes "2.5.1" in its place.
  4. When MSH-2 holds a duplicate character, _roles gives it its first role in MSH-2 order. The parser may read it differently.
  5. Reviewers suggested two deeper alternatives: build the ACK, parse it back, and rebuild under the defaults on a mismatch; or reuse _builtin_hl7.separators and escape_leaf. The leaf import rule blocks the reuse.
  6. The rule is restated in several docstrings (SDS-3.5).
  7. The engine build_ack unit tests sit in test_harness_fuzz.py; test_transports.py is their natural home.
  8. messagefoundry/CLAUDE.md says "don't hardcode |^~\&". This fallback is a deliberate exception and should be recorded there.
• Banner: BACKLOG #2875 FIXED: build_ack falls back to |^~\& when the inbound delimiters collide with the ACK's fixed text; the tee matches; the strict xfail and the DAST known defect are retired.

Hosted legs to read

• The linux pytest and mypy legs.
• The advisory nightly dast.yml dast-ingress job. Its known_defects is now empty, so it tolerates no reply findings.

[wshallwshall]

QA (lane A, BACKLOG #2875, head 8e514fc): code-review skill at level xhigh (BUILDER.md step 11), args exactly as briefed. Three runs; a JSON findings list each time, no separate tag. Repair rounds followed runs 1 and 2. A third, limited round went past the two-round cap for two reasons: the fix turned a DAST strict xfail into an XPASS, which would turn the default pytest leg red; and round 2 had blanked MSA-2 for letters the ACK never writes (D, Q, Z), which would make verify_ack_control_id retry forever. A round-1 MSA-3 change was reverted because it broke the PHI redactor and the event-loop budget tests. Shipped open: findings 1-8 in the PR body.

[wshallwshall]

Lander: enqueuing at pinned head 8e514fc. QA (xhigh) covers this head, and no finding needs the owner. Every open finding is filed in the vault ledger (#3725; vault PR 2494, merged 11:16Z, completeness-reviewed). Required checks are green; the only red, 'ASVS anchors (advisory)', is not required.