FhirPeek refuses a repeated JSON key on every parse path (BACKLOG #2812)

changelog.d/2812-fhir-repeated-json-key-refused.security.md

@@ -0,0 +1,12 @@
+- **`FhirPeek.parse` now refuses a FHIR body in which a JSON object repeats a key.** A body for a
+  single-resource interaction, such as create, update or a conditional one, that repeats a key is
+  dead-lettered with code `bad-body`, and nothing is sent. A Router that peeks such a body takes the
+  error path, and a `fhir_lookup` whose reply repeats a key fails. A wrapped update's 2xx reply that
+  repeats a key is read keeping the first and keeping the last value of each key. When the two
+  readings would be judged differently, the update is a permanent refusal, code
+  `reply-repeats-a-key`, and is not recorded as delivered; ADR 0095 Amendment B.1 states the rule.
+  The text of the `bad-body` refusal now names which check fired. A `transaction` or `batch` Bundle
+  was already refused for a repeated key (vault BACKLOG #2804). Other reads still keep the last of
+  two equal keys, at least `FhirResource.parse`, `RawMessage.json()` and the classification of
+  other FHIR replies. See [CONNECTIONS.md](../docs/CONNECTIONS.md), "Interactions".
+  (`vault BACKLOG #2812`)

docs/CONNECTIONS.md

@@ -2389,6 +2389,13 @@ source (`Http()`, File, a `Loopback` re-ingress) as a `RawMessage`.
 per entry); the engine never orchestrates cross-entry atomicity. The engine does screen every entry first,
 as [A Bundle connection screens every entry](#a-bundle-connection-screens-every-entry) says.
 
+On every interaction but `transaction` and `batch`, a body in which a JSON object repeats a key is refused
+before a byte is sent, for the reason rule 2 of
+[the Bundle screen](#a-bundle-connection-screens-every-entry) gives: a permanent refusal, code
+`bad-body` (vault BACKLOG #2812). A Bundle that repeats a key is refused by the screen instead, code
+`bundle-entry-refused`. `FhirPeek.parse` raises `FhirPeekError` on such a body, so a Router that peeks one
+takes the error path, and so does a `fhir_lookup` whose reply repeats a key.
+
 **Conditional knobs (idempotency / concurrency).** FHIR's native answer to the at-least-once duplicate
 problem — opt-in, off by default: `if-none-exist` (create only if no match; the search rides the
 `If-None-Exist` **header**), `conditional-update` (the server resolves which resource to update; the search
@@ -2456,6 +2463,15 @@ What a site needs to know:
 - If a server answers 2xx while the entry's own `response.status` failed, the message is classified on
   that entry status, like any other HTTP status. An entry status that does not read as an HTTP code is
   logged as a warning, and the 2xx reply counts as delivered.
+- A reply in which a JSON object repeats a key is read twice, once keeping the first value of each key
+  and once keeping the last (vault BACKLOG #2812). When the two readings reach the same judgment and
+  agree on what a delivery takes, and no occurrence of a key repeated three or more times differs
+  from both its first and last value, the reply is judged as above. Otherwise the update is a permanent
+  refusal, code `reply-repeats-a-key`, in either capture mode. It is not sent again and not recorded
+  as delivered, as with a reply over the byte bound below. The full rule, with what counts as
+  agreement and its costs, is in
+  [ADR 0095](adr/0095-connection-lifecycle-scheduler-and-credential-fault-stop.md), Amendment B.1,
+  the 2026-10-17 entry.
 - A reply body over the byte bound is not kept, so the engine cannot see the entry's status. The
   update is then a permanent refusal, code `reply-too-large`, in either capture mode, and it is not
   sent again. It is not recorded as delivered. See *A 2xx reply over the byte bound* under [REST](#rest--rest).

docs/adr/0095-connection-lifecycle-scheduler-and-credential-fault-stop.md

@@ -413,3 +413,41 @@ the lane, and only because the proxy handler screened a challenge that was not i
 
 Tests: `tests/test_digest_refusal_classification.py`, the section headed BACKLOG #2989, and
 `tests/test_over_cap_2xx_reply_is_sent_once.py`, the section at its end.
+
+#### B.1, 2026-10-17: a FHIR update reply that repeats a JSON key and reads two ways (vault BACKLOG #2812, batch 265)
+
+> **Recorded 2026-10-17 by a Builder seat for batch 265.** It describes the code in the pull
+> request for vault BACKLOG #2812, not the #2989 work above. This entry is the one statement of the
+> rule; other documents link here.
+
+A FHIR `update` or `if-match` goes out as a one-entry `transaction`, and the engine reads the
+entry's status in the 2xx reply to judge it. `json` keeps the last of two equal keys and a server
+may mean the first, so a reply that repeats a key is read twice, keeping the first value of each
+key and keeping the last. When both readings reach the same judgment (delivered with one capture
+outcome, retried, or dead-lettered with one code) and agree on the values a delivery takes (the
+resource type, whether there is an entry response, and the entry's `etag`, `location` and
+`lastModified`), the reply is judged as any other. So `200 OK` against `201 Created` is delivered,
+and two statuses that both classify as transient are retried. When the readings differ in any of
+those, in either capture mode, or the keep-first read cannot parse a body the plain read takes,
+the delivery is a permanent refusal, code `reply-repeats-a-key`, not sent again and not recorded
+as delivered, as a reply the engine needs and cannot read is above. The same refusal applies when
+a key occurs three or more times in one object, at any depth, and some occurrence equals neither
+its first nor its last value, because the two readings alone would never see it (so
+`200 OK`, `404 Not Found`, `200 OK` dead-letters, as does such a repeat inside `meta`).
+
+It fails closed only on disagreement because a reply that reads one way under both rules carries
+no ambiguity to resolve, and refusing it would dead-letter an update that applied, which a replay
+of an `if-match` update turns into a 412. On `main` before this, `404 Not Found` then `200 OK` was
+delivered on the last key.
+
+**Provenance: decided 2026-10-17 under the owner's standing driver rule, by the Lander's send-back
+of engine PR 2356, and narrowed by the batch 265 Manager to compare judgments rather than status
+text. The owner has not ruled on it and may overturn it.**
+
+Costs, at least these. A reply nested in the narrow band between the hook's depth limit and plain
+`json`'s is refused with this code although it may repeat no key, and its text then says it does.
+An ETag that differs with capture off refuses, though nothing reads it in that mode.
+
+It is raised in `transports/fhir.py` (`_agreed_reading`, comparing `_reply_reading`), not by
+`read_2xx_reply_text`. Tests: `tests/test_fhir_transport.py`, the section headed vault BACKLOG
+#2812.

messagefoundry/parsing/fhir/peek.py

@@ -31,7 +31,11 @@
 
 from messagefoundry.parsing.fhir._deps import load_fhirpathpy
 from messagefoundry.parsing.fhir.errors import FhirPeekError
-from messagefoundry.redaction import json_loads_or_refusal
+from messagefoundry.redaction import (
+    RepeatedKeyError,
+    json_loads_or_refusal,
+    refuse_repeated_keys,
+)
 
 __all__ = ["FhirPeek"]
 
@@ -56,7 +60,17 @@ def parse(cls, raw: str | bytes, *, format: str | None = None) -> FhirPeek:
         """Peek the FHIR resource in ``raw``.
 
         Raises :class:`FhirPeekError` if the body is not parseable FHIR JSON (not JSON, or not a resource
-        *object*), or if it is FHIR-XML (deferred to a hardened path; JSON-only MVP per ADR 0022)."""
+        *object*), or if it is FHIR-XML (deferred to a hardened path; JSON-only MVP per ADR 0022).
+
+        Also refused: a JSON object that repeats a key, at any depth (vault BACKLOG #2812; see
+        :func:`~messagefoundry.redaction.refuse_repeated_keys`). A caller gates on the peeked value
+        while the body itself goes on, sent verbatim or read by other code, so it is refused rather
+        than canonicalised. A caller may judge a refused body further on its own terms, as the
+        wrapped-update reply does. Only keys equal once JSON-decoded count, so gaps remain,
+        at least these: a server that folds case, normalises Unicode, or cuts a key at an escaped
+        NUL can still read two keys the peek reads as distinct. The other way, ``bytes`` are decoded
+        with replacement first, so two distinct invalid-UTF-8 keys can collapse into one and be
+        refused. The refusal names neither the key nor a value."""
         if isinstance(raw, (bytes, bytearray)):
             raw = bytes(raw).decode("utf-8", "replace")
         raw = raw.lstrip("")  # tolerate a leading UTF-8 BOM (json.loads would otherwise choke)
@@ -67,7 +81,9 @@ def parse(cls, raw: str | bytes, *, format: str | None = None) -> FhirPeek:
             )
         # PHI rule: name the failure, never echo the body. No handler here, so the decode error
         # (which holds the body) is on neither chain (BACKLOG #2048).
-        parsed, refusal = json_loads_or_refusal(raw)
+        parsed, refusal = json_loads_or_refusal(raw, object_pairs_hook=refuse_repeated_keys)
+        if refusal == RepeatedKeyError.__name__:
+            raise FhirPeekError("a JSON object in the FHIR body repeats a key")
         if refusal is not None:
             raise FhirPeekError(f"body is not parseable JSON for FHIR ({refusal})")
         if not isinstance(parsed, dict):

messagefoundry/redaction.py

@@ -90,6 +90,7 @@
 from typing import Any
 
 __all__ = [
+    "RepeatedKeyError",
     "clamp_untrusted",
     "codec_safe_line",
     "codec_safe_str",
@@ -99,6 +100,7 @@
     "prints_a_codec_error",
     "redact",
     "redact_untrusted",
+    "refuse_repeated_keys",
     "safe_error",
     "safe_exc",
     "safe_name",
@@ -3178,7 +3180,7 @@ def json_loads_or_refusal(
     ``UnicodeDecodeError`` on bytes, or json's depth-limit ``RecursionError``, which is a
     ``RuntimeError`` a ``ValueError`` arm does not reach -- BACKLOG #1600). A ``ValueError``
     subclass that ``object_pairs_hook`` raises comes back as its class name too, so a caller can
-    tell its own refusal apart (vault BACKLOG #2804 refuses a repeated key that way).
+    tell its own refusal apart: :func:`refuse_repeated_keys` refuses a repeated key that way.
 
     It returns rather than raises so a caller's refusal is raised outside any handler (BACKLOG
     #2048). The decode error holds the WHOLE input (``JSONDecodeError.doc``,
@@ -3193,3 +3195,21 @@ def json_loads_or_refusal(
     except (ValueError, RecursionError) as exc:
         refusal = type(exc).__name__
     return None, refusal
+
+
+class RepeatedKeyError(ValueError):
+    """A JSON object repeats a key. Carries neither the key nor any value, so its class name, the
+    hint :func:`json_loads_or_refusal` returns for it, is content-free."""
+
+
+def refuse_repeated_keys(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
+    """An ``object_pairs_hook`` for :func:`json_loads_or_refusal` that refuses a repeated key at any
+    depth, raising :class:`RepeatedKeyError`.
+
+    ``json.loads`` keeps the LAST of two equal keys; a server may keep the first. A body the engine
+    gates on one value and forwards verbatim would then be read two ways (vault BACKLOG #2804 for a
+    Bundle, #2812 for a single FHIR resource), so it is refused rather than canonicalised."""
+    obj = dict(pairs)
+    if len(obj) != len(pairs):
+        raise RepeatedKeyError
+    return obj