Bind message signatures to the overlay and bound DHT storage - #724
Bind message signatures to the overlay and bound DHT storage#724RyanKung wants to merge 21 commits into
Conversation
Closes #704, closes #701. Every MessageVerification now signs a domain-separated transcript `len(tag) || tag || network_id || ts || ttl || data`, verified against the receiver's own overlay and message family. Transaction and payload signatures over the same hash are no longer interchangeable, node descriptors and onion backward payloads sign under their own tags, and a signature issued inside one overlay does not verify inside another. Message constructors and PayloadSender take a MessageSigner (a session key acting inside one overlay); MessageVerificationExt::verify takes the receiver's network_id. DHT entries carry a retention bound stamped at the operation boundary with DEFAULT_TTL_MS and joined by max; every storage read retires an expired value before serving, replicating, or acknowledging it. The single write funnel rejects entries whose bound is expired or beyond MAX_TTL_MS, and whose CRDT versions run ahead of the receiver's clock by more than the message skew tolerance, so a forged register floor cannot pin a key. The native file store enforces its byte budget by retiring the least recently written keys and rejecting oversize values, and the fetched-entry cache is bounded; both share one write-ordered map. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signatures from earlier builds no longer verify, so the wire change gets a minor bump; the changelog records the breaking changes and the storage bounds. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Retention policy is now a property of the entry kind. Data entries keep DEFAULT_TTL_MS / MAX_TTL_MS (10 min / 100 min); relayed messages held for an offline destination are stamped with DEFAULT_RELAY_ENTRY_TTL_MS (24 h) and admitted up to MAX_RELAY_ENTRY_TTL_MS (7 d), so a peer that returns within a day of the last relayed message still receives its inbox. Materialization also bounds each carrier in encoded bytes next to the existing payload count (ENTRY_DATA_MAX_BYTES = 1 MiB for data, RELAY_INBOX_MAX_BYTES = 16 MiB for a relay inbox), keeping the newest payloads that fit, so an inbox that keeps receiving messages retains only its most recent ones and a payload that alone exceeds the budget is dropped rather than retained over it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Pushed a1856c8: retention is now a property of the entry kind, and each carrier has a byte budget.
The 1 MiB / 16 MiB figures are proposals; both are one constant each. |
…e receiver overlay Review round for #724 (three independent reviews). The whole-carrier byte budget is withdrawn: "the newest payloads that fit" depends on payloads a replica may already have dropped, so it is not a lattice morphism and replicas diverge. Admission instead bounds every payload element at ENTRY_PAYLOAD_MAX_BYTES, an element-intrinsic predicate that commutes with union; with ENTRY_DATA_MAX_LEN it bounds a carrier at their product. The relay-inbox policy is withdrawn as well: EntryKind::RelayMessage has no production producer and the kind is peer-declared, so a wider policy only widened an attacker's budget (#725 tracks the real inbox). Retention now has one policy and lives in dht/entry/retention.rs with its laws. Admission is applied to the peer-supplied delta, never to the join result: PeerRing::operate_storage_entry is the single funnel for local and remote operations, join_storage_entry admits replicated values, and each boundary reads the clock once (the sync batch shares one admission time between validation and persistence). The wall-clock wrappers on Entry and EntryOperation are gone; the operation-boundary time is explicit. Descriptors are signed by the node's MessageSigner and verified under the receiver's network_id; signing a body that states another overlay is refused, so the matches_network post-filters are redundant and removed. MessageSigner is generic over how it holds the key (borrowed, Copy form for signing functions; owned form for chunked transfers and onion exit runtimes) and is the only signing authority. Domain tags are built with domain_tag!, which checks the length-prefix law at compile time. SledStorage writes the temporary file before touching the index, restores the previous record when the rename fails, clears per file, removes stale .tmp files on open, and retires a record the current schema cannot decode (postcard is not self-describing, so a legacy value cannot be lifted by a serde default). Lock poisoning is reported as Error::StorageLockPoisoned. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Pushed db7d770, addressing the three independent review rounds (consensus items first). Design
Abstraction
Not changed, on purpose
|
Closes #725. A CustomMessage that reaches the node responsible for its destination's ring position while the destination is not connected is held in the carrier `destination + 1` (EntryKind::RelayMessage) by an ordinary Extend write, routed to the carrier's storage owner like any other. Storage admits a relay element only under a witness the owner verifies itself: it decodes as a signed MessagePayload addressed to the inbox's peer, carries an application message, and both signatures verify inside the local overlay regardless of proof liveness (MessageVerificationExt::verify_signature) because retention, not the proof, governs how long the inbox holds a message. The kind is therefore no longer self-declared, which makes the per-kind policy safe: inboxes are retained 24 h after the last held message, up to 7 d. Once the peer is online its inbox key lies in its own storage interval and ownership hand-off moves the carrier to it. Every stabilization round the peer drains its local inbox to the application (on_validate, then on_inbound) and compacts the delivered messages out of the carrier; compaction now applies to relay carriers, so delivered elements and their tombstones fall below the floor at every replica. The stabilizer carries the swarm callback for this, so Swarm::stabilizer returns a Result. Const-generic redundancy stays at the public storage API only; the internal operate/fetch helpers take the configured redundancy. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Pushed 1a614af: the relay inbox (#725) is now implemented in this PR instead of being deferred, so the PR also closes #725.
|
The ownership hand-off (sync_entries_with_successor) ran only inside the NotifyPredecessorReport handler, one of three inputs that can move the successor head; a head moved by correct_stabilize's topology query or by a directly connected peer left entries at a node that no longer owned them until they expired. Placement is a function of the ring state, so the stabilizer now offers the live local entries beyond (self, head] to the current head every round; deliveries are joins and cleanup is ack-gated, so repetition is idempotent. A transition-triggered one-shot send was rejected: it can precede the peer's admission of this node and is dropped, and repair does nothing at redundancy 1. The report handler only connects now, and the SendStorageSync effect, whose sole producer it was, is removed. topology::successor_head names the head for find_successor and the stabilizer alike. Closes #726. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 5 — 73487b6: ownership hand-off as a stabilization invariant (#726). Finding. Change. The stabilizer runs Why not a transition effect. I first emitted a Tests. |
The pure topology step now states the head law: a transition whose successor head moves emits SuccessorHeadChanged(head), once, after its own actions. The shell lowers it to CoreEffect::RequestStorageRepair, which only records the repair intent; sending at admission time was rejected because the delivery can precede the peer's own admission of this node and is dropped. The storage repair pass restores the whole placement invariant: it drains the local inbox, offers the live entries beyond (self, head] to the head as an ack-gated ownership hand-off, and republishes to missing affine owners, all through the existing repair window with its fresh-connection grace. The maintenance loop runs the pass when requested and periodically, so the inbox drain, previously reachable only through the one-shot stabilize(), now runs in production as well. Closes #726. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 6 — 01b82e1: the head change requests a repair round instead of sending (#726). Head law (pure). Placement repair pass. A production gap this closed. Tests. Trade-off to be aware of. Hand-off latency is now bounded by the repair schedule: the quiet gap after stabilization, the 30 s fresh-connection grace, and one delivery batch per repair step. For the inbox that means a returning peer receives its held messages roughly half a minute after connecting. Lowering the grace for |
…ing-placed Review of the relay inbox (#725) found it unsound as shipped: it never drained under the production storage mode, its compaction floor dropped undelivered messages and could be pushed by anyone, its witness bound only the content shape so any peer could fill, replay, read, or wipe a victim's inbox, and admissibility depended on the sender's session staying live. An inbox element is now a HeldMessage: the payload wrapped and signed by the node that held it, under its own domain tag, with the hold instant as the signature timestamp. The witness verifies the holder inside the receiver's overlay and the payload as of the hold instant (MessageVerificationExt::verify_at; the liveness-free verify_signature is gone), requires a CustomMessage addressed to the recipient, and rejects a reset floor. Authority is checked at the write: a hold only from the node the owner routes the destination to (PeerRing::inbox_hold_authority), a removal only from the recipient, a relocation only as an ownership hand-off from the predecessor; a relay carrier is never fetched, cached, replicated, or returned to a lookup by anyone but its recipient. Removal is per element by add dot; the inbox keeps the newest RELAY_INBOX_MAX_LEN elements and as many tombstones. Relay carriers are placed by the ring geometry in every storage mode, so the placement law holds with virtual nodes enabled. Delivery goes through the inbound pipeline (validation, dispatch, on_inbound under the inbound deadline) via swarm::callback:: deliver_local_payload, and the stabilizer resolves the swarm callback at delivery time, so Swarm::stabilizer is infallible again and set_callback after listen is honoured. The storage maintenance phase is two steps, deliver_inbox then repair_storage; a head change is a local PeerRingAction::StorageRepairDue. Also: a node with successors but no predecessor no longer claims the whole ring and its interval is (predecessor, self] exactly; a destination counts as offline only with no connection attempt at all; ENTRY_PAYLOAD_MAX_BYTES is 32 KiB with a compile-time law that a full carrier fits one transport message; descriptors verify only when the body states the receiver's overlay and onion backward payloads are verified under the receiver's overlay; MessageSigner::to_owned is owned; storage count overflow has its own error. Closes #726. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 7 — 3083402: the relay inbox is redesigned after a three-reviewer read of the full diff (findings summarized below), plus the non-inbox findings from the same review. Inbox findings that were structural (all fixed here):
Other findings fixed: the head change is a local Left as is, deliberately: Trade-off stated in SECURITY.md: a malicious holder is one identity in one ring position; it can hold junk for the peers it is responsible for (newest 64 per inbox) or redeliver inside the sender's proof lifetime, and every element names it by signature. Held messages are relocated in the clear between owners like every DHT value. Gates: core dummy 726, node 323, clippy native / wasm core / browser / ffi, rustdoc, taplo, typos, nightly fmt. |
…ailing the batch Self-review of the inbox redesign: a relay carrier offered by anyone but the receiver's predecessor, or under any purpose but an ownership hand-off, was rejected with an error, which failed the whole sync batch including the data topics sharing it, the same one-element-rejects-all non-monotonicity the review found in admission. The carrier is not invalid, only not this receiver's to take yet, so it is now skipped without an acknowledgement (its owner keeps it and offers it again) and the rest of the batch is accepted; test added. Also: the hold authority is derived only for a hold, not for a removal; the stacked doc comment on PeerRing::operate_storage_entry is merged; a redundant error alias and an inline path in a test are removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 8 — 61c6cb4: self-review of round 7 (full read of Found and fixed:
Checked and sound: the holder signs the canonical postcard encoding of the payload (the same bytes Gates: core dummy 727, node 323, clippy native / wasm core / browser / ffi, rustdoc, taplo, typos, nightly fmt. |
…ner key, shared fixtures The items left open after the round-7 review: - DHTSyncLockError, CallbackSyncLockError, and StorageLockPoisoned named one failure; they are Error::LockPoisoned. - storage::sled::SledStorage has been a file-per-key store under a byte budget since the budget landed; it is storage::file::FileStorage. - MessageSigner no longer hands out its session key; the onion runtimes only needed session_public_key. - StorageSyncBatch::new takes the admission clock instead of reading it. - The retention fixtures (bounded, live_at, expired) are one family in crate::tests (live, live_entry, with_retention, expired), and every fully qualified crate::tests::live_entry call site imports it; the test_entry NOW_MS constant is declared before its uses. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 9 — f2a54c9: the items round 7 left open are closed.
CHANGELOG records the three renames under breaking changes. Gates: core dummy 727, node 323, clippy native / wasm core / browser / ffi, rustdoc, taplo, typos, nightly fmt. |
…, retire per element
Fresh review of the relay inbox:
- The relocation law read its sender from the relay path, a peer-declared
field; any connected node could forge a hand-off from the receiver's
predecessor and inject held messages. The sender is now the transaction
signer, the one origin the transport authenticated.
- The hold instant is signed by the holder and was bounded only above, so
a holder could judge any old message at a time of its choosing and the
documented replay bound did not exist. The write law now requires the
held message's sender proof to be live by the owner's own clock; the
witness stays timeless for relocation and delivery, which keeps them
monotone.
- Authority is judged before any signature is verified and a relay delta
larger than the inbox is refused before the first verification, so a
stranger costs the owner nothing.
- The drain tombstoned only after delivering every element; cut short by
the step deadline it redelivered the same prefix forever. It now retires
each element as it is delivered, and builds the delivery pipeline once.
- The witness binds the payload's relay destination too, so a drained
message is never forwarded; a relay carrier is absent to every lookup.
Also: Entry::partition_inbox / handlers::inbox::deliver_inbox naming,
EntryKind::{is_relay_inbox, replication}, topology::is_responsible_for as
a pure law with tests, SwarmCallbackSlot newtype, LocalDelivery,
MessageSigner::sign_at, MessageVerification::{verify_live, verify_live_at}
replacing verify_unexpired, DomainTag::new asserts its law once,
EntryWitness::Reference, SwarmTransport::session, placed_beyond,
StorageSyncPurpose::is_ownership_handoff, one retire-on-read step,
FileStorage docs and test shape, and the tests the review found vacuous.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 10 — d78f7fb: fresh three-reviewer read of the full diff, plus my own; everything below is fixed in this commit. Correctness (relay inbox)
Abstraction / naming / docs
Gates: core dummy 730, node 323, clippy native / wasm core / browser / ffi, rustdoc, taplo, typos, nightly fmt. |
The stabilizer owned the application callback slot and called through message::handlers::inbox into swarm::callback to deliver the inbox, so the DHT layer depended on the swarm's delivery pipeline. It now names the one intent the storage maintenance phase has toward the application layer, dht::InboxDelivery::deliver_inbox, and Swarm::stabilizer injects the swarm's interpreter, swarm::inbox::SwarmInboxDelivery, which owns the callback slot and the LocalDelivery pipeline. message::handlers::inbox keeps only the producer (holding for an offline destination). The tombstone-cap test read a synthetic clock while its holder signatures read the real one; two rounds of signing outran the skew tolerance and the witness refused the holds. It reads the real clock. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 11 — b28dede: inbox delivery is an intent the DHT emits and the swarm interprets.
Gates: core dummy 730, node 323, clippy native / wasm core / browser / ffi, rustdoc, taplo, typos, nightly fmt. |
…ne placement Two ways a relay inbox could be denied or misplaced by a peer: - Storage slots were keyed by placement alone, so a data topic any node parks at `d + 1` through `storage_store` made every hold for `d` fail with `EntryKindNotEqual` for as long as the topic was refreshed. `StorageKey = (kind, placement)` partitions storage by kind; relay slots render as `relay:<placement>`, data slots keep the bare placement so earlier values stay addressable, and a lookup reads the data slot only (the relay-absent branch of `SearchEntry` is now a property of the key, not a match). - The affine placement set was derived from the configured redundancy for every kind, so a placed operation or synced entry could name a rotated replica key for a relay carrier and be admitted there under the authority for that position's own inbox. The set is now `rotate_affine(kind.replication(redundancy))`: one placement, the DID, for a relay carrier. Tests: key law (partition, rendering, round trip), the parked topic and the hold coexisting at one position through the write funnel, and the single-placement law for placed operations and synced entries. The inbox fixture moves to `crate::tests::held_inbox_for(destination, holder)`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 12 — bbf7636: storage partitioned by kind, one placement per relay carrier A fresh read of the whole diff found two ways a peer could deny or misplace an inbox; both are fixed in this commit. 1. Any node could block a peer's inbox by parking a data topic at its position. Storage slots were keyed by placement alone, and 2. A relay carrier could be admitted at a rotated replica key. Tests: key partition/rendering/round trip; a parked topic and a hold coexisting at one position through the write funnel ( Checked and not changed: Gates: core (dummy) 733, node 323, clippy native/wasm core/browser/ffi, rustdoc |
…mped value is retired The browser test `test_storage_repair_load_does_not_starve_three_node_stabilization` seeded node1's storage with `Entry::new` values, which carry no retention bound. Under the retention law such a value is not live, so the first read retired it and the repair pass never republished it; the test has failed in CI on every commit of this branch since retention landed. The fixture now uses `crate::tests::live_entry`, as the native repair tests already do. A storage-level test pins the law that bit the fixture: a value written without a bound is reported absent, removed, and never republished. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 13 — 1d51e1d: the CI failure on every commit of this branch QACI's "Run core browser tests" step has failed on every push since retention landed, always in Fix: the fixture is stamped live. A storage-level test pins the law that bit it, Verification: the rest of the core browser suite (28 tests) passes locally under headless Chrome; the three-node test itself cannot complete on my machine on |
`FileStorage` retired a record from its index before removing the file (`retire_until_fits`, `clear`), so a removal the file system refused left an unindexed file consuming the budget until the next open, against the module's own index law; a failed retirement during `put` also left the temporary file behind. Retirement now removes first and forgets after (`WriteOrderedMap::oldest` peeks; `retire_indexed` is the one place a record leaves the index), `put` commits through `commit_record`, which restores the previous record on any failure and always cleans its temporary file. Tests: a record whose path is occupied by a directory stays indexed and counted, the write that needed its bytes fails without a temporary file, and succeeds once the file is back; `oldest` peeks without removing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 14 — 03ff70c: fresh full read of This round I read the parts the earlier rounds only skimmed: the storage backends ( Found and fixed: the file store forgot a record before its file was gone. Checked and not changed (with the reason, so you can disagree):
Gates: core (dummy) 735, node 323, clippy native/wasm core/browser/ffi, rustdoc |
…dopt self as predecessor `FileStorage::get` and `get_all` read a record under the read guard, release it, and retired the record if it did not decode. A `put` of the same key between the read and the retirement was then deleted with the garbage it replaced. `retire_observed` re-reads under the write guard and removes the file only while it still holds the observed bytes. `rectify_predecessor` now leaves the current value when the candidate is the local node, so `(pred, local]` can never be emptied by a self-reference; the shell already made this unreachable, the pure transition is now total on its own. Tests: an undecodable record survives a retirement for bytes it no longer holds and is retired by the read that observes it; a self candidate leaves the predecessor and the responsibility interval alone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 15 — dd5b738, 128e8e1: the round-14 "checked" items, fixed where they were real, and a re-read of rounds 12–15 together Two of the six items I had listed as checked were defects once looked at again, and both are fixed with tests:
The other four stay, with the reason: the drained inbox living 24 h as tombstones is the lattice law (a join's bound is the max, for tombstones as for adds; special-casing removal would make the bound depend on the operation, not the value); Re-reading rounds 12–15 as one diff found nothing further; the sync-report tests now name the inbox slot as a Gates: core (dummy) 738, node 323, clippy native/wasm core/browser/ffi, rustdoc |
…ry runs the logical stage alone Retention is refreshed by what is held, never by a removal: `Entry::tombstone` keeps the carrier's own bound instead of joining the removal's, so an inbox drained to tombstones expires when its last hold would have. The law is stated in the retention module and pinned for both carrier kinds. The inbound pipeline's connection-independent stage (application validation, handler dispatch, `on_inbound`) is now `LogicalInbound`; `InboundProcessor` composes it behind reassembly and admission, and `LocalDelivery` runs it alone, so a relay-inbox drain no longer builds a reassembler it never feeds. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 16 — a64d833: the two remaining round-15 items, fixed
The two protocol items stay as they are, because they are the protocol: a stale predecessor is kept until a closer candidate notifies or failure detection removes it (CorrectChord rectify, with Gates: core (dummy) 739, node 323, clippy native/wasm core/browser/ffi, rustdoc |
…te delivery seams With a removal no longer joining its own bound, a tombstone applied to an absent carrier produced a result without retention that the write funnel persisted and the next read retired. `operate_storage_entry` now writes only a live result and leaves the slot empty otherwise, so a stored value is always live when written (test for a data topic by value and a relay inbox by dot). Exposure: `InboxDelivery`, `Stabilizer::new`, and `SwarmCallbackSlot` are crate-private; nothing outside the crate can reach a `SwarmTransport` to build a stabilizer with, and the callback slot is `Swarm::set_callback`'s implementation. `StorageKey::inbox_of(destination)` names the inbox slot once instead of composing kind and key at every site. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 17 — b0f33c8: fresh read with the placement and exposure passes This pass asked two questions of every type the branch adds: is it where it belongs, and who can reach it; plus a doc-vs-code sweep after round 16's retention change. Found and fixed
Checked, unchanged: the Gates: core (dummy) 740, node 323, clippy native/wasm core/browser/ffi, rustdoc |
…per ring Every storage operation read a slot, computed, and wrote it back with no ordering between callers, and the store orders single puts only. The inbound actor and the stabilizer now write the same slots concurrently: a hold arriving while the recipient drains its inbox, a hand-off joining while a repair pass reads. Two interleaved read-modify-writes overwrote each other, and a held message could be lost. `PeerRing::storage_transition` (an async mutex) now covers `operate_storage_entry`, `join_storage_entry`, the ack-gated removal (`remove_storage_entry_confirmed_by`, so the comparison and the removal are one transition), and the retirement a read performs. It is held across the store's own awaits only. Test: a store double that yields at every access interleaves two holds, and a hold with a removal, on one inbox; both holds survive. Without the transition the first pair keeps one element. Also inlines the one-use `Stabilizer::deliver_inbox` wrapper. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 18 — df7f64b: concurrency pass This pass asked, for every storage slot, who writes it and whether their writes are ordered. Found and fixed: two interleaved read-modify-writes on one slot overwrote each other. Every storage operation reads the slot, computes, and writes back, and the store orders single puts only; nothing ordered the callers. Before this branch that was latent (one writer per slot in practice). The relay inbox makes it real: the inbound actor writes a hold into Also inlined the one-use Checked, unchanged: the fetched-entry cache is put-only (no read-modify-write), so it stays outside the transition; Gates: core (dummy) 741, node 323, clippy native/wasm core/browser/ffi, rustdoc |
`validate_inbox_relocation` returned a `Result` whose error was only ever turned into a boolean by the sync batch, which skips a carrier it may not take instead of failing. It is now `relocates_from_predecessor`, a predicate, and the phantom `Error::RelayInboxNotRelocatable` is gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 19 — fa65c24: vocabulary and documentation pass This pass swept the repository for identifiers the branch renamed or removed ( Found and fixed: a law that returned an error nobody could receive. Also run this round: the local browser suite (28 tests, the three-node WebRTC case excluded as before) passes on the storage transition, so One observation outside this PR's scope, not changed: Gates: core (dummy) 741, node 323, clippy native/wasm core/browser/ffi, rustdoc |
Closes #704, closes #701, closes #725, closes #726.
Signature domain separation (#704)
Every
MessageVerificationnow signs a domain-separated transcriptwhere the domain
(tag, network_id)names the message family and the overlay. Every signature isissued by a
MessageSigner(a session key acting inside one overlay) and verified against thereceiver's domain, never a value carried in the message, so:
Adoes not verify inside overlayBeven when the sessionkey is shared;
TransactionandMessagePayloadsignatures over the same transaction hash are no longerinterchangeable, and node descriptors / onion backward payloads sign under their own tags;
MessageSignerand verified under the receiver'snetwork_id; signing a body that states another overlay is refused, so a verified descriptor'sstated overlay is the overlay it was signed for (the former
matches_networkpost-filters aregone).
API shape:
DomainTag(built withdomain_tag!, which checks the length-prefix law at compile time),SigningDomain, andMessageSigner<S: Borrow<SessionSk>>are exported fromrings_core::message.MessageSigner<&SessionSk>is theCopyform signing functions take;MessageSigner<SessionSk>is what long-lived runtimes (chunked transfers, onion exits) store, sothe key/overlay pairing is never split across two fields.
Transaction::new*,MessagePayload::new*,PayloadSender, and the descriptor constructors takea
MessageSigner;MessageVerificationExt::verifyand the descriptorverify_signature/is_live_at/latest_valid_by_*take the receiver'snetwork_id.This is a wire-incompatible change (signatures from earlier builds no longer verify), hence the
bump to 0.21.0.
Storage bounds (#701)
Native storage enforces its byte budget.
SledStoragekeeps an in-memory index of filesizes in write order (rebuilt from the directory on open, stale
.tmpfiles removed). Aputbeyond the budget retires the least recently written other keys until the value fits; a value
larger than the whole budget is rejected with
Error::StorageValueExceedsCapacity; the index isupdated only after the file-system operation succeeded, so a failed write leaves index and
directory consistent; the budget is restored on open. A record the current schema cannot decode
(written by an earlier build) is retired on the read that discovers it.
MemStorage::boundedgives the local fetched-entry cache least-recently-written eviction at
LOCAL_CACHE_CAPACITYentries. Both backends share one
WriteOrderedMap.Entries carry a retention bound.
Entry::expires_at_msis stamped at the operation boundary(
EntryOperation::stamped(now, actor)) withDEFAULT_TTL_MSand joins bymax, so the productof the payload lattice and the bound lattice stays a join-semilattice. Every storage read goes
through
PeerRing::live_storage_entry/live_storage_entries, which retire an expired valueand report it absent, so hand-off, republish, lookup, and ack-delete never see an expired value.
A stored value without a bound is treated as not live. The retention model lives in
dht/entry/retention.rs.Admission bounds peer-supplied deltas.
Entry::validate_admissible_at(now)requires a livebound at most
now + MAX_TTL_MS + TS_OFFSET_TOLERANCE_MS, every version (dots, tombstones,register) at most
now + TS_OFFSET_TOLERANCE_MS, and every payload element at mostENTRY_PAYLOAD_MAX_BYTES(32 KiB). It is enforced on the peer-supplied delta at the twowrite funnels (
PeerRing::operate_storage_entryfor operations,join_storage_entryforreplicated values), pre-checked in the sync batch so a failing entry rejects the batch before
any earlier entry is written (one admission clock per batch), and applied in
local_cache_put.A
u128::MAXregister can no longer pin a key; a locally derived compaction floor is nevermistaken for a peer clock running ahead.
The per-payload size bound is element-intrinsic, so filtering by it is a lattice morphism and,
with the existing
ENTRY_DATA_MAX_LENcount cap, bounds a carrier at their product, which acompile-time law keeps inside one transport message. A byte
budget over the whole carrier was tried and rejected: "the newest payloads that fit" depends on
the sizes of payloads a replica may already have dropped, so it is not a lattice morphism and
replicas diverge (three-element counterexample in the review).
One storage transition per ring. Every read-modify-write of a slot (
operate_storage_entry,join_storage_entry, the ack-gatedremove_storage_entry_confirmed_by, and the retirement aread performs) runs under
PeerRing::storage_transition. The inbound actor and the stabilizerwrite the same slots concurrently (a hold arriving while the recipient drains its inbox), and
the store orders single puts only, so two interleaved read-modify-writes used to overwrite each
other; a test with a yielding store double shows the lost hold without the transition.
Relay inbox (#725)
EntryKind::RelayMessagegains its producer, its witness, its authority, and its policy:CustomMessagereaching the node responsible for its destination's ringposition (
destination ∈ (predecessor, self]; a node with successors but no known predecessoris uninformed, not responsible) while the destination has no connection attempt at all is
held: wrapped in a
HeldMessageunder the holder's signature (domainrings-core:relay-inbox:held-message:v1, timestamp = hold instant) and written into the inboxcarrier
destination + 1by an ordinaryExtend. Relay carriers are placed by the ringgeometry in every storage mode, so the placement law holds with virtual nodes enabled (the
production default); a relay carrier has exactly one placement, its DID, under any configured
redundancy (
kind.replicationbounds the affine set a placed operation or synced entry mayname), and its own storage namespace (
StorageKey = (kind, placement); relay slots render asrelay:<placement>, data slots keep the bare placement), so a data topic any node parks atd + 1throughstorage_storecannot shadow the inbox kept ford. A lookup reads the dataslot only.
dht/entry/inbox.rs). An element is admissible iff it decodes as aHeldMessagewhose payload is a
CustomMessagewhose transaction and relay are both addressed toinbox_destination(entry.did)(so the recipient has nothing to forward), held no later than thereceiver's clock, whose holder signature verifies inside the receiver's overlay, and whose
payload verifies as of the hold instant (
MessageVerificationExt::verify_at, which judgesproof liveness and session validity at a given instant). Admissibility is therefore monotone
in time. A reset floor is rejected, and a delta larger than the inbox is refused before the
first signature is verified.
Extend) only from thenode the owner itself routes the destination to (
PeerRing::inbox_hold_authority, the localanswer of
find_successor), every element held by that node, and only while the held message'ssender proof is still live by the owner's own clock: the holder signs the hold instant, so this
freshness bound is what makes it honest and bounds replay by the sender's proof lifetime.
Authority is judged before any signature is verified. A removal (
Tombstone, per add dot) onlyfrom the recipient; a relocation only as an
OwnershipHandoffwhose authenticated sender(the transaction signer, never the relay path) is the receiver's predecessor, otherwise the
carrier is skipped without an ack; no other operation. A relay carrier is never fetched,
cached, replicated, or returned to a lookup.
RELAY_INBOX_MAX_LEN(64) messages per inbox and as many tombstones;retention 24 h after the last hold, at most 7 d; data topics keep 10 min / 100 min.
dht::InboxDelivery::deliver_inbox, andSwarm::stabilizerinjects the swarm's interpreter(
swarm::inbox::SwarmInboxDelivery), so the DHT never reaches into the delivery pipeline. Theinterpreter reads the local inbox, retires the elements that fail the witness, and then,
element by element, delivers through the logical stage of the inbound pipeline
(
swarm::callback::LocalDeliveryoverLogicalInbound, the connection-independent stage theper-connection
InboundProcessoralso runs: application validation, dispatch,on_inbound,each under the inbound deadline) and tombstones the delivered element by its dot, so a pass cut
short by its step deadline resumes where it stopped. A removal never extends a carrier's
retention, so a drained inbox expires when its last hold would have. The callback (
SwarmCallbackSlot) is resolved at deliverytime, so
Swarm::stabilizeris infallible andset_callbackafterlistenis honoured.Delivery is at least once.
dht/entry/test_inbox.rs), dot-based retirement against astale copy, caps, and two end-to-end cases (
tests/default/test_inbox.rs): the peer returnsthrough its successor or straight through its predecessor; in both the owner's pass hands the
carrier over, the recipient drains it, and the owner's copy is removed by the ack.
Ownership hand-off follows the successor head (#726)
The hand-off used to run only inside
HandleMsg<NotifyPredecessorReport>, one of three inputsthat can move the successor head (the others:
correct_stabilize's topology query and a directlyconnected peer admitted as head). Placement is a function of the ring state, so:
topology::stepemitsTopologyAction::SuccessorHeadChanged(h)iff thehead moved to
h, once per transition, after the event's own actions;topology::successor_headis the single definition of the head, shared with
find_successor.CoreEffect::RequestStorageRepair, which onlyrecords the repair intent (
request_storage_repair). A send at admission time was tried andrejected: it can precede the peer's own admission of this node and is dropped as coming from an
unadmitted connection.
deliver_inbox, thenrepair_storage, whichrestores placement in one bounded pass (offer the live entries beyond
(self, head]to thehead as an
OwnershipHandoffwith ack-gated cleanup, republish to missing affine ownersadditively) through the existing repair window (tracked sends,
STORAGE_REPAIR_FRESH_CONNECTION_GRACE_MS, one delivery per step). The maintenance loop runs thephase when requested and periodically, so the inbox drain, previously reachable only through the
one-shot
stabilize(), now runs in production as well. A head change is the localPeerRingAction::StorageRepairDue.SendStorageSynceffect, whose sole producer it was, isgone.
Tests: head law in
topology/tests.rs;test_stabilization/test_storage_handoff.rs(directconnection moves the head, admission sets the request, the pass hands off after the grace, the
ack removes the local copy); a second inbox end-to-end test where the returning peer connects
straight to its predecessor.
Tests
verify.rs: transcript layout, overlay binding, tag binding, ext-verify uses the type's tag,owned/borrowed authorities sign identically, over-long tag label rejected.
test_payload.rs: transaction/payload signatures not interchangeable; foreign overlay rejected.test_entry.rs: stamping law,maxjoin of bounds across every operation, liveness, admissionbounds for lifetime, versions, and payload size, normalization/affine preserve the bound, legacy
value without bound is not live.
dht/storage/tests: live reads retire expired values, write funnel rejects pinned register /expired / over-long bound, join keeps the later bound, cache shares admission and retention.
memory.rs/test_sled.rs/write_ordered.rs: eviction order, rewrite does not evict,oversize rejected without change, reopen restores the budget in modification order.
🤖 Generated with Claude Code