Let a newer card document refresh a resolved query-backed field

[habdelra]

What this changes

A query-backed field's search resource is created once per (instance, field). getSearch captures its seed argument at creation and the resource latches after applying it, so a refetched owner — whose document carries the field resolved by the server at read time — updated fieldState and reached the resource with nothing.

A resolved result set now carries an identity and an index generation, and a document arriving for the owner offers that set to the running resource through a new SearchResource.reseed.

Why this shape

Two axes, two questions. The identity says whether two result sets differ at all; the generation says which of them is newer. Keeping them separate is deliberate: a realm generation moves on every write anywhere in the realm, so folding it into the identity would make every set look different from every other and re-apply answers that had not changed.

Ordering, not arrival. A search stamps the generation the event that drove it committed — the index it read is at or after that — and a document carries the generation its row was serialized at, off meta.generation, which exists to tell fresh index data from stale. Without this the last arrival wins, which loses a completed search to a document resolved before it: the owner is reindexed and its reload issued, a second write lands and the field's own search answers it, then the first document arrives carrying membership from before that write. The floor only ever rises, so a set whose generation cannot be established keeps the last one established — that refuses a fresh document at worst, where the other direction lets a stale one through. Where either generation is absent, the identity decides alone.

Two residual limits: the event's generation is a floor for the search's true generation rather than its exact value, so a narrow window remains where a document in between is accepted; and a search driven by a query change rather than an event has no generation to stamp, so it leaves the floor where it was.

Two gates decide whether an offer is taken. The outer one is the document: captureQueryFieldSeedData marks a handover pending and the first read that acts on it spends the mark, so the offer belongs to a document arriving rather than to a field being read — without it, a plain read would hand the last document's answer back over a fresher set. The inner one is the identity, read off the resource rather than remembered by the caller, because a remembered value would go on claiming a set a search had replaced and would then turn away the document that corrects it. It deliberately survives a failed search: a failure computed nothing, so what a document last asserted is still the last thing anyone asserted, and clearing it would let the next read reinstate those rows and with them clear the error the failure is reporting.

The identity covers everything the seed asserts, not just its rows. A page-clamped field gains a match it cannot surface and reports the same row against a higher total; a realm that stops answering leaves the rows it did contribute while turning the total into a floor. Identifying by rows alone calls both of those the answer already in hand, and leaves the field reporting a shortfall of none. So it is the resolved query, its ids, the reported total, and the realms that failed — and only an indexer-resolved umbrella (links.search, written nowhere else) gets one at all, so a raw source document can never displace a real answer.

An offer a realm failure cut short sends the field back to a live query. Its rows are a floor, not an answer. The initial seed reaches a query for this case by leaving the query signature unset and falling through to modify's search, but nothing brings a running resource back through that fall-through — its query and realms are unchanged, so the next modify skips. reseed runs that query directly, and only when live, matching the initial seed's refusal to query inside a prerender.

An in-flight application is bound to the identity it was handed over as. applySeed runs unbounded, so two documents arriving close together resolve their rows concurrently and whichever finishes last wins; since the resource records the newest identity synchronously, the loser could leave it holding an older answer under a newer identity. Each run re-checks before mutating and abandons otherwise. Not by making the task restartable: updateInstances splices _instances synchronously and reconciles store references only after awaiting each row's hydration, so a cancellation between the two leaves rows nothing holds a reference for — and the next run diffs against that already-spliced array, so those rows never get one.

One function builds the seed for both paths. Resource creation and the handover share queryFieldSeed, so both describe the same set the same way. The count ladder in it (an indexer-reported total, else a floor labelled incomplete when a realm failed, else explicitly unknowable) is what the field's shortfall signal reads.

Where this earns its keep. For a same-realm query field the owner's index event refreshes the field by search anyway, and the handover is a free head start — the document already paid for the resolution. It is the only refresh when the resource never subscribed to the realm the write landed in (a query whose realms set excludes the owner's realm), or when the subscription missed the event.

The seed shape is an exported StoreSearchSeed<T>, generic in the row type and threaded through GetSearchResourceFuncOpts<T>, so a FileDef search seeds with file-meta rows instead of being pinned to CardDef.

Scope

displayedInstances still reconciles a handed-over set against the filter and against store residency, exactly as it does a searched one. That reconciliation is also why the tests below assert on the match count rather than on membership — see the test plan.

Test plan

packages/host/tests/integration/components/query-field-membership-status-test.gts:

• a document fetched after the field resolved hands it the fresher result set — starts from the document the realm actually serves and splices in a third member, spelled the way the realm spells the others. The load-bearing assertion is totalMatchCount moving 2 → 3: membership cannot carry this test, because deserializing the document deposits the spliced member in the store (setCardNonTracked → makeTracked) and the client-side merge then adds a resident matching card to the displayed set whether or not anything was superseded. The count comes off the resource's own meta, which only an applied result set moves.
• a document reporting a higher match count refreshes a page-clamped field — firstMatch holds its one row while the count moves to the document's 3 and isPartial stays true, then back to 2 on a document restoring the earlier count. Bypasses the merge entirely (total !== _instances.length makes it ineligible). The second leg fails if the identity is treated as a one-way latch.
• a document resolved before the result set the field holds is declined — two documents with derived generations; the older one does not walk the count back.
• a document reporting an unreachable realm sends the field back to a live query — counts _federated-search requests across the handover: zero for a document that resolved cleanly, one for a document naming a realm that did not answer.
• a document that did not resolve the field cannot displace a result set — guards the gate rather than the handover: it holds trivially where nothing is handed over, and its job is to catch a gate that stops requiring an indexer-resolved umbrella.

Not covered: the path where a completed search clears the applied identity and a later document restores the set the search moved off. Reaching it needs realm-event delivery to move the resource mid-test, and every framing collided with the reconciliation above.

Isolating the handover end-to-end needs a shape the current specs don't have: owner in realm A, targets in realm B, the field's query scoped to B so the resource subscribes to B only; resolve the field, block the client's search endpoints, add a matching target in B, then write the owner in A. No event reaches B so no search fires, while the store still refetches the owner's document from A with the field resolved. Left as a follow-up.

Not run in this environment: the host suite needs the realm servers, and this session's network policy blocks the container registry the dev stack's Postgres/Synapse come from. Verified here: ember-tsc --noEmit clean for @cardstack/host (which compiles the changed packages/base files), packages/base lint clean, prettier and eslint clean on the changed files. CI runs the suite.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FH1ee2ghdpunvh13cdtq9p

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-04T13:33:41.475934Z	926e0ea	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 926e0ea461

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

🟡 Changes recommended

The new reseed flow has a concurrency hazard (out-of-order seed/search application) and a typing inconsistency in GetSearchResourceFuncOpts that undermines the new generic StoreSearchSeed<T>.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates query-backed field behavior so that when a newer owner document arrives with an already-resolved relationship result set, the existing per-(instance,field) search resource can be “reseeded” to reflect that fresher authoritative result without waiting for live realm events.

Changes:

• Add seed “identity” tracking for query-backed fields and trigger reseeding when the document-provided identity differs from the resource’s applied identity.
• Introduce SearchResource.reseed() to apply a new seed into a running search resource while preserving the “seed suppresses search” behavior.
• Refactor the seed shape into an exported StoreSearchSeed<T> type and add host integration tests covering the supersession and non-authoritative document cases.

File summaries

File	Description
packages/host/tests/integration/components/query-field-membership-status-test.gts	Adds integration coverage proving a newer resolved document can supersede a running query-field search result set, and that non-authoritative docs cannot.
packages/host/app/resources/search.ts	Adds reseed() to apply a new seed to an already-running SearchResource and update prior-query/realm suppression state.
packages/base/query-field-support.ts	Tracks seed identity and applies reseeding on reuse when a newer authoritative document result set arrives.
packages/base/card-api.gts	Exposes StoreSearchSeed<T> and adds an optional reseed() hook on StoreSearchResource.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[habdelra]

should the realm generation be part of the identity? as shifting card data may change membership in the query. altho that might destroy caching altogether by making many more misses

[habdelra]

[Claude Code 🤖] I don't think it should — and your second thought is half the reason, though I'd put the decisive half differently.

The identity isn't answering "is this answer still true of the realm". It answers only "does this offered set differ from the one the resource is already holding" — a dedup key for the handover, not a freshness check. So the test for what belongs in it is: could two seeds share an identity and still leave the resource in different states? Everything a seed can put into the resource is already covered:

what the seed sets on the resource	covered by
the rows	the resolved ids, in document order — so a reordering misses too
the realms searched	parseRealmsParam derives them from the seed URL, and the URL is in the identity
page.total / totalUnknown / incomplete	the reported total, and the realms that failed
the interpolated query	the seed URL

Card data shifting under the query is the case that comes out the same either way. If the shift changes membership, the ids or the total move and the identity misses already. If it doesn't — a member's fields changed but the set didn't — re-applying produces identical resource state: applySeed splices the same rows in the same order and sets the same meta. The members are store instances shared by identity, so their new content reaches the field through the store whether or not a seed was applied. Generation would turn a no-op into a no-op-after-work.

And the miss volume isn't hypothetical: the realm generation bumps on any write anywhere in the realm, so every capture after any write would miss, and each miss re-runs applySeed per query field on the owner — splice, per-row store peek, reference reconciliation, flush — to arrive where it already was. On a document that names ids without inlining them (the prerender skip shape) that's also a store.get per id.

Where generation would belong is the question this PR deliberately doesn't answer: whether the resource's set is behind the realm. The identity can't express that at all — a set that came from a search carries no seed identity — and answering it would mean comparing the generation the document was read at against the generation the resource's set was derived at, which the resource doesn't record. Today that staleness is the realm-event subscription's job, with this handover covering the gap where the subscription missed the write.

---

Generated by Claude Code

[github-actions]

Preview deployments

• Host staging preview

• Host production preview

Host Test Results

    1 files      1 suites   2h 8m 19s ⏱️
4 619 tests 4 605 ✅ 14 💤 0 ❌
4 634 runs  4 620 ✅ 14 💤 0 ❌

Results for commit 3ea963a.

Realm Server Test Results

    1 files    190 suites   1h 7m 16s ⏱️
2 525 tests 2 525 ✅ 0 💤 0 ❌
4 707 runs  4 707 ✅ 0 💤 0 ❌

Results for commit 3ea963a.

[habdelra]

[Claude Code 🤖] An independent review pass turned up two things worth flagging here, one of which corrects something I asserted earlier in this PR.

Correction: the first test was vacuous, and my reasoning for it was wrong. I claimed the spliced third member "never enters the store, so reconciliation cannot supply it either". That is false. _createFromSerialized deposits every resource it deserializes via store.setCardNonTracked (card-api.gts:5020) and promotes it with store.makeTracked (card-api.gts:5354), so deserializing the spliced included resource makes Person/three store-resident whether or not anything was handed over. The resource is client-filter-eligible at that point (total === _instances.length === 2), so displayedInstances merges the resident matching card into the displayed set on its own — membership was 3 either way. Worse, the test's premise inverted: the spliced member had to claim a matching name to survive reconciliation, and that is exactly the condition that made the test pass without the change.

Fixed in 3ea963a8: the load-bearing assertion is now totalMatchCount, which comes off the resource's own meta and which the client-side merge cannot move. Membership is kept as corroboration and labelled as such. The "cannot displace" test is likewise relabelled — it guards the gate, not the handover, and holds trivially where nothing is handed over.

The handover had no ordering, and that was a regression. A completed search reported no seeded identity, so any identity-bearing document beat it regardless of which was resolved later. Two index events in quick succession reach it: the owner is reindexed and its reload issued, a second write lands and the field's own search answers it, then the first document arrives carrying membership from before that write and overwrites the answer. main gets that case right, so this was strictly worse than doing nothing.

@habdelra — this is where your generation question lands, and you were pointing at something real. Not in the identity: folding it in there would make every set look different from every other and re-apply unchanged answers, for the reason in my earlier reply. But the ordering question needs exactly that signal, and both sides already carry one — index events carry the generation the pass committed, and a card document carries meta.generation, stamped (per its own comment) "so a consumer can tell fresh index data from stale". So a search now stamps the generation of the event that drove it, a document carries the generation its row was serialized at, and the newer of the two wins instead of the last to arrive. Two axes doing two jobs: identity says whether the sets differ, generation says which is newer.

The floor only ever rises, so a set whose generation cannot be established keeps the last one established — that refuses a fresh document at worst, where the other direction lets a stale one through. Where either generation is absent the identity decides alone, as before.

Two residual limits, both stated in the description: the event's generation is a floor for the search's true generation rather than its exact value, so a narrow window remains; and a search driven by a query change rather than an event has no generation to stamp.

---

Generated by Claude Code

[habdelra]

[Claude Code 🤖] CI status on 3ea963a8: Realm Server Tests (4, 6) failed, and I can't yet say whether it's this PR's.

What the job metadata says:

• The realm server test suite step ran 15:16:11 → 15:17:05 — 54 seconds. A healthy shard runs for minutes; shards 1 and 2 were still executing well past that mark on the same commit.
• Upload junit report completed in zero seconds and produced no artifact, so no junit file was written.

Together those say the suite aborted before executing test bodies rather than failing an assertion. That is consistent with either a module-load failure or a transient environment problem on that runner, and not with a behavioural regression — a regression in the shared packages/base code this PR touches would have aborted shards 1 and 2 the same way, and it didn't.

What blocks me from naming the cause: the failing output is not reachable from this environment. The job-log API returns tails only, and this job's tail is the echoed realm-server log plus teardown; the junit artifact does not exist; and the uploaded log artifacts are served from productionresultssa*.blob.core.windows.net, which this session's egress policy denies (connect_rejected). So I have the job's shape but not its error.

Next step, deliberately not taken yet: re-run that shard once. I'm holding the re-run until the remaining four Realm Server shards report on this commit, because that is the cheaper discriminator — if another shard aborts the same way it is this PR's and broad, and if they all pass then shard 4 is isolated and the re-run is the right confirmation. Everything else on this commit is green so far: Lint, Host Tests shards 1 and 2, Live Tests, the realm performance benchmark, the test-web-assets build, and both preview deploys.

---

Generated by Claude Code

[habdelra]

[Claude Code 🤖] The discriminator came back and it points away from this PR, so I've spent the one re-run on Realm Server Tests (4, 6).

• Realm Server shards 1, 2, 3, 5 and 6 all passed on 3ea963a8. Only shard 4 aborted, at the 54-second mark.
• The merged Realm Server Test Results check is green: 2104 tests, 0 failures. Shard 4 contributed no failing test — it contributed no test at all, which is why that total sits below a full six-shard run.
• Host Tests: 14 of 16 shards green, 3 and 13 still running. Matrix Client Tests, Boxel CLI Tests, Software Factory, Live Tests and both preview deploys are green.

A regression in the shared packages/base code this PR touches could not abort one shard while five others run its neighbours to completion, so the re-run is the right confirmation rather than a way of wishing the failure away. If it fails again at the same point, the failure is real and reproducible and I'll treat it as this PR's regardless of which files it names — I'll say so and dig in from the shard's module list, since the failing output itself is unreachable from my environment (job-log API is tail-only, no junit was written, and the log artifacts are on an egress-denied host).

---

Generated by Claude Code