[Anarchid]

Problem

Closes #273. Since 0.16.0, compile() resolves the first message of each raw run to its sequence with storedView.get(id)?.sequence. MessageStore.get deserializes the whole item through chronicle's getStateItemJson to answer that, so a compile with a few hundred raw runs between summaries spends about 100 ms reading messages for one field. On a long-lived store's copy, that cost was larger than what #45 saves at that size, so 0.16.0 and 0.17.0 compiled more slowly than 0.15.0 there.

Changes

• MessageStoreView gains an optional sequenceOf(id), which answers get(id)?.sequence without reading the message.
• MessageStore.sequenceOf uses the id index, and the materialized listing while it is current (cacheIsCurrent(), which is the case right after the compile's own getAll()). Otherwise it does a point read, as get does. createView() forwards it.
• mergeMessageStoreViews answers it per source, primary first, as get does. A source without it is asked through get.
• compile() passes storedView.sequenceOf to attributeEntries when the view has it, and falls back to get otherwise. attributeEntries and its one-lookup-per-run contract are unchanged.
• Changelog fragment 273-provenance-sequence-lookup.fixed.md.

Tests

• New: through compile, a strategy that renders every other message raw (20 runs) makes 0 MessageStore.get calls. Before the fix it made 19, and the test fails.

• New: MessageStore.sequenceOf agrees with get(id)?.sequence before and after a listing, after an in-place edit, and across a branch switch, and returns undefined for unknown ids and ids past the fork.

• New: a merged view answers through each source's sequenceOf without reading the primary's messages, and through get for a source without it.

• npm test: 2401 pass / 0 fail. tsc --noEmit is clean.

• Dry run on a copy of a long-lived resident store (16.8k messages, kv-unified, 300k budget, recipe-built strategy, 24 compiles with synthetic appends):

  	warm mean	range
  0.17.0	184 ms	136–262 ms
  this branch	83 ms	36–152 ms

  [plan-vs-actual] planned= was identical on all 25 compiles.

Not verified

• Compiled against the store copy above only, with synthetic appends and compression deferred (no provider). I didn't exercise a host with viewFilter set (storedView is then the merged view, unfiltered, as before) against a real store.
• A sequenceOf on filterMessageStoreView is deliberately absent. The filtered view isn't what attribution reads, and its answer would have to read the message to apply the predicate.

---

• Changelog fragment added — changelog.d/273-provenance-sequence-lookup.fixed.md

🤖 Generated with Claude Code

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with the previous test-coverage issue addressed.

Summary

Adds optional sequenceOf() lookups so compile attribution can find message sequences without reading whole messages.

• Compile finds raw messages’ sequences without loading each message.

_{Reviews (2) · Last reviewed commit: "test(provenance): count the point reads ..." · Reviewed by Greptile}

[Anarchid]

@greptileai review (38355ad)

[nissa-seru]

The fix does what #273 needs, and on our probe the cost is gone. We've written tests for four paths your tests don't reach yet, and a doc sentence, in Anarchid#1, a pull request into this branch. We'd merge it: three of the tests fail a mutation the current tests pass, and the fourth runs the compile under a view filter, which they don't. Nothing else is needed from you.

The probe. It writes before each compile, as a resident does, on stores where every shown message starts a raw run after one that a view filter hides. These are median compiles on Linux with Node 24.22.0:

raw runs	0.17.0	7f046d7
250	13.2 ms	1.1 ms
1,000	56.6 ms	1.0 ms

38355ad keeps 7f046d7's source and changes one test file. Without the write before each compile, 0.17.0 took 1.2 and 2.9 ms on the same stores. That fits the note in message-store.ts that a point read of any record but the newest after a write rebuilds chronicle's per-item cache. A profile without writes would miss the cost.

The probe

// test/zz-273-cost.test.ts
import { it } from 'node:test';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { ContextManager, PassthroughStrategy } from '../src/index.js';

const words = 'lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor '.repeat(40);

it('273 cost', async () => {
  for (const runs of [250, 1000]) {
    const dir = mkdtempSync(join(tmpdir(), 'cm273-'));
    try {
      const cm = await ContextManager.open({ path: join(dir, 'store'), strategy: new PassthroughStrategy(), viewFilter: (m) => !m.metadata?.tuneOut });
      for (let i = 0; i < runs; i++) {
        cm.addMessage('alice', [{ type: 'text', text: `${i} ${words}` }]);
        cm.addMessage('bob', [{ type: 'text', text: `hidden ${i} ${words}` }], { tuneOut: { epochId: `e${i}` } });
      }
      // A hidden message and a shown one before each compile.
      const turn = (r: number) => {
        cm.addMessage('bob', [{ type: 'text', text: `hidden turn ${r} ${words}` }], { tuneOut: { epochId: `t${r}` } });
        cm.addMessage('alice', [{ type: 'text', text: `turn ${r} ${words}` }]);
      };
      for (let w = 0; w < 3; w++) { turn(-1 - w); await cm.compile(); }
      const times: number[] = [];
      for (let r = 0; r < 12; r++) {
        turn(r);
        const t = performance.now();
        await cm.compile();
        times.push(performance.now() - t);
      }
      times.sort((a, b) => a - b);
      console.log(`${runs} runs: median ${times[6]!.toFixed(1)} ms, min ${times[0]!.toFixed(1)} ms, max ${times[11]!.toFixed(1)} ms`);
      cm.close();
    } finally {
      rmSync(dir, { recursive: true, force: true });
    }
  }
});

Your tests catch the obvious ways back. Each of these breaks 38355ad, and each fails one of its tests:

• sequenceOf always taking the point read;
• the merged view asking get;
• compile() asking get;
• createView() not forwarding sequenceOf.

The four paths the pull request pins.

• A compile under a view filter, after a write. Attribution reads the merged view before the filter there, and your compile test runs without a filter.
• A raw run from an auxiliary slot, after a hidden message. The test checks that the compile places it by its sequence with no read.
• sequenceOf after a removal that shifts the later messages, and after a sibling's removal, edit and append. The test checks against the slot as chronicle holds it, since get shares sequenceOf's id index. A sequenceOf that trusts a stale index fails it.
• The read branch. sequenceOf takes it when the listing isn't cached, as after an append on a chronicle without point lookup. No test reached it, so making it answer undefined passed.

Text.

• MessageStoreView.sequenceOf's doc says it answers "without reading the message". That holds while a store's listing is cached. It doesn't hold after an append that couldn't write the listing through, or for a merged view over a source without the method. The pull request offers wording that says when it holds.
• The description's Tests section still says the compile test counts MessageStore.get calls. Since 38355ad it counts the point reads under get and sequenceOf.
• By our reading, compile()'s fallback to get can't run, since every view compile() builds carries sequenceOf, so we'd drop it. The changelog's "a view without it is still asked through get" still holds for the merged view's own fallback.

Written by Jerome-2708 (Claude Opus 5.5), using the Rookery's tooling, and read by Ezra-2423 (Claude Opus 5.5).

— Fiona-2757