Added server-side streams for workflows and external clients. - #2

Open

moedash wants to merge 81 commits into

mainfrom

moe/AI-198-server-side-streams

Commits on Aug 28, 2026

Added Native Streams high-level and detailed design docs.
moedash
committed
253c488
Compared Native Streams against the two existing prototypes.
moedash
committed
37a5a22
Proved the history-node store works as a general append-only log.
moedash
committed
6c6d823
Dropped underscores from the stream-log test names for the linter.
moedash
committed
20f15ed
Folded Option 7 into the comparison and deferred to the Notion doc.
moedash
committed
ee381c6
Took the partition fix and run-transition handling from the Codex design.
moedash
committed
60fc894
Measured what Workflow Streams actually costs today.
moedash
committed
d61adc4
Added the stream CHASM component and its bucketed log.
moedash
committed
eb88450
Defined the StreamService API surface.
moedash
committed
b67261b
Wired StreamService through history and frontend.
moedash
committed
95fbc42
Added end-to-end stream tests and fixed three bugs they found.
moedash
committed
ec0bdf7
Added long-poll tailing and a shard-local tail cache.
moedash
committed
4d26293
Added cap-driven truncation, bucket reclamation, and retention.
moedash
committed
3581563
Made streams listable.
moedash
committed
f29c14f
Attributed stream persistence calls to their namespace.
moedash
committed
c94a2d1
Let callers supply a stream's run id, and measure the difference.
moedash
committed
e49edb3
Measured native streams against the pattern they replace.
moedash
committed
6f5a6e6
Specified the api-go changes Paths A and C need, and why they are blocked.
moedash
committed
aaaa1ec
Added a runnable native-streams demo.
moedash
committed
3bb8f73
Pinned the server to an api fork carrying the stream shapes.
moedash
committed
036606f
Rebuilt the API fork through the project's own pipeline.
moedash
committed
b124727
Added Path A: a workflow publishing to a stream it owns.
moedash
committed
e301a22
Resolved how Path C replay obtains stream bytes.
moedash
committed
7c074ef
Corrected the completed-event field name in sequencing.
moedash
committed
9279de8
Narrowed the in-workflow caveat to measurement coverage.
moedash
committed
4046164
Added a guard for the started-response wire-compatible pairs.
moedash
committed
4b792b0
Threaded stream slices through the workflow task response.
moedash
committed
2463dd7
Added the consumer cursor a workflow keeps for a stream.
moedash
committed
2c7c102
Recorded where the consumer cursor lives and why.
moedash
committed
c99e812
Wired stream delivery and cursor recording into workflow tasks.
moedash
committed
eb999b2
Delivered stream slices to consuming workflow tasks.
moedash
committed
543bf8b
Pinned a stream's floor to its in-workflow consumers.
moedash
committed
5e6e158
Woke a subscribed workflow when its stream runs ahead.
moedash
committed
4c86fa8
Re-supplied consumed ranges to a replaying worker.
moedash
committed
232c886
Let a workflow consume a stream in another execution.
moedash
committed
58fa960
Read a stream's log from the shard that holds it.
moedash
committed
2998caf
Carried stream subscriptions across continue-as-new.
moedash
committed
b8ff6db
Let a workflow subscribe to a stream with a command.
moedash
committed
65f187f
Gave the subscribe command a history event.
moedash
committed
935fbf3
Aligned the stream imports with the repo's alias rules.
moedash
committed
ec69563
Recorded a stream publish as a history event.
moedash
committed
5a4fe2d
Measured what a history event per publish costs.
moedash
committed
a497d26
Bounded a poll's read and keyed the tail cache by log.
moedash
committed
f318f7a
Bounded the stream's unbounded per-host and per-stream tables.
moedash
committed
e36d24f
Separated a stream append's txn id by workflow task attempt.
moedash
committed
606d6a9
Delivered stream ranges on the inline workflow task path.
moedash
committed
6c58e6b
Stopped re-supplying a consumed range to a sticky task.
moedash
committed
c5469f5
Gave a repeated subscribe its history event.
moedash
committed
c57ed7e
Judged the payload-leak check against a leak, not against zero.
moedash
committed
628d509

Commits on Aug 30, 2026

Added a harness comparing bucket 2 against the Redis prototype.
moedash
committed
15941b5

Commits on Sep 1, 2026

Commits on Sep 2, 2026

Numbered every stream write from one sequence.
moedash
committed
0e74266
Kept the scavenger off stream logs, and the subscribe event in order.
moedash
committed
8363ddf
Made a lost consumer notification retry instead of vanishing.
moedash
committed
fa6b439
Routed the two cross-execution steps that could be routed.
moedash
committed
56b4398
Measured Cassandra partitioning before committing to a substrate.
moedash
committed
b56177e
Moved the stream log onto a table of its own.
moedash
committed
c272bbb
Removed the transaction id the new substrate does not need.
moedash
committed
61f87af
Stopped a consumer holding the stream's floor.
moedash
committed
c5c08a3
Designed stream log replication, which the substrate change made cheap.
moedash
committed
343d19d
Worked out which rows a replicated stream has to send.
moedash
committed
b3c6d63
Turned the replication note into a CHASM framework request.
moedash
committed
6418bbf
Fixed the drain order and stopped reporting an absence as a zero.
moedash
committed
5596837
Made the benchmark latency sink count each token once.
moedash
committed
feab9aa
Recorded the answer on where a stream's bytes belong.
moedash
committed
b9abd71
Re-ran the Option 5 against Option 7 head to head.
moedash
committed
b86fbab
Moved the stream payload onto the component as a chasm.Map.
moedash
committed
fc4d68b
Removed the tail cache, which no longer has a read to save.
moedash
committed
fde8a7b
Read Path C deliveries from the component holding them.
moedash
committed
8ed6d78
Re-measured both benchmarks against the component-held payload.
moedash
committed
e056229

Commits on Sep 3, 2026

Commits on Sep 4, 2026

Fixed the lint issues.
moedash
committed
d4ef5bd