Conversation
Records live in the component next to the workflow that owns them, so a publish rides the same commit. Retention, truncation and the consumer floor are the component's own state, and the caps that bound resource use come from namespace-scoped dynamic config.
This was referenced Sep 25, 2026
The item budget compared an absolute head offset against a count, so a stream whose floor had moved, or one created above zero, was over budget the moment it existed. The close reason now carries its encoding, and the record kind is settled on a copy rather than through the caller's protos.
The stored record mirrors the public one field for field, and the public one no longer reserves -1 for an unnumbered record.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the CHASM stream component.
A stream is a CHASM component holding records in batches, with a frontier, producer cursors and consumer pins. It lives next to the workflow that owns it, which is what lets a publish ride the workflow task's own commit. Nothing wires it up yet: there is no service, no command and no delivery here, only the component and its unit tests.
What changed?
chasm/lib/stream/stream.go: the component. Appends, batching, the frontier, producer dedup by(producer, attempt, sequence), closing, truncation with a floor no active consumer can be truncated past, the cap that reclaims the oldest batches, and the read window a consumer pages through. A filtered read advances only past the messages it examined, so a page whose batches did not reach the window end cannot skip unexamined offsets.chasm/lib/stream/cursor.go: the consumer cursor, held by the reader rather than the stream, so folding in a delivered range commits with the event that records it.chasm/lib/stream/config.go: the caps as namespace-scoped dynamic config, with the old constants as defaults, plus a byte and item budget for a stream a workflow owns that sits well under the mutable state limit.chasm/lib/stream/messages.go: the byte and count caps on what one read returns.chasm/lib/stream/proto/v1/{message,stream_state,tasks}.protoand their generated Go: the record, the batch, the component's state and the two side-effect tasks.Why?
The payload lives in the component, not in History and not in a log table. That is the property everything above this layer depends on, so it is worth reading before the wiring.
How did you test it?
All green.
Series
Server 2 of 6. Replaces #2.
Previous: #3
moe/AI-198-srv-1-api-pin. Next: #5moe/AI-198-srv-3-stream-service.Review round
checkBudgetcompared an absolute head offset against an item count, so a stream whose floor had moved, or one created above zero, read as over budget the moment it existed. It measures held records now, which is head minus base. The close reason onTerminategoes throughpayload.EncodeString, so an SDK data converter can read it, and the record kind is settled on the copy the store serializes rather than through the caller's protos.Known limits, argued rather than fixed:
StreamLifecycle.max_itemsstops being a rolling window. The consumer's floor sits where it subscribed, because replay re-reads every range its History recorded back to that point, so the cap behaves as a lifetime quota and appends past it are refused instead of reclaiming behind the consumer. The two promises cannot both hold and the one kept is that a workflow can replay a decision it already made. Deregistering the consumer releases it.ForgetConsumerreleases the floor when a run closes. A closed run takes no further task, but it can still be queried and it can be the base of a reset, and neither is served once truncation has taken the bytes. Both are refused with the offset the stream now starts at.Limitstreats zero as "use the default", so none can be configured to zero. Turning streams off for a namespace is whatstream.enabledis for.Review round: api surface
The public
StreamRecord.sequenceno longer reserves -1 for a record whose producer does not number its records, because an unsetint64reads as zero. The stored record mirrors the public one field for field, so its comment and the tests that leaned on the sentinel follow.Brought onto current main (
d8f9c6d86b2c) by merge, which came through without conflicts and needed no code change on this layer.