Conversation
A publish appends to a stream co-located with the workflow, so it commits with the workflow task and a task that fails takes the publish with it. Both commands write an offsets-only event, because a command that produces none puts the command-to-event matching every SDK replays against out of step.
This was referenced Sep 25, 2026
…workflow-commands # Conflicts: # chasm/lib/workflow/stream_commands.go
A subscribe no longer reads any NotFound as proof that the name is the workflow's own, since a routing failure would then bind it to different, empty data. Subscriptions per execution are bounded, publishes count against the shared byte budget, and both commands answer the namespace switch.
…workflow-commands
Each call had a deadline of its own and nothing bounded how many a task could carry, so the execution's lock hold grew with the subscription count.
…workflow-commands
The append command only ever names a stream this workflow owns, and the subscribe command takes a name or an id. The appended event now bounds its batch the way every other range does.
…workflow-commands
…workflow-commands
…workflow-commands
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR gives a workflow the commands to publish to and subscribe to a stream.
AppendStreamRecordsappends to a stream the workflow owns, which is a co-located subcomponent, so the batch and the frontier land in the workflow task's own commit.SubscribeStreamrecords a cursor. Both write an offsets-only event.What changed?
chasm/lib/workflow/stream_commands.go: the two command handlers, their event definitions and the library that registers them. A publish clears the producer id, which is how a reader tells the owning workflow's records from an outside producer's.respondworkflowtaskcompleted/stream_appends.goresolves the staged subscriptions between the commands and the commit, through the routed stream client. The pin goes on the stream before the cursor goes on the workflow, so an interruption leaves a pin nothing reads rather than a cursor no truncation floor protects.service/history/history_engine.goinstalls the routed stream client on the workflow task paths;service/history/configs/config.gocarries the namespace limits into them.tests/stream_workflow_test.goandtests/stream_rejection_test.goprove publishing end to end, that a refused publish fails the task with a cause and appends nothing, and that a publish on a failed task leaves no event, no offset and no message behind.Why?
The publish path is the one that makes the design worth having: it costs one event per batch and it is atomic with the workflow task. That is easier to judge on its own than mixed with the delivery machinery.
How did you test it?
All green.
tests/stream_replay_test.go,tests/stream_consume_test.goandtests/stream_consumer_lifecycle_test.goarrive here holding only the helpers the tests above share. They gain their own tests in the next layer, which is where a subscription starts being delivered to.Series
Server 4 of 6. Replaces #2.
Previous: #5
moe/AI-198-srv-3-stream-service. Next: #7moe/AI-198-srv-5-delivery-replay.Review round
A subscribe no longer reads any
NotFoundfrom the routedRegisterStreamConsumeras proof that the id names one of the workflow's own streams. The stream's own shard answersstream_absentfor that one case, and a registry miss or a shard that has moved stays an error, so a routing failure can no longer bind the workflow to a different, empty stream with a History event that looks identical to the intended subscription. A refusal fromSubscribeToExternalStreamnow fails the workflow task with a cause like every other refusal in the same loop, instead of failing the completion call.Both commands answer
stream.enabled, since the command path does not go through the stream service and the frontend gate does not cover it. Subscriptions per execution are bounded bystream.maxSubscriptionsPerWorkflow, counting what the current task has staged, because each one costs a routed call on the completion path and another on every task start, both with the execution's lock held. The whole set of those calls shares one deadline,stream.RoutedSetBudget, so the lock hold no longer grows with the subscription count. A publish counts against the shared byte budget of every stream the execution owns.Known limits, argued rather than fixed:
TestOwnedStreamSubscriptionEndsAtContinueAsNewin Delivered stream records on Workflow Tasks and on replay. #7 pins the behaviour. A subscription to a stream in another execution does carry.StreamCursorsstill keys an owned stream by name and a standalone one by id in one map, andStreamRange.stream_idinherits that. The collision is refused rather than resolved, so nothing binds to the wrong data, but History does not say which origin a range came from. Qualifying the key is an api change and belongs with the field rename.Review round: api surface
The api branch renamed
AppendStreamRecordsCommandAttributes.stream_idtostream_nameandSubscribeStreamCommandAttributes.stream_idtostream_name_or_id, and gaveWorkflowStreamRecordsAppendedEventAttributesafrom_offsetand ato_offsetin place of a first offset and a count. Field numbers are unchanged, so the wire does not move. This branch owns the command handlers, so it is where the reads change.handleAppendStreamRecordsCommandresolvesGetStreamName(), which is what it always did with the value.handleSubscribeStreamCommandreadsGetStreamNameOrId(), which is honest about a value the server resolves as an owned name first, then as a standalone id, and failing both creates an owned stream from.RecordStreamRecordsAppendednow takes the two offsets and passes the batch'sNextOffsetstraight through rather than a count.