fix(temporal): an activity failure reports the attempt the server ran (#547) - #548
Conversation
…#547) A failure or a timeout read on Temporal always said attempt 1, where the journal backends report the attempt that failed, so $e->attempt() and the exception's message depended on the backend. The history now records each ActivityTaskStarted's attempt, and ACTIVITY_TASK_FAILED and ACTIVITY_TASK_TIMED_OUT read it through their started_event_id. With no started event (a schedule-to-start timeout) it stays 1, as on the journal. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gplanchat
left a comment
There was a problem hiding this comment.
Review by dave, at 9d11935d. Verdict: changes needed, 1 finding. The parity fix is right. The claim that replay is safe is not.
F1: in-flight Temporal runs that put $e->attempt() into a payload diverge after upgrading
The PR body argues that divergence is checked by slot and kind, not payload. It is checked on payload too. ExecutionContext::activity() calls refusePayloadDivergence() with activityPayloadForSlot() (src/Durable/ExecutionContext.php:113-119), and child workflows and Nexus go through the same rule. Before this PR, attempt() was always 1 on Temporal. After it, the same code computes the real attempt, so a run whose history recorded 1 in a payload now fails its replay.
Proved with a throwaway WorkflowTaskRunner probe (deleted). The workflow catches the failure of double and schedules greet('attempt-' . $e->attempt()). The history holds double started as attempt 3 and then failed, followed by greet scheduled with {"name":"attempt-1"}, as the code before this PR recorded it.
- On this branch:
WorkflowTaskFailure: Replay divergence at activity slot 1 of execution "wf": "greet" is still the same activity, but its payload changed at byte 17. History recorded {"name":"attempt-1"}, code scheduled {"name":"attempt-3"}. - With
main'sTemporalExecutionHistory: no divergence; the run waits ongreet.
This is exactly the guarantee the review rules name (replay determinism), so it needs a decision rather than a merge as is. Options:
- (a) Keep the change and document the break. Add an UPGRADE entry: on Temporal, a run in flight whose workflow passes
$e->attempt()into an activity, child or Nexus payload diverges after the upgrade; drain those runs first, or branch on a version (versionForChangeId). Add a test pinning that a plain replay, where the attempt reaches no payload, does not diverge. - (b) Report the real attempt only to histories recorded after the upgrade. No marker exists for that today, so this would need a new one, which is heavier.
For 0.1.0-beta1 I lean (a), since the journal backends always reported the real attempt, but it is the project lead's call.
Checked and OK
- Parity: the journal reports the real attempt, and this PR aligns Temporal. The new tests fail on
mainand pass here: third-attempt failure, start-to-close timeout after attempt 2, schedule-to-start keeping 1. started_event_id: Temporal records only the last attempt'sActivityTaskStarted, andFAILED/TIMED_OUTpoint at it. Earlier attempts leave no event to miss, so an old history cannot carry a retried attempt without its started event.
Non-blocking
A schedule-to-close timeout that elapses while attempt N is still waiting in the queue has started_event_id = 0, so it reports attempt 1. ActivityTaskTimedOut carries no attempt, so nothing better is readable there. Worth one line in the docblock of attemptOf().
Gates
259 Temporal unit tests pass; PHPStan and Psalm are clean.
…, review) Three runner tests over a history where double(2) ran three attempts, failed, and the workflow scheduled greet(...) under the old reading: - copying $e->attempt() into greet's payload diverges on replay (it passes with main's reader, which still says attempt 1); - a workflow that does not copy it replays with no divergence; - UPGRADE.md's way out, a change point that keeps attempt 1 for runs that started before it, replays with no divergence. The attempt docblock also names the schedule-to-close case that reads 1. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…#547, review) In-flight Temporal runs that put the attempt into a later payload diverge on replay once the real attempt is read. The entry says who is affected, the error they would see, and the two ways out: drain, or a change point that keeps attempt 1 for runs that started before it, the snippet a test replays. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gplanchat
left a comment
There was a problem hiding this comment.
Re-check by dave, at a899ec83. Verdict: OK. F1 is resolved by option (a), as the project lead ruled.
- The three new runner tests pass. With
main'sTemporalExecutionHistoryswapped in, exactly one of them fails,testAnAttemptCopiedIntoAPayloadDivergesOnARunRecordedUnderTheOldReading, so it pins the change. The plain-replay test and the versioned way out pass on both. - The UPGRADE "Replay:" entry names who is affected, the divergence message, and both ways out. Its snippet uses the real API (
WorkflowEnvironment::version($changeId, ChangePoint::DEFAULT_VERSION, 1), the same pattern as the method's own docblock), andtestTheVersionedWayOutReplaysARunRecordedUnderTheOldReadingexercises it. - The docblock now names the schedule-to-close-while-queued case.
Closes #547.
Done when
$e->attempt()on the journal backend and on Temporal.AnActivityFailureReportsItsAttemptTest::testAFailureOnTheThirdAttemptReportsTheAttemptTheJournalReportsbuilds the journal's own exception (ActivityFailureEventFactorythentoThrowable(), attempt 3), and asserts the sameattempt()and the same message on Temporal.ACTIVITY_TASK_FAILEDandACTIVITY_TASK_TIMED_OUTread the attempt.ExecutionContext::activity()refuses a payload divergence (ExecutionContext.php:113-119), as dave found.testAnAttemptCopiedIntoAPayloadDivergesOnARunRecordedUnderTheOldReadingpins the one affected pattern. A run recorded under the old reading (attempt 1), whose workflow copies$e->attempt()into a later payload, now fails replay loudly. With main's reader it does not throw.testAPlainReplayOfARetriedThenFailedActivityDoesNotDiverge: a workflow that does not copy the attempt replays the same history unchanged.testTheVersionedWayOutReplaysARunRecordedUnderTheOldReading: UPGRADE.md's snippet, aversion('real-activity-attempt', …)change point that keeps attempt 1 for runs that started before it, replays the old history without divergence.UPGRADE.mdhas a "Replay:" entry: who is affected, the error they would see, and the two ways out (drain, or the change point).What changes
TemporalExecutionHistorynow readsACTIVITY_TASK_STARTEDand records each started event'sattempt. Temporal writes only the last attempt's started event, so that is the attempt that failed. The failure and timeout branches look it up through theirstarted_event_id, falling back to 1 when there is none.Checks
composer cs:check, PHPUnit (2,355 tests), PHPStan and Psalm all pass locally.Reviewer: dave.