Skip to content

fix(subscriptions): keep catching up when the checkpoint timer fires - #10

Merged
yordis merged 1 commit into
mainfrom
yordis/fix-subscription-checkpoint-catch-up
Sep 14, 2026
Merged

yordis merged 1 commit into
mainfrom
yordis/fix-subscription-checkpoint-catch-up

Conversation

@yordis

@yordis yordis commented Sep 14, 2026

Copy link
Copy Markdown
Member
  • A subscription catching up would jump its checkpoint past every unread event between the last sent event and the head, silently losing them; upstream reports this hitting production twice on a replay from origin, skipping roughly 40 million events
  • Carrying it here rather than waiting for upstream to merge, because a subscription that quietly skips events is indistinguishable from one that has caught up

Cherry-picked from commanded/eventstore#327 (cb21b09), authored by Doru Pirvu, including its regression test.

A subscription that was catching up moved to the subscribed state when
its checkpoint_after timer fired. With the queue drained it stopped
reading, and the next live event matched last_received + 1 because
notifications received during catch-up are tracked. That event was
sent and acknowledged, so the checkpoint jumped past every unread event
between last_sent and the head.

Persist the checkpoint but stay in the current state for
request_catch_up, catching_up and max_capacity.

(cherry picked from commit cb21b0924492f96a7f06e6963d6459c2ed84b00b)
Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Changes subscription checkpoint FSM transitions in a path that previously caused silent mass event loss in production replays; incorrect behavior would still be hard to detect without thorough catch-up testing.

Overview
Fixes a bug where a subscription catching up could persist its checkpoint and transition to subscribed while events were still in-flight or not yet delivered, effectively skipping everything between the last sent event and the stream head.

checkpoint handling in SubscriptionFsm no longer forces :subscribed from catch-up-related states. After persist_checkpoint, the FSM stays in request_catch_up, catching_up, or max_capacity (unchanged behavior when already subscribed). That keeps draining the backlog instead of looking “caught up” after the timer fires.

Adds a regression test that arms a short checkpoint_after, appends events mid catch-up, sleeps through the timer while acking one-by-one with buffer_size: 1, and asserts no events are missed.

Reviewed by Cursor Bugbot for commit f28afe5. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitai Bot commented Sep 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 58 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: d12d669c-ff8a-4972-8c9b-6458e2207231

📥 Commits

Reviewing files that changed from the base of the PR and between f6a1fc6 and f28afe5.

📒 Files selected for processing (2)
  • lib/event_store/subscriptions/subscription_fsm.ex
  • test/subscriptions/subscription_catch_up_test.exs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@yordis
yordis merged commit 23cbf8e into main Sep 14, 2026
8 checks passed
@yordis
yordis deleted the yordis/fix-subscription-checkpoint-catch-up branch September 14, 2026 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants