Skip to content

perf: skip notification reads without local subscribers - #328

Open
esse wants to merge 1 commit into
commanded:masterfrom
esse:fix/skip-unsubscribed-notification-reads
Open

esse wants to merge 1 commit into
commanded:masterfrom
esse:fix/skip-unsubscribed-notification-reads

Conversation

@esse

@esse esse commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Every node currently reads and deserializes each event notification before discovering that the local PubSub registry has no recipient. In a cluster with subscribers concentrated on a few nodes, this multiplies database reads and decoding work across the other nodes.

Filter notification topics before reading storage, using the existing local Registry. Exact-stream and $all notifications are checked separately. Persistent subscriptions still register before catching up from their durable checkpoint; acknowledgement and checkpoint handling are unchanged.

The regressions cover skipped reads, unrelated/exact/$all subscribers, selectors and mappers, unregistering, and persistent catch-up after a skipped notification. Both fail against the original publisher and pass with the filter.

Validation on Elixir 1.16.1 / OTP 26.2.5.7 with an isolated PostgreSQL 12 server:

  • Both new regressions fail on the original publisher and pass with the filter.
  • mix test.all: passes for the default, JSONB, text-identifier and migration configurations (407 tests and 2 doctests per configuration, with the suite's normal exclusions).
  • mix format --check-formatted, mix compile --warnings-as-errors, and mix dialyzer: pass.

Comment on lines +64 to 65
|> Stream.filter(&PubSub.subscribed?(event_store, &1.stream_uuid))
|> Stream.map(&read_events(&1, state))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would appreciate it if this were placed behind a feature flag since this affects the system level architecture, this is not a local decision.

A subscribe can appear between the filter and map, and I do not know who in the codebases I maintain understands the implications of this change.

I would rather continue letting the Subscription engine handle the optimizations you are trying to implement than use :eventstore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants