feat(events): delivery channels and coalesced webhook deliveries - #14
Merged
Merged
Conversation
…are retried The worker compared a stored Date with an ISO string, which never matches in mongo: a failed webhook was never attempted again. Also adds the pure coalescing decision used by the next commit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An event can now be restricted to some of its channels (stored events, subscriber notifications, webhooks). A coalesced event replaces the pending delivery for the same webhook subscription and topic key instead of queuing another, which bounds storage for high-frequency signals. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A delivery that completed while a newer one was queued was picked first and reset, leaving two pending deliveries for the same subscription and topic, or restarting the backoff of a failing endpoint. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Events can now limit where they are delivered, and can coalesce their webhook deliveries.
channels(events,notifications,webhooks, all by default) selects which of the three channels an event goes to: stored event history, subscriber notifications, webhook deliveries.channelsandcoalesceare delivery instructions and are not stored on the event.coalesce: truerewrites the queued delivery for the same webhook subscription and topic key instead of adding another one. A waiting delivery is picked first, then a retrying one (its backoff is kept), then a completed one (reset to waiting). A delivery already being sent is left alone and a new one is queued.nextAttemptis now stored as an ISO string. The worker compares it with an ISO string, so failed webhooks were never retried before.@data-fair/lib-common-typesto ^1.24.0 for the new event fields.Why: high-frequency signals flooded webhook deliveries and event storage, and failed webhooks were never retried.
Heads-up:
errorwebhooks written before this change still have a DatenextAttempt. The worker will never retry them, and a coalesced event writes into them because they look like they are retrying. A one-off migration (convert them to ISO strings, or reset them) may be needed.