Repository navigation
Conversation
… delivered A nil-error PublishBatch response carrying no per-event results is reported to every event's callback as ErrCodeResultsMismatch since dc8a1dc removed the len(resp.Results) > 0 guard in sendBatch. Servers that accept a batch without per-event detail do exist: local-cre chip-router v1.x forwards every event to Kafka but returns an empty results array. With the durable emitter on top this is not a cosmetic mismatch: every event is treated as undelivered and retransmitted after 60s, forever — the ack never comes, so the node re-sends its whole event stream endlessly. On a chainlink v2.67.2-rc.0 node against chip-router v1.0.1 this produced 301k 'failed to deliver event. Relying on retransmit.' warnings on a single node within minutes of startup, a ~20x duplicated Kafka topic stream, and metric-event delivery latency gaps long enough to break downstream freshness gates. Restore the pre-dc8a1dc3 semantics: a nil-error response with an empty results array resolves every callback with nil (delivered); a non-empty array is still correlated per event, and a partial array still mismatches for the missing tail (unchanged, covered by the existing test). Adds a regression test for the zero-results case.
|
👋 cawthorne, thanks for creating this pull request! To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team. Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks! |
📊 API Diff Results
|
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The focused compatibility fix matches the stated semantics and includes appropriate regression coverage.
Review effort: Balanced
Findings: None
What changed in this PR
Restores compatibility with batch servers that acknowledge delivery without per-event results, preventing unnecessary durable-emitter retransmissions.
Changes:
- Treats successful empty
PublishBatchresponses as delivered. - Adds regression coverage while preserving partial-result mismatch handling.
| File | Description |
|---|---|
pkg/chipingress/batch/client.go |
Handles empty successful responses as batch success. |
pkg/chipingress/batch/client_test.go |
Tests callbacks for empty-result responses. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
It seems this might have been intentional: This might actually be the fix: c.c. @jmank88 |
Summary
A nil-error
PublishBatchresponse carrying no per-event results is reported to every event's callback asErrCodeResultsMismatchsince dc8a1dc removed thelen(resp.Results) > 0guard insendBatch. Servers that accept a batch without per-event detail do exist: the local-cre chip-router v1.x forwards every event to Kafka but returns an emptyresultsarray.With the durable emitter on top, this is not a cosmetic mismatch:
Evidence (chainlink v2.67.2-rc.0 node vs chip-router v1.0.1)
Clean A/B against the same router, same environment, only the chainlink-common pin differing (
v0.11.2-0.20260914191328(chipingressv0.0.11-0.20260724142814) vsv0.11.2-0.20260929093916(chipingressv0.0.11-0.20260915184316)):DurableEmitter: failed to deliver event. Relying on retransmit.-1: server returned 0 results for 1/500 eventsThe events were delivered to Kafka each time (the router forwards on receive) — only the acks were missing — so the retransmit storm duplicates the entire stream rather than losing data. The delivery-latency gaps are long enough to break downstream freshness-gated consumers (e.g. recording rules gating on
time() - <metric>_ts < 120).Fix
Restores the pre-dc8a1dc3 semantics in
sendBatch:nil(delivered);RESULTS_MISMATCHfor the missing tail (unchanged, existing test covers it).Adds a regression test for the zero-results case.
Alternatives considered
Telemetry.ChipIngressBatchEmitterEnabled = falsefalls back to the legacy per-event emitter and avoids the contract — but it's a throughput regression at scale and the field is framework-generated in CRE environments, so it isn't reachable for affected deployments.