Skip to content

fix(disk buffer): backpressure on runtime capacity errors - #26077

Draft
Jansen-w wants to merge 6 commits into
vectordotdev:masterfrom
Jansen-w:fix/disk-buffer-enospc-backpressure
Draft

fix(disk buffer): backpressure on runtime capacity errors#26077
Jansen-w wants to merge 6 commits into
vectordotdev:masterfrom
Jansen-w:fix/disk-buffer-enospc-backpressure

Conversation

@Jansen-w

@Jansen-w Jansen-w commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Runtime filesystem capacity errors can be temporary, but disk-buffer writes previously treated them as fatal I/O failures. This change makes disk_v2 writers:

  • recognize StorageFull, ENOSPC, and EDQUOT errors;
  • retain partially written bytes and resume from the committed offset without duplication;
  • apply backpressure within the owning write or flush future while waiting for reader progress or exponential retry;
  • preserve logical when_full behavior while physical capacity exhaustion consistently backpressures every policy;
  • emit one structured capacity-exhausted event and one recovery event per episode, with buffer path, operation, retry count, delay, and duration;
  • preserve structured fatal synchronization errors and keep startup/non-capacity failures fatal.

Production writes remain session-owned synchronous syscalls. A kernel write cannot be portably cancelled and can delay cancellation until it returns; keeping it session-owned prevents writes from outliving the buffer lock or racing a reopened buffer.

Vector configuration

No new configuration is required. The behavior applies to existing disk_v2 buffers.

How did you test this PR?

  • cargo test -p vector-buffers (132 passed, 2 pre-existing ignored)
  • cargo clippy -p vector-buffers --all-targets -- -D warnings
  • cargo fmt --all -- --check
  • cargo vdev check events
  • cargo vdev check changelog-fragments
  • Deterministic regressions for partial writes, cancelled flush/write resume, small and large writes after cancellation, timer/reader wakeups, retry ownership and telemetry, atomic/fallback file opens, rotation synchronization, startup failures, and non-capacity errors

Real exhausted-filesystem and Windows-specific behavior remain covered through deterministic fault injection rather than platform integration tests.

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. A changelog fragment is included.
  • No. A maintainer will apply the no-changelog label to this PR.

References

None.

@Jansen-w Jansen-w closed this Aug 10, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 10, 2026
@Jansen-w Jansen-w reopened this Aug 26, 2026
@Jansen-w
Jansen-w force-pushed the fix/disk-buffer-enospc-backpressure branch from abf82d6 to 95542cc Compare August 26, 2026 17:28
@github-actions github-actions Bot added docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: external docs Anything related to Vector's external, public documentation and removed docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. labels Aug 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

domain: external docs Anything related to Vector's external, public documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant