bench: concurrent positioned-read harness (the shape ecstore serves)#5
Merged
Conversation
`streaming_bench` answered "should streaming reads use io_uring" (no:
kernel readahead wins). It could not answer "is io_uring worth anything
for rustfs", because a single sequential stream exercises neither of
io_uring's levers: batched submission, and keeping the IO off a blocking
thread. Both only appear under many concurrent positioned reads on one
disk — what erasure-coded shard reads (`pread_bytes`) actually do.
Adds `examples/concurrent_pread_bench.rs` + `bench-concurrent-pread.sh`
with four strategies that isolate where the time goes:
std_open_pread today's StdBackend: spawn_blocking{open; pread}
std_cached_pread spawn_blocking{pread} on a pre-opened fd
uring_open_read today's UringBackend: spawn_blocking{open;stat} + read_at
uring_cached_read pre-opened fd + read_at, no spawn_blocking
std_open vs uring_open prices today's wiring; *_cached vs *_open prices
the per-read open; uring_cached is the ceiling an fd cache would buy.
The sweep runs a cold leg (drop_caches) and a warm leg. The warm leg is
not a curiosity: on a throughput-throttled disk the cold leg saturates
the device and hides the software overhead being priced.
Same hardening as streaming_bench: the file is created only when absent
(create_new + O_NOFOLLOW, never truncates a caller-supplied path even
though the sweep runs as root), content is offset-addressable, and
BENCH_VERIFY=1 checks every delivered byte at its claimed offset so a
strategy reading the wrong offsets cannot post a fast number. The runner
replays all four strategies under verification before measuring.
Verified: clippy -D warnings clean; all four strategies verify byte-exact
under real io_uring at an unaligned read size; the guards reject a
wrong-size path and a symlink.
Co-Authored-By: heihutu <heihutu@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the concurrent positioned-read benchmark — the workload
pread_bytesactually serves (many independent preads per disk under load), which is where io_uring's levers (batched submission, no blocking-pool hop) can appear.streaming_benchcould not answer this: a single sequential stream exercises neither lever.Four strategies isolate the cost:
std_open_pread(today's StdBackend),std_cached_pread(prices the open away),uring_open_read(today's UringBackend, stillspawn_blockingfor open+stat),uring_cached_read(the fd-cache ceiling).Cold + warm legs: on a throughput-throttled disk the cold leg saturates the device and hides the very overhead being priced, so the warm leg is where software cost is visible.
Carries the same hardening as
streaming_bench: never truncates or follows a symlink at a caller-supplied path (the sweep runs as root), offset-addressable content, andBENCH_VERIFY=1byte-exact checking replayed across all strategies before any measurement.Findings from the first run are posted to the backlog issue. Library code unchanged.