Skip to content

feat(io): io_uring TCP path for HTTP/1.1 and HTTP/2 - #416

Merged
gg582 merged 2 commits into
devfrom
feat/uring-tcp
Oct 18, 2026
Merged

gg582 merged 2 commits into
devfrom
feat/uring-tcp

Conversation

@gg582

@gg582 gg582 commented Oct 18, 2026

Copy link
Copy Markdown
Member

Summary

Extends the io_uring TCP path (issue #406) with a multishot accept for the async server: one persistent IORING_OP_ACCEPT + IORING_ACCEPT_MULTISHOT SQE on the listen socket replaces the one-shot POLL_ADD + accept4() drain loop. Each completion is an accepted, already SOCK_CLOEXEC | SOCK_NONBLOCK socket, dispatched straight into the HTTP pool / HTTPS dispatcher.

Note on scope: exploration showed the read side of #406 mostly already existed — the Linux reactor defaults to io_uring (with silent epoll fallback) and the RX-uring path (cwist_reactor_recv_arm, issue #179) already does IORING_OP_RECV for HTTP/1.1 keep-alive reads. The missing piece on the connection path was accept, which this PR lands.

Design

  • cwist_reactor_accept_arm() (private reactor_rx.h): arms one multishot accept per listen fd. The accept slot is a persistent registration (unlike the recycled one-shot ev_ctx slots) and tags bit 1 of user_data (bit 0 is the RX-uring conn tag; malloc alignment keeps both clear). The run loop's CQE dispatch gains a branch for it; cqe->res >= 0 is the new fd, and a completion without IORING_CQE_F_MORE is terminal: the slot is unlinked, the callback is notified once with -errno (or -ECANCELED), and the slot is freed. Slots still live at destroy are freed there.
  • async_server.c: async_accept_dispatch() now holds the shared per-socket handoff (fd_forget, TCP_NODELAY/QUICKACK, HTTPS-vs-HTTP dispatch), used by both the accept4 loop and the uring callback. cwist_async_server_loop() arms the multishot accept first and falls back to the existing cwist_reactor_add(... async_accept_cb ...) arm on failure.
  • New knob: CWIST_ACCEPT_URING (0 disables; default on when the reactor has a ring). Documented in docs/runtime-config.md and docs/ko/runtime-config.md; make env-check passes.

Fallback behavior (all silent)

  1. No io_uring ring (old kernel, seccomp, CWIST_REACTOR_BACKEND=epoll) → existing epoll reactor, unchanged.
  2. Ring exists but CWIST_ACCEPT_URING=0 or the arm fails → one-shot POLL + accept4 loop.
  3. Multishot accept terminates at runtime (e.g. -EINVAL on a kernel without IORING_ACCEPT_MULTISHOT) → terminal completion hands the listener back to the one-shot POLL loop, once.

Tests

  • tests/test_reactor_accept.c (registered as test_reactor_accept, runs 3 ways): functional accept of 64 connections on one arm (fd flags verified), CWIST_ACCEPT_URING=0 arm refusal, forced-epoll arm refusal, and a terminal-error run (accept on a non-listening socket → exactly one negative callback).
  • tests/test_http_uring_hammer.c (registered as test_http_uring_hammer): forked cwist app server, 8 client threads x 200 sequential keep-alive requests = 1600 requests; run on the default (io_uring + multishot accept), CWIST_ACCEPT_URING=0, and CWIST_REACTOR_BACKEND=epoll configurations.
  • ASan/UBSan: make SANITIZE=address,undefined WERROR=1 test_reactor_accept test_http_uring_hammer test_reactor_wake test_reactor_timer test_rx_epoll test_https test_http2 test_http test_linux_writer_fast — all pass, zero sanitizer/leak reports (io_uring behaved normally under ASan here, kernel 6.12).
  • Regression smoke: make -j12 WERROR=1 test_reactor_wake test_reactor_timer test_rx_epoll test_http test_http2 test_https test_linux_writer_fast — all pass (both backends where the test supports it).
  • make env-check and make api-check pass (no public API added).

Benchmarks (local, Linux 6.12, single 5 s runs — indicative, not gates)

benchmarks/web-frameworks/cwist_bench, wrk -t4 -c64 -d5s keep-alive:

config req/s p99 p99.9 server CPU (5 s)
io_uring + multishot accept (new default) 296k 3.78 ms 5.41 ms 2.36 s
io_uring, CWIST_ACCEPT_URING=0 298k 3.83 ms 5.50 ms 3.04 s
CWIST_REACTOR_BACKEND=epoll 314k 3.56 ms 4.88 ms 2.34 s

Connection churn (ab -n 30000 -c 64, one connection per request, 2 runs each): multishot accept ≈14.2k req/s vs ≈13.8k with the knob off (~+3%, consistent across runs); epoll ≈14.6k.

Reading: at keep-alive steady state the accept path is cold, so the change is neutral (within noise) there — its value is fewer syscalls and no per-batch re-arm under connection churn. The epoll backend being slightly ahead of io_uring overall on this box is a pre-existing characteristic of the reactor (read path untouched by this PR), not caused by this change.

Not done (deferred, explicit)

  • Fixed/registered buffers for read/write: not landed. The RX-uring path already does plain IORING_OP_RECV into the connection stash; registered buffers conflict with the stash's realloc-on-growth lifecycle and were judged too invasive for this pass.
  • io_uring write path: writes stay on writer_fast speculative send/sendmsg + the parked POLLOUT resume. An IORING_OP_SEND resume is a possible follow-up.
  • TLS through io_uring: HTTPS connections accept through the new path but BoringSSL stays synchronous on its pool threads; driving SSL via uring-backed socket I/O (BIO or WANT_READ/WANT_WRITE integration) is a larger change, deferred.
  • c100k HTTPS gate: NOT RUN — too heavy for this environment; CI gates should cover it.
  • Full benchmarks/webserver + https_gates gates: not run locally; the wrk/ab numbers above are the locally feasible subset.

Closes #406

gg582 added 2 commits October 18, 2026 19:34
Extend the io_uring TCP path with a multishot IORING_OP_ACCEPT on the
async server's listen socket: one persistent SQE replaces the one-shot
POLL_ADD + accept4 drain loop, and each completion IS an accepted,
already CLOEXEC|NONBLOCK socket.

- reactor: accept slots tag bit 1 of user_data (bit 0 is the RX-uring
  conn tag); the slot is persistent, freed on terminal completion or at
  reactor destroy. CWIST_ACCEPT_URING=0 refuses the arm.
- Fallback is silent in both directions: no io_uring ring keeps the
  epoll path, and a terminal completion (e.g. -EINVAL on kernels without
  multishot accept) hands the listener back to the one-shot POLL loop.
- TLS connections flow through the same accept into cwist_https_dispatch;
  BoringSSL stays synchronous on its pool threads (documented delta).

Tests: tests/test_reactor_accept.c (functional run, knob-off refusal,
forced-epoll refusal, terminal error) and tests/test_http_uring_hammer.c
(8x200 keep-alive requests on default, CWIST_ACCEPT_URING=0, and
CWIST_REACTOR_BACKEND=epoll). Both registered in make test; ASan/UBSan
clean.
@gg582
gg582 merged commit 8f84093 into dev Oct 18, 2026
25 checks passed
gg582 added a commit that referenced this pull request Oct 19, 2026
* feat(io): multishot io_uring accept for HTTP/1.1 and HTTP/2

Extend the io_uring TCP path with a multishot IORING_OP_ACCEPT on the
async server's listen socket: one persistent SQE replaces the one-shot
POLL_ADD + accept4 drain loop, and each completion IS an accepted,
already CLOEXEC|NONBLOCK socket.

- reactor: accept slots tag bit 1 of user_data (bit 0 is the RX-uring
  conn tag); the slot is persistent, freed on terminal completion or at
  reactor destroy. CWIST_ACCEPT_URING=0 refuses the arm.
- Fallback is silent in both directions: no io_uring ring keeps the
  epoll path, and a terminal completion (e.g. -EINVAL on kernels without
  multishot accept) hands the listener back to the one-shot POLL loop.
- TLS connections flow through the same accept into cwist_https_dispatch;
  BoringSSL stays synchronous on its pool threads (documented delta).

Tests: tests/test_reactor_accept.c (functional run, knob-off refusal,
forced-epoll refusal, terminal error) and tests/test_http_uring_hammer.c
(8x200 keep-alive requests on default, CWIST_ACCEPT_URING=0, and
CWIST_REACTOR_BACKEND=epoll). Both registered in make test; ASan/UBSan
clean.

* fix(http): make cached keep-alive timeout atomic under TSan
@gg582
gg582 deleted the feat/uring-tcp branch October 20, 2026 03:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant