From 769d7611f921cea300243ca7007aaede6c64995f Mon Sep 17 00:00:00 2001 From: Lee Yunjin Date: Fri, 16 Oct 2026 23:49:17 +0900 Subject: [PATCH 1/4] perf(reactor): return the arena top to the OS after sustained idleness glibc never gives back the free top of the main arena unless asked, so a mostly-idle deployment retains post-load heap forever. Measurements on the rss_gate workload (gdb-forced malloc_trim(0) after one load round): median 1224 KiB per worker returned, master 1632 KiB; stable 5 s later. Track an idle streak per reactor (CLOCK_MONOTONIC anchor of the first event-free wake) and call malloc_trim(0) on a zero-event wake once the streak reaches CWIST_MALLOC_TRIM_IDLE_SEC seconds (default 30, cached getter, 0 = off). Runs exclusively on wakes that dispatched nothing, so there is no hot-path cost; weak malloc_trim declaration keeps non-glibc Linux libcs (musl) linking with the trim degraded to a no-op. Also fixes the io_uring idle check being unreachable: the timeout enter returns -ETIME (not 0), which previously hit 'continue' before the head/tail check. Verified end-to-end: workers 1248-1524 KiB and master 1480 KiB returned after load on the auto-trim path; rss_gate.sh PASS (idle 36696 KiB, growth 232 KiB); wrk c64/c256 within noise. --- docs/ko/runtime-config.md | 1 + docs/runtime-config.md | 1 + src/sys/io/reactor.c | 81 ++++++++++++++++++++++++++++++++++++--- 3 files changed, 78 insertions(+), 5 deletions(-) diff --git a/docs/ko/runtime-config.md b/docs/ko/runtime-config.md index e49c0bd6..0a7e2092 100644 --- a/docs/ko/runtime-config.md +++ b/docs/ko/runtime-config.md @@ -30,6 +30,7 @@ | `CWIST_DRAIN_TIMEOUT` | supported | 초, `0`–`3600` | `5` | 종료 시 앱을 해제하기 전에 처리 중인 요청을 기다리는 시간. 범위 밖 값은 stderr에 알리고 무시합니다. | | `CWIST_FD_LIMIT_TARGET` | supported | 0보다 큰 정수 | `1050000` | 서버가 시작할 때 스스로 올리는 열린 파일 soft limit(hard limit까지). [C1M 파일 한도](c1m-file-limits.md) 참고. | | `CWIST_MALLOC_ARENA_MAX` | tuning | 0 이상 정수 | 설정 안 함(glibc 기본값) | glibc 전용: `mallopt(M_ARENA_MAX)`. `1`은 할당기 동시성을 조금 내주고 RSS를 크게 줄입니다. | +| `CWIST_MALLOC_TRIM_IDLE_SEC` | tuning | 초, `0`–`86400` | `30` | glibc 전용(그 외 libc에서는 동작 없음): 이벤트 없이 유휴 상태가 누적된 시간이 이 값에 도달하면 리액터가 이벤트 없는 wake에서 `malloc_trim(0)`을 호출해 glibc 아레나 상단의 여유 메모리를 OS에 반환합니다. 유휴 wake에서만 동작해 핫 패스 비용은 없습니다. 로드 라운드 후 워커당 중간값 약 1.2 MiB 절감. `0`이면 끕니다. | ## Classic 스레드 풀 diff --git a/docs/runtime-config.md b/docs/runtime-config.md index 83ecc9ba..1e1c83b7 100644 --- a/docs/runtime-config.md +++ b/docs/runtime-config.md @@ -30,6 +30,7 @@ Each variable has one status. The status says what v4 promises about it. | `CWIST_DRAIN_TIMEOUT` | supported | seconds, `0`–`3600` | `5` | How long shutdown waits for in-flight requests before the app is destroyed. Out of range values are reported on stderr and ignored. | | `CWIST_FD_LIMIT_TARGET` | supported | integer > 0 | `1050000` | Open-file soft limit the server raises itself to at start (capped by the hard limit). See [C1M file limits](c1m-file-limits.md). | | `CWIST_MALLOC_ARENA_MAX` | tuning | integer ≥ 0 | unset (glibc default) | glibc only: `mallopt(M_ARENA_MAX)`. `1` trades some allocator concurrency for a much lower RSS. | +| `CWIST_MALLOC_TRIM_IDLE_SEC` | tuning | seconds, `0`–`86400` | `30` | glibc only (no-op elsewhere): seconds of cumulative event-free reactor idleness after which the reactor calls `malloc_trim(0)` on a zero-event wake, returning the free arena top to the OS. Runs only on idle wakes, so the hot path pays nothing. Measured ≈1.2 MiB median per worker after a load round. `0` disables the trim. | ## Classic thread pool diff --git a/src/sys/io/reactor.c b/src/sys/io/reactor.c index 06a7faf3..8e1eddf6 100644 --- a/src/sys/io/reactor.c +++ b/src/sys/io/reactor.c @@ -52,6 +52,10 @@ #include #include #include +/* malloc_trim(3): glibc-only . The weak declaration below lets the + * binary link (and the trim degrade to a no-op) on Linux libcs without it, + * e.g. musl. */ +#include #ifndef IORING_ENTER_EXT_ARG #define IORING_ENTER_EXT_ARG (1U << 3) @@ -284,6 +288,10 @@ struct cwist_reactor { uint64_t max_us; uint64_t over_5ms; /* samples beyond 5000 us */ } probe[2]; + /* CLOCK_MONOTONIC timestamp of the first event-free wake that started the + * current idle streak (0 = not idle). Run thread only. Drives the idle + * malloc_trim cadence; see reactor_idle_note_wake(). */ + uint64_t idle_since_ns; #endif }; @@ -1406,6 +1414,60 @@ static uint64_t reactor_round_budget_us(void) { return (uint64_t)v; } +/* Idle malloc trim (measured ~1.2 MiB median per worker returned to the OS on + * an idle HTTPS worker after a load round): glibc never gives back the free + * top of the main arena unless asked. CWIST_MALLOC_TRIM_IDLE_SEC bounds how + * many seconds of cumulative event-free idleness a reactor may accumulate + * before it calls malloc_trim(0) on a wake that saw zero events -- a mostly + * idle deployment sheds arena top memory without any hot-path cost. The knob + * is cached after the first read like the other env knobs in this file; 0 + * disables the trim. */ +static uint32_t reactor_malloc_trim_idle_sec(void) { + static _Atomic int cached = -1; + int v = atomic_load_explicit(&cached, memory_order_relaxed); + if (v < 0) { + const char *s = getenv("CWIST_MALLOC_TRIM_IDLE_SEC"); + if (s && *s) { + char *end = NULL; + long parsed = strtol(s, &end, 10); + v = (end != s && parsed >= 0 && parsed < 86400) ? (int)parsed : 30; + } else { + v = 30; + } + atomic_store_explicit(&cached, v, memory_order_relaxed); + } + return (uint32_t)v; +} + +/* Weak: musl and other non-glibc Linux libcs have no malloc_trim. */ +extern int malloc_trim(size_t __pad) __attribute__((weak)); + +/* A wake that dispatched events (or ran posted work) ends the idle streak. */ +static void reactor_idle_note_eventful(cwist_reactor_t *r) { r->idle_since_ns = 0; } + +/* Bookkeeping for a wake that saw zero events. Starts an idle streak on the + * first such wake; once the streak has lasted CWIST_MALLOC_TRIM_IDLE_SEC + * cumulative seconds, returns the arena top to the OS and re-anchors the + * streak. Runs on the run thread only, after the wait returned empty. */ +static void reactor_idle_note_wake(cwist_reactor_t *r) { + uint32_t sec = reactor_malloc_trim_idle_sec(); + if (sec == 0 || &malloc_trim == NULL) { + r->idle_since_ns = 0; + return; + } + struct timespec ts; + clock_gettime(CLOCK_MONOTONIC, &ts); + uint64_t now = (uint64_t)ts.tv_sec * 1000000000ull + (uint64_t)ts.tv_nsec; + if (r->idle_since_ns == 0) { + r->idle_since_ns = now; + return; + } + if (now - r->idle_since_ns >= (uint64_t)sec * 1000000000ull) { + malloc_trim(0); + r->idle_since_ns = now; + } +} + #if defined(__x86_64__) || defined(__i386__) #define reactor_cpu_relax() __builtin_ia32_pause() #elif defined(__aarch64__) @@ -1499,17 +1561,19 @@ void cwist_reactor_run(cwist_reactor_t *reactor) { if (!cq_pending) { int ret = sys_io_uring_enter_timeout(reactor->impl.ring_fd, 0, 1, IORING_ENTER_GETEVENTS, &idle_ts); - if (ret < 0) { - if (errno == EINTR) continue; - if (errno == ETIME) continue; - break; - } + /* EINTR and ETIME both leave the CQ untouched, so the + * head/tail check below still decides eventfulness. */ + if (ret < 0 && errno != EINTR && errno != ETIME) break; } uint32_t head = __atomic_load_n(reactor->impl.cq_head, __ATOMIC_ACQUIRE); uint32_t tail = *reactor->impl.cq_tail; if (head == tail) { + /* Zero-completion wake: pure idle. Feeds the idle malloc_trim + * cadence; see reactor_idle_note_wake(). */ + reactor_idle_note_wake(reactor); continue; } + reactor_idle_note_eventful(reactor); reactor_tsan_acquire(&reactor->impl); reactor->dispatching = true; const uint64_t round_budget_us = reactor_round_budget_us(); @@ -1635,6 +1699,13 @@ void cwist_reactor_run(cwist_reactor_t *reactor) { if (errno == EINTR) continue; break; } + if (n == 0) { + /* Event-free idle wake: feeds the idle malloc_trim cadence; + * see reactor_idle_note_wake(). */ + reactor_idle_note_wake(reactor); + } else { + reactor_idle_note_eventful(reactor); + } /* Cooperative queuing, same cadence as the io_uring branch (see * the comment there): epoll_wait hands back up to 1024 events in * one array, and this loop used to run them all with no drain From a13f76a37820c4d85a74887e34854c1fa2c1a7e9 Mon Sep 17 00:00:00 2001 From: Lee Yunjin Date: Fri, 16 Oct 2026 23:49:17 +0900 Subject: [PATCH 2/4] perf(tls): bound the BoringSSL server session cache Nothing called SSL_CTX_sess_set_cache_size, so the internal server session cache kept BoringSSL's 1024*20-session default: under pathological session-ID churn that is up to ~20 MiB per worker retained forever. Measured real usage is zero (SSL_CTX_sess_number stays 0 through wrk/ab/h2load/openssl-s_client churn, TLS 1.2 included -- TLS 1.3 tickets are stateless), so this is bounded-retention hardening with no hot-path cost (the cap is consulted on insert only). Set SSL_CTX_sess_set_cache_size(ctx, N) at both SSL_CTX sites (https.c, http3.c) with N from CWIST_TLS_SESS_CACHE_SIZE (default 1024, 0 = BoringSSL default), via a cached getter in tls_chain. --- docs/ko/runtime-config.md | 1 + docs/runtime-config.md | 1 + src/net/http/http3.c | 1 + src/net/http/https.c | 1 + src/net/http/tls_chain.c | 33 +++++++++++++++++++++++++++++++++ src/net/http/tls_chain.h | 6 ++++++ 6 files changed, 43 insertions(+) diff --git a/docs/ko/runtime-config.md b/docs/ko/runtime-config.md index 0a7e2092..e4e5e16a 100644 --- a/docs/ko/runtime-config.md +++ b/docs/ko/runtime-config.md @@ -58,6 +58,7 @@ | `CWIST_HTTPS_IDLE_TIMEOUT_MS` | supported | 밀리초, 0보다 큼 | `30000` | HTTPS 연결을 닫기 전 유휴 시간. | | `CWIST_HTTPS_PARK` | tuning | `0`/`false`면 끔 | 켜짐 | 유휴 TLS 연결이 풀 스레드를 붙잡지 않고 reactor에 대기합니다. | | `CWIST_HTTPS_HS_SHARDS` | tuning | `1`–`16` | CPU 코어 수, `4`–`16`으로 제한 | 핸드셰이크 샤드 수. 연결 폭주 튜닝용. | +| `CWIST_TLS_SESS_CACHE_SIZE` | tuning | 정수, `0`–`1048576` | `1024` | BoringSSL 서버측 세션 캐시 상한(`SSL_CTX_sess_set_cache_size`)으로, 모든 서버 TLS 컨텍스트(HTTP/1.1, HTTP/2, HTTP/3)에 적용됩니다. 워커 프로세스당 TLS 1.2 세션 ID 보유량을 제한합니다. `0`이면 BoringSSL 기본값(제한 없음)을 씁니다. 세션 삽입 시에만 참조합니다. | ## HTTP/2 diff --git a/docs/runtime-config.md b/docs/runtime-config.md index 1e1c83b7..ea9738be 100644 --- a/docs/runtime-config.md +++ b/docs/runtime-config.md @@ -58,6 +58,7 @@ Each variable has one status. The status says what v4 promises about it. | `CWIST_HTTPS_IDLE_TIMEOUT_MS` | supported | milliseconds > 0 | `30000` | Idle time before an HTTPS connection is closed. | | `CWIST_HTTPS_PARK` | tuning | `0`/`false` disables | on | Park idle TLS connections in the reactor instead of holding a pool thread. | | `CWIST_HTTPS_HS_SHARDS` | tuning | `1`–`16` | CPU cores, clamped to `4`–`16` | Handshake shards. For connect-burst tuning. | +| `CWIST_TLS_SESS_CACHE_SIZE` | tuning | integer, `0`–`1048576` | `1024` | Cap on the BoringSSL server-side session cache (`SSL_CTX_sess_set_cache_size`), applied to every server TLS context (HTTP/1.1, HTTP/2, HTTP/3). Bounds TLS 1.2 session-ID retention per worker process. `0` leaves the BoringSSL default (uncapped). Consulted on session insert only. | ## HTTP/2 diff --git a/src/net/http/http3.c b/src/net/http/http3.c index ccb977aa..41c7f5b3 100644 --- a/src/net/http/http3.c +++ b/src/net/http/http3.c @@ -2577,6 +2577,7 @@ static int cwist_h3_ticket_key_cb(SSL *ssl, uint8_t *key_name, uint8_t *iv, EVP_ */ static void cwist_h3_setup_session_tickets(SSL_CTX *ssl_ctx) { SSL_CTX_set_session_cache_mode(ssl_ctx, SSL_SESS_CACHE_SERVER); + SSL_CTX_sess_set_cache_size(ssl_ctx, cwist_tls_sess_cache_size()); pthread_once(&g_h3_ticket_key_ex_data_once, cwist_h3_ticket_key_ex_data_init); if (g_h3_ticket_key_ex_data_idx < 0) return; diff --git a/src/net/http/https.c b/src/net/http/https.c index 78e19c1b..4f8e001d 100644 --- a/src/net/http/https.c +++ b/src/net/http/https.c @@ -1180,6 +1180,7 @@ static void cwist_https_apply_base_tls_defaults(SSL_CTX *ssl_ctx) { * OpenSSL refuses to resume cached sessions. Combined with HTTP/1.1 * keep-alive this removes most full (asymmetric) handshakes. */ SSL_CTX_set_session_cache_mode(ssl_ctx, SSL_SESS_CACHE_SERVER); + SSL_CTX_sess_set_cache_size(ssl_ctx, cwist_tls_sess_cache_size()); SSL_CTX_set_session_id_context(ssl_ctx, (const unsigned char *)"cwist-v1", 8); SSL_CTX_set_timeout(ssl_ctx, 300); } diff --git a/src/net/http/tls_chain.c b/src/net/http/tls_chain.c index 2bcc856c..b9f5e9b8 100644 --- a/src/net/http/tls_chain.c +++ b/src/net/http/tls_chain.c @@ -15,6 +15,7 @@ #include #include +#include #include #include #include @@ -504,6 +505,38 @@ static X509 *cwist_fetch_issuer_from_aia(X509 *subject) { return issuer; } +/** Bounded retention for the BoringSSL server-side session cache. + * + * BoringSSL's implicit cap is 20k entries per SSL_CTX (ssl.h + * SSL_SESSION_CACHE_MAX_SIZE); nothing in CWIST called + * SSL_CTX_sess_set_cache_size, so a deployment seeing TLS 1.2 session-ID + * resumption could accumulate up to that per worker process with no bound it + * could tune. The knob caps the cache on every server context. Measured + * session-cache usage under wrk/ab/h2load and openssl s_client TLS 1.2 churn + * is 0 (TLS 1.3 tickets are stateless and never enter the server cache), so + * today this is retention hardening, not an RSS win. The cap is consulted on + * session insert only: zero hot-path impact. + * + * @return The validated CWIST_TLS_SESS_CACHE_SIZE env override or 1024; + * cached after first read. 0 leaves the BoringSSL default uncapped. + */ +unsigned long cwist_tls_sess_cache_size(void) { + static _Atomic long cached = -1; + long v = atomic_load_explicit(&cached, memory_order_relaxed); + if (v < 0) { + const char *s = getenv("CWIST_TLS_SESS_CACHE_SIZE"); + if (s && *s) { + char *end = NULL; + long parsed = strtol(s, &end, 10); + v = (end != s && parsed >= 0 && parsed <= 1048576L) ? parsed : 1024; + } else { + v = 1024; + } + atomic_store_explicit(&cached, v, memory_order_relaxed); + } + return (unsigned long)v; +} + /** * @brief Autoload missing intermediate CA certificates into an SSL_CTX chain. * diff --git a/src/net/http/tls_chain.h b/src/net/http/tls_chain.h index 6f64dd84..283ddb50 100644 --- a/src/net/http/tls_chain.h +++ b/src/net/http/tls_chain.h @@ -5,4 +5,10 @@ int cwist_tls_autoload_intermediates(SSL_CTX *ssl_ctx); +/* Bounded retention for the BoringSSL server-side session cache. + * CWIST_TLS_SESS_CACHE_SIZE cached getter (implemented in tls_chain.c): + * the cache cap applied to every server SSL_CTX; 0 keeps the BoringSSL + * default (no explicit cap). See docs/runtime-config.md. */ +unsigned long cwist_tls_sess_cache_size(void); + #endif /* CWIST_NET_HTTP_TLS_CHAIN_H */ From bddcc7a2cd66d21be04d3252ab952f99d2c00d21 Mon Sep 17 00:00:00 2001 From: Lee Yunjin Date: Fri, 16 Oct 2026 23:49:17 +0900 Subject: [PATCH 3/4] perf(http): start a virgin C1M stash at the 4 KiB idle floor, not 16 KiB The first arm of a new connection grew the rbuf stash straight to CWIST_HTTP_READ_BUFFER_SIZE (16 KiB), and the doubling ladder in http_async_stash_grow used the same base -- so every young connection paid 16 KiB of rbuf (12 KiB beyond the 4 KiB idle floor it would be shrunk to anyway after 10 s idle) even though the median request never exceeds the floor. Grow virgin stashes to CWIST_ASYNC_STASH_IDLE and make the doubling base the floor; any request above the floor regrows through the unchanged ladder to the same cap as before. Measured RSS win (1 worker, connections younger than the idle window, deterministic across repeats): 4000 idle keep-alive connections retain 24.8 KiB/conn before, 12.9 KiB/conn after -- -11.7 KiB/conn (~47 MB per 4000). Perf gate: wrk c64/c256 small requests and 8 KiB POST within noise (8 KiB POST c64 +2.6%, c256 +2.9%, P99.999 never worse); test_idle_reaper/fairness/rx_epoll/pipeline/async_defer pass with default and override env. --- src/net/http/http.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/src/net/http/http.c b/src/net/http/http.c index c593aea3..93acc477 100644 --- a/src/net/http/http.c +++ b/src/net/http/http.c @@ -1056,7 +1056,11 @@ static bool http_async_arm_wait(int fd, cwist_http_async_conn_t *conn, }; if (cwist_reactor_rx_supported(reactor) && !conn->peer_eof && !conn->rx_recv_inflight && !conn->rx_prefers_poll) { - if (conn->cap == 0 && !http_async_stash_grow(conn, CWIST_HTTP_READ_BUFFER_SIZE)) { + /* Virgin stash: grow to the 4 KiB idle floor, not the 16 KiB read + * buffer. A mostly-idle C1M connection then pays ~4 KiB, not 16 KiB, + * of rbuf on first arm; any request above the floor regrows through + * http_async_stash_grow exactly as a regrown stash does today. */ + if (conn->cap == 0 && !http_async_stash_grow(conn, CWIST_ASYNC_STASH_IDLE)) { /* Stash allocation failed: the legacy fill path retries growth * when POLL fires and closes on failure, matching today. */ } else if (conn->cap > 0) { @@ -4858,7 +4862,12 @@ static bool http_async_stash_grow(cwist_http_async_conn_t *conn, size_t need) { if (conn->rx_recv_inflight) return false; if (need > CWIST_ASYNC_STASH_MAX) return false; if (conn->cap >= need) return true; - size_t cap = conn->cap ? conn->cap : CWIST_HTTP_READ_BUFFER_SIZE; + /* Doubling base: a virgin stash starts at the 4 KiB idle floor (the + * first-arm site grows it to exactly that), so an idle C1M connection + * pays floor memory, not CWIST_HTTP_READ_BUFFER_SIZE. Above the floor + * the doubling ladder is unchanged: need 16 KiB on a virgin stash ends + * at the same 16 KiB cap as before. */ + size_t cap = conn->cap ? conn->cap : CWIST_ASYNC_STASH_IDLE; while (cap < need) { if (cap > CWIST_ASYNC_STASH_MAX / 2) { cap = CWIST_ASYNC_STASH_MAX; From 40b5587547b03b9c8d71aa56de31450b2a3c3169 Mon Sep 17 00:00:00 2001 From: Lee Yunjin Date: Fri, 16 Oct 2026 23:49:17 +0900 Subject: [PATCH 4/4] perf(http): shrink the stashes when a connection defers (SSE/long-poll) A connection parked on cwist_async (SSE, long-poll) used to hold its full rbuf+obuf stashes for the entire deferral -- the idle shrink only ran at re-arm, which a parked connection never reaches. Shrink toward the idle floor at defer: ~28 KiB/conn retained becomes ~KB for the whole deferral (measured: 3000 parked connections retain 28.8 KiB/conn before, 16.9 KiB/conn after, -11.9 KiB/conn). The shared helper is hysteresis-gated (correct for keep-alive: a busy connection must not shrink->regrow per request), and that hysteresis is exactly wrong at defer -- the stash was just grown, yet the connection now parks indefinitely. Age the anchor out before calling the same helper: one shrink->regrow pair per defer cycle instead of unbounded retention. RX-uring invariant holds (defer runs with no RECV SQE in flight, same as re-arm). Perf A/B within noise; sse/reactor tests pass. --- src/net/http/http.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/src/net/http/http.c b/src/net/http/http.c index 93acc477..80956329 100644 --- a/src/net/http/http.c +++ b/src/net/http/http.c @@ -1096,7 +1096,27 @@ static void http_async_dispatch(int fd, cwist_http_async_conn_t *conn, if (action == CWIST_ASYNC_DEFER) { /* The handler parked the request on a cwist_async; the completion - * path owns fd and conn now and re-arms or closes when done. */ + * path owns fd and conn now and re-arms or closes when done. + * + * A deferred connection used to hold its full rbuf/obuf stashes for + * the entire deferral (SSE, long-poll): the idle shrink only ran at + * re-arm, which a parked connection never reaches. Shrink toward the + * idle floor here instead, so a parked connection pays ~KB, not + * ~32 KB, for the whole deferral. + * + * Defer-specific shrink: the shared helper is keep-alive hysteresis- + * gated (it must be: a busy connection would otherwise + * shrink->regrow every request, a P99.999 stall source). That + * hysteresis is exactly wrong at defer -- the stash was grown moments + * ago, yet the connection is now parked indefinitely and the capacity + * is not needed until the completion re-arms (which regrows on + * demand). So age the anchor out before calling it; the resulting + * shrink->regrow pair costs one realloc per defer cycle, not per + * request. RX-uring invariant holds: defer runs with no RECV SQE in + * flight (rx_recv_inflight was cleared by the RECV completion entry, + * or the wait was a POLL), same as at re-arm. */ + conn->stash_grown_sec = 0; + http_async_stash_idle_shrink(conn); return; } if (action == CWIST_ASYNC_DETACH) {