Conversation
- client_builder.rs: always set builder.referer() so referer=False disables Referer on redirects - lib.rs/async/client.rs: remove #[pyo3(get,set)] on proxy field so custom get_proxy/set_proxy updates shared ProxyMatcher - hs.rs: fallback for SelectedUnofferedCipherSuite when server selects advertised-but-unsupported legacy CBC/RSA suites, preserving ja4/ja4_ro/akamai
… tls buffer h2 connector allows non-http proxy schemes, total timeout surfaces as socks connect error, tls bridge drains plaintext buffer.
Ensure Store is empty after clear_recv_buffer test so the unstable Drop assertion does not panic.
Parse /etc/hosts once via OnceLock<Arc<HostsMap>> with with_capacity and Box<str>+Arc<[IpAddr]> shared across clients. Disables hickory's per-client Hosts::from_system which rehashes 6→13→27→400 MB on 500k entries. Hosts now always on.
Pool::Drop previously aborted the driver even when active_streams>0, causing broken pipe DecodeError with many concurrent temp clients. Gate on active_streams==0; if >0 detach via detach_driver_until_idle until idle, or leak handle+Arc when no runtime (Python GC off-runtime) to avoid spawn panic. Map body errors via primp_body_error_to_pyerr checking is_timeout() before is_decode() with full source chain, so mid-body timeout becomes TimeoutError not DecodeError. Add pool_concurrent_drop tests (100 concurrent temp clients)
Replace std SipHash13 with foldhash::fast::RandomState via foldhash::HashMap/HashMapExt for H1/H2/H3 pools, DNS LRU/cache/hosts and client dns_overrides, and IndexMap<_,_,RandomState> for h2 store. 2× faster hash_one (17ms vs 107ms) and ~2.2× faster pool/LRU ops (256-entry H1/H2, 1024-entry DNS), IndexMap ~2×. Benchmark: size 256 std 615ms -> foldhash 278ms, LRU 4.77ms -> 3.52ms, client_creation unchanged (noise).
Parsing /etc/hosts with 470k entries blocks the current_thread runtime for 3.6s (1.2s counting + 1.9s inserts), exceeding the 200ms hanging-DNS deadline. hanging_dns_with_short_connect_timeout and explicit_dns_timeout then surface as connect timeout instead of DNS. Load hosts in a detached thread via HOSTS_INIT_TRIGGER and serve the first lookup from inner resolver; also estimate HashMap capacity from file size to avoid the counting pass (1.6s vs 3.6s). Fixes hanging_dns_with_short_connect_timeout_is_dns_error and explicit_dns_timeout_is_dns_error (were 3.4s, now 0.2s/0.1s).
aws-lc-sys 0.44.0 bcm.c fails on manylinux2014 (GCC 4.8.5) for powerpc64le-unknown-linux-gnu (exit 1, no diagnostic). manylinux 2_28 provides GCC 7.5 and succeeds for that arch. Keep other linux targets on auto, only ppc64le on 2_28.
…irects leak checks On Windows 127.0.0.1:1 with timeout=2 raises TimeoutError (operation timed out) instead of ConnectError (connection refused on Linux). Accept both to keep leak regression tests green on Windows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Major release
v2.0.0—main→dev—367 files +32k / -73k.Former wrapper around forked
reqwest/hyper/hyper-util/hyper-rustls/tokio-rustlsreplaced by a standalonecrates/primpbuilt directly onhyper,hyper-util,h2andrustls. The Python extension is rebound to the new native core.Breaking Changes / Migration
Workspace
primp-reqwest,primp-hyper,primp-hyper-util,primp-hyper-rustls,primp-tokio-rustlscrates/primpis now the product;Cargo.tomlworkspace.memberstrimmed toprimp,primp-rustls/rustls,primp-h2,primp-python[profile.bench]added (lto = false,codegen-units = 256)Dependencies
primp/Cargo.tomlno longer re-exportsreqwest/*; now declareshyper@1.8,hyper-util@0.1,tower@0.5+tower-http@0.7(follow-redirect,decompression-*),http-body/http-body-util,tokio,foldhash@0.2,rustls@0.23(impersonate,aws-lc-rs,ml-dsa),hickory-resolver@0.26optional,quinn/h3optionalprimp-rustls0.23.40→0.23.42,hashbrown0.16→0.17primp-h2tracks upstream0.4.17/0.4.18Features
default = [cookies, gzip, brotli, zstd, deflate, query, form, json, multipart, stream, system-proxy, charset]impersonate/default-tls/http2/rustls-no-provider(always on viarustls/h2)charset/hickory-dns/http3remainVersions
primp1.3.1→2.0.0primp-python1.3.1→2.0.01.89,panic = "abort"/lto = "fat"/codegen-units = 1unchangedPython
pyo3/pyo3-async-runtimes0.28→0.29html2text0.16→0.17jiter@0.16,log;crate-type = cdylibdoc disabledFeatures
Core
src/{lib, config, error, connect, tls, tls_bridge, impersonation}+async_impl/{client, body, request, response, multipart, negotiate, range_guard, h1_client, h2_client, h3_client}+dns/{cache, lru, hosts, resolve, plain, doh, dot, gai, hickory}+imp/{chrome, edge, opera, firefox, safari}+retry/redirect/proxy/cookiePool / ALPN
busy/idle_since,PlaceholderGuard, origin-form rewrite, lazyevict_stale(90s / 256)AtomicU64 last_used,active_streamsrefcount, owner-only abort +detach_driver_until_idle,watchsentinel dedup, busy-gate, single conn per key, REFUSED retry ≤32, backpressure 1 KiB gateclose_notify), DNS LRU 30s/5s +DynResolver::http_resolve+ChainedResolverImpersonation
chrome_144-152,edge_144-151,opera_126-135,firefox_140,146-151,safari_18.5/26/26.3/26.4+randomemulator_extension_orderper-version (GREASE0x6a6a/0x0a0a, ALPS0x44cd,compress_certificatebrotli-vs-zlib, ML-DSA150+/134+)SelectedUnofferedcipher fallbackDNS / Proxy / Cookie
OnceLockHostsMap(Box<str>+Arc<[IpAddr]>), non-blocking threaded load (3.6s → 0.2s)DNSTimeoutcapping; SOCKS primp-owned (bracket-strip, ATYP=4, 255 B caps, 8 KiB CONNECT,with_timeout)Proxy-Authorizationper-hop re-attachpsl@2.1) rejection, one-shot cookie merge,referer=FalsehonoredFixes / Performance
Dropno-abort whileactive_streams > 0→ fixesbroken pipeunder 100 concurrent temp clients; off-runtime GCmem::forgetleak,primp_body_error_to_pyerris_timeoutbeforeis_decode(pool_concurrent_drop)foldhash::fast::RandomStatefor H1/H2/H3/DNSHashMap+ h2IndexMap(~2×:17msvs107mshash,615ms → 278mspool 256-entry)StatusError.status_code/urlrustls@0.23.42/h2@0.4.17, header-abuse (>4× max_header_list_size→GOAWAY ENHANCE_YOUR_CALMvs431+RST) + flow-control hardening; clippy clean,manylinux_2_28forppc64le(GCC 7.5), WindowsTimeoutErrorfor127.0.0.1:1Python
primp-pythonto nativeprimp@2.0(Client/AsyncClient/RequestBuilder/Response)future_into_coroutine(14 sites,create_task-checked),ACTIVE_TASKS+SHUTTING_DOWN+call_soon_threadsafeinterpreter-exit safety,py.detach+block_on,get_runtimePyOnceLock::get_or_try_init→PyRuntimeErrorareadcache_drained,exhausted Arc<AtomicBool>iterators (TOCTOUstream_exhausted→ quiet EOF)chunk_size/timeoutvalidation,text_markdown(html2text),DNSTimeoutErrorPyOnceLockTests / Docs
h1_fallback,h2_pool_saturation,range_accept_encoding(RangeGuard→identity),cookie_psl,proxy_review,redirect,retry(tower×pool3×33=99),tls_close_notify,pool_concurrent_drop,socks_ipv6,window_increment,test_pool_concurrent_drop.pytest_client/request_params/streaming/exceptionsprimp.pyi/ examples updated;READMEprimp = "2"