feat: initial proxybench evidence-first benchmark utility

[pnlabs-dev]

Summary

Introduces the first public PN Labs implementation of proxybench: a zero-runtime-dependency, local-only utility for comparing sanitized proxy/web-retrieval outcomes by usable-result efficiency rather than proxy count.

Core metrics

• usable success rate + descriptive Wilson interval;
• requests per usable result;
• rotations per usable result with explicit coverage;
• p50 / p95 latency with explicit coverage;
• cost per usable result with explicit coverage;
• categorical outcome counts;
• directional A/B deltas;
• operator-defined acceptance gates.

Public data boundary

The event schema accepts only usable, latency_ms, cost_units, rotated, and outcome. Unknown fields are rejected. URLs, IPv4/IPv6 addresses, provider identifiers, credentials, cookies, headers, payloads, customer identifiers, and production infrastructure context are not required and are outside the public schema.

The CLI avoids intentionally echoing raw JSON lines, unsupported values, or local input paths in validation errors.

Product boundary

This repository is the transparent measurement baseline. Private routing/scoring implementation, provider selection, target × egress learning, promotion/demotion intelligence, private benchmark data, and production infrastructure remain outside OSS.

Security hardening

• no runtime network I/O or telemetry;
• no credential handling or provider login;
• unknown event fields rejected;
• synthetic public fixtures only;
• vendor-neutral hygiene gate checks secret/token shapes, credential-bearing URLs, IPv4/IPv6 literals, internal-hostname shapes, sensitive filenames, and emails;
• matched values are never printed;
• GitHub Actions permissions are contents: read;
• checkout credentials are not persisted;
• external actions are pinned to full release commit SHAs.

Clean history

The branch was rebuilt from the reviewed final tree as one commit directly on top of main.

• base: 80dbf8a97a0a6ad4360a1b0f7acba40f59fd14f3
• head: e992310afb67cb07c87d60a8e2d8a9f68d468281
• PR commits: 1
• changed files: 20

Final validation

Exact clean-history head e992310afb67cb07c87d60a8e2d8a9f68d468281:

• Python 3.10: PASS
• Python 3.11: PASS
• Python 3.12: PASS
• public repository hygiene gate: PASS
• package build/install from source: PASS
• compile check: PASS
• unit tests: 20/20 PASS
• installed CLI summarize smoke test: PASS
• installed CLI compare + gate smoke test: PASS

CI run: 32399449277.

Review posture

Metric semantics, missing-data behavior, privacy/data minimization, IP/secret hygiene, CI packaging, commercial boundary, and active branch history have been reviewed. Descriptive intervals and operational gates are explicitly not presented as causal proof.

[pnlabs-dev]

PN Labs final review — merge-ready

Second-pass review is complete on the clean one-commit branch. No remaining source-level blocker found.

Verified boundaries:

• measurement only; no network/proxy orchestration;
• unknown input fields rejected;
• no raw-line/path echo in handled validation errors;
• cost/rotation ratios fail closed under partial coverage;
• descriptive intervals and gates do not claim causality;
• proprietary routing/scoring/provider logic remains excluded;
• public hygiene gate covers credentials, credential-bearing URLs, IP literals, internal hostnames, sensitive filenames, and emails without echoing matches;
• package install, 20 tests, and installed CLI smoke checks pass across Python 3.10/3.11/3.12.

Active PR history is one commit directly on top of main. Recommendation: publish via squash/merge once ready.