diff --git a/.gitignore b/.gitignore index c66870f..1a5a072 100644 --- a/.gitignore +++ b/.gitignore @@ -21,3 +21,8 @@ dev_roadmap.md /hotmem/manifest.json /data/hotmem/manifest.json planned-evolution.md + +# native helper spike (#48) — generated corpus and build artifacts +bench/native_spike/corpus/ +bench/native_spike/**/*.so +bench/native_spike/**/*.o diff --git a/bench/native_spike/README.md b/bench/native_spike/README.md new file mode 100644 index 0000000..8fc996a --- /dev/null +++ b/bench/native_spike/README.md @@ -0,0 +1,316 @@ +# Native Helper Spike (#48) — Rust, C, or WebAssembly fast-path primitives + +Status: **complete** · Profile: `reduced` · Runs: 5/arm (median + p95) · Corpus seed: 48 + +## TL;DR recommendation + +**Do not ship a native helper yet.** Neither measured candidate clears the +issue's gate (≥3x on the 100MB+ checksum path AND clean single-wheel packaging +AND graceful fallback): + +- **Checksum path (B1/B4):** both C arms are **~2.2x slower** than the current + Python path. CPython's `hashlib` is already OpenSSL-backed C (AVX2, no SHA-NI + on this host); a self-contained C or ctypes helper cannot beat it, and a + Rust/PyO3 helper would bind to the same OpenSSL. The real inefficiency in the + current path is algorithmic — verified hydration **reads every byte range + twice** — and pure-Python fixes deliver more than any native arm: + single-read = +16%, streaming = **1.26x faster + 10x less peak memory** + (219 MB → 20 MB on a 100 MB range). +- **JSONL scanning path (B2):** the C scanner with full JSON validation is + **3.55x faster** than the real `JSONLInspector._stream` on 100 MB (48x for + scan-only) — it clears the 3x *number*, but its packaging story (ctypes + + per-platform `.so`) is the fragile option in the matrix, and the *portable* + native option (WASM) shows **no gain** (1.04x — per-byte host↔WASM boundary + cost erases it). A pure-Python policy change (stop validating every line with + `json.loads`) is worth ~5x on its own. +- **Bundle parse (B3):** dominated by `embed_text` (md5 per character trigram) + at 37–78% of parse time. The lever is algorithmic, not native. + +Follow-ups (pure Python, no new dependencies) are listed below. Revisit native +helpers when HotMem handles GB-scale file-backed memories in production, or if +JSONL inspection becomes hot enough to justify a Rust wheel spike. + +## Environment + +| | | +|---|---| +| CPU | Intel Core i7-10610U @ 1.80GHz (4C/8T, AVX2, **no SHA-NI**) | +| Kernel | 6.18.33.1-microsoft-standard-WSL2 | +| Python | 3.12.3 (`venv/`, hotmem 0.2.1 editable) — OpenSSL 3.0.13 for hashlib | +| C | gcc/cc 13.3.0, `-O2 -Wall -Wextra -std=c11` | +| WASM | wasmtime 48.0.0 (pip, bench-only) | +| Box | shared dev machine; loadavg 3–6 during the run (per-cell context in results.json) | + +Environment quirks discovered and worked around (documented so numbers can be +interpreted honestly): + +- `resource.ru_maxrss` is **broken on this kernel** (reported ~11x the true + high-water mark; `/proc/self/status` VmHWM is correct). All RSS numbers here + are `/proc`-sourced. +- `random.Random.randbytes` is pathologically slow on this box (~60 MB/s with + RSS anomalies). Corpus bytes use SHAKE-256 counter-mode instead (~116 MB/s, + constant RSS). + +## Methodology + +- **Real HotMem paths, not synthetic throughput.** The Python baselines call + the actual production code: `LocalFilesystemAdapter.read_range` + + `provenance.verify_range` (the exact double-read sequence + `memory.hydrate_memory_detailed` performs), `JSONLInspector._stream` / + `.inspect()` (with its production LRU checksum cache cleared per run), and + `bundle.parse_bundle`. +- **Corpus** (`gen_corpus.py`, deterministic, seed 48, ~384 MB): binary files + (1 KB → 100 MB), JSONL files (10/50/100 MB) shaped exactly like real + `swap.jsonl` snapshot records (30 fields, ~1.2 KB/row) with one malformed + line inserted at a known offset (~60% into each file), and loose bundle trees + (100/1000/2500 dirs). Regeneration is byte-identical (verified); the manifest + (`manifest.json`, committed) records sizes, SHA-256 digests, row counts and + bad-line offsets. +- **Arms** run serially, each (bench, arm, input, cache-mode) cell in a fresh + subprocess so `/proc` VmHWM peaks are per-cell. 5 timed runs per cell + (median + p95 reported); warm mode adds one uncounted warm-up run. +- **Cold cache** via `posix_fadvise(DONTNEED)` on the input file before each + cold run (unprivileged; not a full drop-caches, so cold numbers are a lower + bound on true cold cost). +- **Memory guards:** each cell checks `MemAvailable` (guard = working-set + estimate + 250 MB) and records `skipped: insufficient_memory` rather than + thrashing. No cell skipped in this run. +- **Correctness parity is asserted, not assumed.** B1: every arm's digest must + equal a plain-`hashlib` reference. B2: all arms must agree on row count; + validating arms on the first-bad-line index; scanning arms on sampled line + boundaries. B4: both arms must verify the identical manifest file set. The + harness fails loudly on any mismatch. All parity checks passed (see tables). +- **Optional-boundary check:** with `HOTMEM_SPIKE_DISABLE_NATIVE=1`, C and WASM + arms return `skipped: helper_unavailable` instead of crashing — the same + graceful-degradation contract a production helper would need. Passed. + +Reproduce: + +```bash +cd bench/native_spike +python gen_corpus.py --profile reduced # ~384 MB into corpus/ (gitignored) +make -C c_checksum && make -C c_jsonl # or let the harness build them +python run_bench.py --profile reduced # full matrix → results.json (~11 min) +python run_bench.py --quick # smoke: 1 run, small inputs +python run_bench.py --report # re-render markdown from results.json +``` + +The `full` profile adds a 200 MB checksum arm (auto-gated on memory) for a +quieter box. `results.json` (committed) is the source of truth; tables below +are generated from it. + +## B1 — Range SHA-256 checksum (real verify_range path) + +Median ms, warm / cold. Peak warm RSS MB in parens. `+N@off` = byte range +[offset, offset+N). + +| file (range) | py double-read¹ | py single | py streaming | c pread | c mmap | py single gains | c mmap vs py single | +|---|---|---|---|---|---|---|---| +| 1 KB @0 | 0.3 / 0.6 (20) | 0.1 / 0.5 (19) | 0.0 / 0.4 (18) | 0.0 / 3.7 (18) | 0.0 / 7.2 (18) | 2.91x | 0.47x | +| 100 KB @0 | 0.7 / 1.7 (20) | 0.5 / 1.3 (19) | 0.8 / 1.6 (18) | 1.3 / 1.9 (18) | 1.1 / 1.8 (18) | 1.39x | 2.09x | +| 1 MB @0 | 10.0 / 13.4 (22) | 6.1 / 7.1 (20) | 5.9 / 7.9 (19) | 12.2 / 14.4 (19) | 13.2 / 15.0 (19) | 1.65x | 2.18x | +| 10 MB @0 | 93.9 / 90.5 (40) | 60.9 / 95.8 (29) | 88.4 / 110.9 (20) | 145.1 / 159.8 (19) | 131.4 / 146.0 (28) | 1.54x | 2.16x | +| 2 MB @5243003 | 23.0 / 25.2 (25) | 16.1 / 19.5 (21) | 17.6 / 21.9 (20) | 36.6 / 47.2 (19) | 31.2 / 43.7 (21) | 1.43x | 1.94x | +| **100 MB @0** | 850 / 986 (219) | 736 / 771 (118) | **673 / 835 (20)** | 1744 / 1716 (19) | 1625 / 1783 (118) | 1.16x | 2.21x | +| 25 MB @52428923 | 220 / 236 (70) | 157 / 232 (44) | 147 / 180 (20) | 356 / 383 (19) | 395 / 437 (43) | 1.40x | 2.51x | + +¹ `py_current_double_read` reproduces today's verified hydration exactly: +`memory.py:244` reads the range, then `provenance.verify_range` +(`provenance.py:103`) reads it **again** and hashes it. Both reads hit the +page cache, so the second read is cheap — the extra cost is ~15% at 100 MB, +not the ~2x a naive double-I/O model predicts. + +Parity: all arms produce identical digests on every case. **PASS** + +Findings: + +- **Native loses on hashing.** C pread/mmap are 2.2–2.5x *slower* than + Python's single-read: `hashlib.sha256` is OpenSSL C with AVX2; the spike's + self-contained C SHA-256 cannot match it (and linking OpenSSL from a helper + would add packaging fragility for zero gain). On a SHA-NI host OpenSSL gets + faster too, so this conclusion is stable. +- **The real win is algorithmic.** `py_single_read` (+16% at 100 MB) removes + the redundant read; `py_streaming` is the best arm overall (1.26x vs current + path, 673 ms median) and holds peak RSS at **20 MB regardless of range size** + vs 219 MB for the current double-read path. +- **mmap is not free.** The C mmap arm faults in the whole range (RSS 118 MB on + the 100 MB range) — it saves the copy but not the memory. + +## B2 — JSONL scanning (real JSONLInspector path) + +Median ms, warm / cold. `py_stream_real` = production `_stream` (count + full +per-line `json.loads` validation); `py_inspect_full` = production +`inspect(count_rows=True)` end-to-end (adds whole-file checksum + column +inference); scan-only arms count rows/sample boundaries without validation. + +| file | py real _stream | py inspect() e2e | py scan-only | c scan+valid | c scan-only | wasm scan | rows | +|---|---|---|---|---|---|---|---| +| 10 MB | 122 / 123 | 179 / 181 | 23 / 32 | 34 / 40 | 2.7 / 17 | 113 / 101 | 8,894 | +| 50 MB | 1185 / 792 | 886 / 971 | 195 / 147 | 262 / 243 | 22 / 59 | 672 / 667 | 44,464 | +| **100 MB** | 1740 / 2259 | 2119 / 2695 | 319 / 395 | **490 / 567** | 36 / 169 | 1667 / 1119 | 88,914 | + +Parity: rows agree across all six arms; first-bad-line index agrees across the +three validating arms; sampled line boundaries agree across the three scanning +arms. **PASS** + +Findings: + +- **Validation dominates the Python path.** Dropping per-line `json.loads` + (py scan-only, 319 ms) is worth 5.5x at 100 MB. The scan loop itself (1 MiB + chunks, `bytes.find`) is already efficient Python. +- **C scan+valid = 3.55x** over the real `_stream` — clears the 3x number — + and C scan-only is 48x. The C validator is strict RFC 8259 (parity with + `json.loads` verified down to escape sequences, numbers, nesting, and the + first-bad-line offsets; the one accepted divergence is `NaN/Infinity`, + which Python tolerates and the spike's corpus/validator do not use). +- **WASM shows no gain** (1.04x): the spike's WAT scanner runs the same + per-byte loop in Cranelift-compiled code, but every byte crosses the + host↔linear-memory boundary through `Memory.write`. This is the honest cost + of the "host orchestrates I/O, module does CPU work" shape at byte + granularity; a WASM helper would need the *whole* scan hot loop plus I/O + batching to compete, at which point it is a second runtime (~5 MB dep) for + what C achieves or Python nearly achieves with a policy change. +- Cold-mode anomalies (e.g. 50 MB `_stream` cold < warm) are shared-box noise; + per-cell loadavg is recorded in results.json. + +## B3 — Bundle parse profile (measurement only, no native arm) + +Median ms, warm. `parse_e2e` = production `parse_bundle()` over the tree; +`embed_only` = the `embed_text()` share of that work (same texts, isolated). + +| tree | parse e2e | embed only | embed share | +|---|---|---|---| +| bundles_100 | 1,093 | 856 | 78% | +| bundles_1000 | 20,038 | 9,908 | 49% | +| bundles_2500 | 37,524 | 14,015 | 37% | + +Findings: + +- **`embed_text` is the bundle bottleneck** (37–78% of parse time). It hashes + every character trigram with a separate `hashlib.md5` call + (`embed.py:44`). Markdown/YAML/JSON parsing — the surface a "WASM markdown + parser" would target — is a small fraction of the remainder. A native parser + would optimize the wrong 20%. +- Per-bundle costs are noisy on this loaded box (10–20 ms/bundle, + super-linear at 1000+ bundles — GC pressure in the record-heavy parse); + ratios, not absolute times, are the signal. +- The fix is algorithmic: batch the trigram hashing (one `hashlib` update over + a packed buffer, or `zlib.crc32`-style rolling hash), or vectorize. That is + a `embed.py` follow-up, not a native helper. + +## B4 — Manifest verification (real whole-file checksum path) + +Median ms, warm / cold. Both arms verify the identical 8-file manifest set +(~295 MB total): Python uses the exact 64 KiB chunk loop from +`storage/local.py:_checksum`; C uses pread. + +| arm | warm | cold | files verified | +|---|---|---|---| +| py_hashlib | 1,966 | 3,497 | 8/8 | +| c_pread | 3,278 | 4,520 | 8/8 | + +Parity: identical file sets verified. **PASS** + +Confirms B1 at workload scale: the C helper is 1.7x *slower* than OpenSSL-backed +`hashlib` chunked streaming — the current Python checksum implementation is +already near the platform's hash ceiling (~0.15 GB/s without SHA-NI). + +## Discovered: `_stream` line-offset bug (documented, not fixed here) + +Parity testing surfaced a real production bug: when a line spans the 1 MiB +read-chunk boundary, `JSONLInspector._stream` computes +`line_start = offset + pos` in `(carry+chunk)` coordinates +(`src/hotmem/inspectors/jsonl_inspector.py:129`), overstating file offsets by +`len(carry)` for every subsequent line. + +Evidence on `events_10mb.jsonl`: the generator placed the malformed line at +byte 6,292,320 (manifest-committed; confirmed by reading the file). The real +`_stream` reports offset 6,292,650 — off by +330 (the carry length when that +line's chunk boundary was crossed). The C scanner reports 6,292,320 exactly. +Affected outputs: `unsupported_reason` offsets and, when chunk-spanning occurs +within the first `sample_size` lines, `byte_ranges` (both feed +`FileInspection` → API/MCP/CLI). Out of spike scope to fix; recommended +follow-up: `line_start = base + pos` where `base = offset - len(carry)` +(the replica's arithmetic, which achieves full parity with the C scanner). + +## Packaging matrix + +| Option | Dependency | Install | Portability | Fallback | Maintenance | Verdict | +|---|---|---|---|---|---|---| +| C via ctypes (this spike) | bundled `.so` per platform | build-from-source or ship N binaries | fragile: ABI/glibc coupling, needs per-OS build matrix | Python path (works) | highest | ❌ measured 2.2x slower on checksum; only wins on JSONL scan where the win is real but packaging is worst | +| WASM via wasmtime | `wasmtime` wheel (~5 MB, pure wheels for all majors) | `pip install hotmem[wasm]` | excellent: CPU/OS-agnostic | Python path (works) | low, but adds a second runtime | ❌ measured no gain (1.04x) at byte granularity | +| Rust via PyO3 (analysis only — not built here: no toolchain on box) | `hotmem-native` wheel via maturin | optional extra; manylinux/macOS/Windows wheels via CI | good | Python path (works) | medium-high: Rust toolchain + 3-OS × N-Python CI matrix | ⚠️ plausible carrier for the C scanner's 3.55x JSONL win *if* that path ever justifies a compiled dep; would bind to the same OpenSSL for hashing, so no checksum win | +| None (pure Python fixes) | — | — | — | — | — | ✅ recommended: single-read verify, streaming hash, validation policy, embed batching | + +Optional boundary (all options): helpers must stay behind an extra/import +guard, fall back to the Python path when absent, and never alter public API +defaults — demonstrated working here via `HOTMEM_SPIKE_DISABLE_NATIVE=1`. + +## Recommendation vs the gate + +Gate: **≥3x on the 100MB+ checksum path AND clean single-wheel packaging AND +graceful fallback.** + +| Criterion | C | WASM | Outcome | +|---|---|---|---| +| ≥3x on 100MB+ checksum | 0.45x (2.2x slower) | n/a (not a checksum candidate) | ❌ fail | +| Clean single-wheel packaging | per-platform `.so` | yes, but ~5 MB second runtime | ❌ / marginal | +| Graceful fallback | yes (demonstrated) | yes (demonstrated) | ✅ | + +**Recommendation: no native helper yet.** Documented bottlenecks and revisit +triggers below. + +## Follow-ups recommended (all pure Python, zero new dependencies) + +1. **Single-read verify in the hydrate path** — `memory.hydrate_memory_detailed` + reads the range, then `verify_range` re-reads it; hash the already-read + bytes (or add a streaming verify API). Expected: +16% on verified hydration + of large ranges, and removes a redundant range copy. +2. **Streaming range hash for large ranges** — chunked seek+hash (the + `py_streaming` arm): 1.26x vs current path at 100 MB and peak RSS 20 MB vs + 219 MB. Natural home: `provenance.verify_range` / adapter layer, switching + on range size. +3. **Fix the `_stream` offset bug** (see above) — correctness, one line of + arithmetic, plus a regression test with a chunk-boundary-spanning fixture + (the spike's fixtures are reusable). +4. **JSONL validation policy** — `_stream` pays `json.loads` per line (5.5x of + the scan cost). Options: validate only sampled lines, or make full + validation opt-in. Semantics decision for #53 — the data says the current + default is expensive at 100 MB scale. +5. **`embed_text` batching** (if bundle-load time matters) — 37–78% of + `parse_bundle`; batch the per-trigram md5 calls or switch to a cheaper + rolling hash. Keep `hotmem-hash-v1` semantics compatible or version the + embedding model string. + +## Revisit triggers for native helpers + +- GB-scale file-backed memories in production where 0.15–2 GB/s hashing becomes + a wall (note: SHA-NI hosts raise the OpenSSL ceiling too). +- JSONL/inspection becoming hot enough that a Rust scanner wheel (maturin, + optional extra) is justified by the measured 3.5x+ — start from this spike's + `scan.c` semantics, which already have a verified parity contract. +- Any Parquet/Arrow *metadata* scanning need — remains out of scope for core + (HotMem references and inspects files; it does not become a query engine). + +## Repository layout / what ships + +``` +bench/native_spike/ # spike only — NOT part of the hotmem package or test suite + gen_corpus.py # deterministic corpus generator (seed 48) + manifest.json # committed: sizes, SHA-256, row counts, bad-line offsets + py_baseline.py # real-path arms + pure-Python candidates + scan replica + c_checksum/ # sha256.{c,h} + range_hash.c + Makefile (self-contained) + c_jsonl/ # scan.{c,h} (line scan + strict JSON validator) + Makefile + wasm_parser/scanner.wat # WAT scanner, compiled at runtime by wasmtime + benchlib.py # loaders (graceful-unavailable), /proc RSS, fadvise, guards + bench_worker.py # per-cell subprocess worker (timing + self-verification) + run_bench.py # orchestrator + parity assertions + markdown report + results.json # committed: full run output (source of truth) + corpus/ # gitignored (~384 MB generated) + *.so # gitignored (built on demand) +``` + +Nothing under `src/`, `tests/`, or `pyproject.toml` was modified. `wasmtime` +is a bench-only venv dependency, not a project dependency. The Python/FastAPI/ +SQLite path remains the default, dependency-free path; native code stays out of +the runtime. diff --git a/bench/native_spike/bench_worker.py b/bench/native_spike/bench_worker.py new file mode 100644 index 0000000..a525181 --- /dev/null +++ b/bench/native_spike/bench_worker.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +"""Subprocess-isolated benchmark worker for the native helper spike (#48). + +One worker process = one (bench, arm, input, cache-mode) cell. It runs the +arm --runs times (plus one uncounted warm-up in warm mode), measures wall +time per run and /proc-based RSS, self-verifies output determinism and +parity, and emits a single JSON object on stdout. Launched by run_bench.py. + +Why subprocess isolation: /proc VmHWM is a process-lifetime high-water +mark, so per-cell RSS peaks are only meaningful in a fresh process +(resource.ru_maxrss is broken on this box; see benchlib docstring). +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +import time +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +import benchlib # noqa: E402 +import py_baseline as pb # noqa: E402 + + +def _measure(fn, *, runs: int, cache: str, cold_paths: list[str], guard_mb: int, summarize): + """Run `fn` `runs` times under the memory guard; return the cell result.""" + avail = benchlib.mem_available_mb() + if avail is not None and avail < guard_mb: + return { + "skipped": "insufficient_memory", + "mem_available_mb": avail, + "guard_mb": guard_mb, + } + + rss_baseline = benchlib.vm_rss_kb() + hwm_reset = benchlib.try_reset_hwm() + + if cache == "warm": + fn() # uncounted warm-up (also validates the arm before timing) + + times = [] + summaries = [] + for _ in range(runs): + if cache == "cold": + for p in cold_paths: + benchlib.fadvise_dontneed(p) + t0 = time.perf_counter() + out = fn() + times.append(time.perf_counter() - t0) + summaries.append(summarize(out)) + + if any(s != summaries[0] for s in summaries[1:]): + return {"error": "nondeterministic_output", "outputs": summaries[:3]} + + return { + **benchlib.stats_ms(times), + "rss_baseline_kb": rss_baseline, + "rss_peak_kb": benchlib.vm_hwm_kb(), + "hwm_reset": hwm_reset, + "output": summaries[0], + "mem_available_mb": avail, + } + + +_BAD_RE = re.compile(r"line (\d+) \(offset (\d+)\)") + + +def _parse_bad(reason: str | None) -> dict | None: + if not reason: + return None + m = _BAD_RE.search(reason) + return {"index": int(m.group(1)), "offset": int(m.group(2))} if m else {"index": None} + + +def cmd_b1(args): + path, offset, length = args.path, args.offset, args.length + reference = pb.reference_range_digest(path, offset, length) + + if args.arm == "py_current_double_read": + fn = lambda: pb.py_current_double_read(path, offset, length, reference) # noqa: E731 + elif args.arm == "py_single_read": + fn = lambda: pb.py_single_read(path, offset, length) # noqa: E731 + elif args.arm == "py_streaming": + fn = lambda: pb.py_streaming(path, offset, length) # noqa: E731 + elif args.arm in ("c_pread", "c_mmap"): + try: + pread, mm = benchlib.c_checksum() + except benchlib.NativeHelperUnavailable as err: + return {"skipped": "helper_unavailable", "detail": str(err)} + call = pread if args.arm == "c_pread" else mm + fn = lambda: call(path, offset, length) # noqa: E731 + else: + return {"error": f"unknown b1 arm: {args.arm}"} + + def summarize(digest: str) -> dict: + if digest != reference: + raise AssertionError(f"parity fail: {digest} != {reference}") + return {"digest": digest[:12] + "…"} + + try: + # Peak live bytes for the double-read path is ~2x range; guard covers + # that plus headroom so cells skip (recorded) instead of thrashing. + return _measure( + fn, + runs=args.runs, + cache=args.cache, + cold_paths=[path], + guard_mb=2 * (length >> 20) + 250, + summarize=summarize, + ) + except AssertionError as err: + return {"error": "parity", "detail": str(err)} + + +def cmd_b2(args): + path = args.path + + if args.arm == "py_stream_real": + fn = lambda: pb.py_stream_real(path) # noqa: E731 + + def summarize(r): + bad = r["first_bad"] + return { + "rows": r["row_count"], + "first_bad_index": bad["index"] if bad else None, + "first_n": [list(t) for t in (r["byte_ranges"] or [])], + } + + elif args.arm == "py_inspect_full": + from hotmem.inspectors.jsonl_inspector import JSONLInspector + from hotmem.storage.local import LocalFilesystemAdapter, _checksum + + adapter = LocalFilesystemAdapter() + meta = adapter.metadata(path) + insp = JSONLInspector() + + def fn(): + # Production LRU-caches whole-file checksums per path; clear it so + # every run pays the honest cold full-inspect cost. + _checksum.cache_clear() + return insp.inspect(path, adapter, meta, count_rows=True) + + def summarize(fi): + bad = _parse_bad(fi.unsupported_reason) + return { + "rows": fi.row_count, + "first_bad_index": bad["index"] if bad else None, + "first_n": [list(t) for t in (fi.byte_ranges or [])], + "checksum": fi.checksum[:12] + "…", + } + + elif args.arm == "py_scan_only": + fn = lambda: pb.py_scan_only(path) # noqa: E731 + + def summarize(r): + return { + "rows": r["rows"], + "first_bad_index": None, + "first_n": [list(t) for t in r["first_n"]], + } + + elif args.arm == "c_scan_full": + fn = lambda: benchlib.c_scan(path, validate=True) # noqa: E731 + + def summarize(r): + return { + "rows": r["rows"], + "first_bad_index": r["first_bad"]["index"] if r["first_bad"] else None, + "first_n": [list(t) for t in r["first_n"]], + } + + elif args.arm == "c_scan_only": + fn = lambda: benchlib.c_scan(path, validate=False) # noqa: E731 + + def summarize(r): + return { + "rows": r["rows"], + "first_bad_index": None, + "first_n": [list(t) for t in r["first_n"]], + } + + elif args.arm == "wasm_scan": + try: + scanner = benchlib.wasm_scanner() + except benchlib.NativeHelperUnavailable as err: + return {"skipped": "helper_unavailable", "detail": str(err)} + fn = lambda: scanner.scan_file(path) # noqa: E731 + + def summarize(r): + return { + "rows": r["rows"], + "first_bad_index": None, + "first_n": [list(t) for t in r["first_n"]], + } + + else: + return {"error": f"unknown b2 arm: {args.arm}"} + + try: + return _measure( + fn, + runs=args.runs, + cache=args.cache, + cold_paths=[path], + guard_mb=150, + summarize=summarize, + ) + except benchlib.NativeHelperUnavailable as err: + return {"skipped": "helper_unavailable", "detail": str(err)} + + +def cmd_b3(args): + tree = Path(args.path) + dirs = sorted(d for d in tree.iterdir() if d.is_dir()) + + if args.arm == "parse_e2e": + from hotmem.bundle import parse_bundle + + def fn(): + total = 0 + warns = 0 + for d in dirs: + records, warnings = parse_bundle(d) + total += len(records) + warns += len(warnings) + return (total, warns) + + def summarize(r): + return {"records": r[0], "warnings": r[1], "bundles": len(dirs)} + + elif args.arm == "embed_only": + # Gather the same fact texts parse_bundle would embed (memory body, + # facts.json items, events.jsonl lines) — uncounted one-time setup. + texts: list[str] = [] + for d in dirs: + texts.append((d / "memory.md").read_text(encoding="utf-8")) + for fact in json.loads((d / "facts.json").read_text(encoding="utf-8")): + texts.append(fact["fact"]) + with open(d / "events.jsonl", encoding="utf-8") as f: + for line in f: + line = line.strip() + if line: + texts.append(json.loads(line)["event"]) + + from hotmem.embed import embed_text + + def fn(): + return sum(len(embed_text(t)) for t in texts) + + def summarize(r): + return {"vectors": r, "texts": len(texts)} + + else: + return {"error": f"unknown b3 arm: {args.arm}"} + + return _measure( + fn, + runs=args.runs, + cache=args.cache, + cold_paths=[], + guard_mb=150, + summarize=summarize, + ) + + +def cmd_b4(args): + manifest = json.loads(benchlib.MANIFEST_PATH.read_text(encoding="utf-8")) + entries = [ + (e["name"], e["size"], e["sha256"]) + for e in manifest["checksum_files"] + manifest["jsonl_files"] + ] + paths = [str(benchlib.CORPUS_DIR / name) for name, _, _ in entries] + + if args.arm == "py_hashlib": + import hashlib + + def fn(): + ok = 0 + for (name, _size, want), path in zip(entries, paths, strict=True): + h = hashlib.sha256() + with open(path, "rb") as f: + # Same 64 KiB chunk loop as storage/local.py _checksum. + for chunk in iter(lambda: f.read(65536), b""): + h.update(chunk) + if h.hexdigest() != want: + raise AssertionError(f"manifest mismatch: {name}") + ok += 1 + return ok + + elif args.arm == "c_pread": + try: + pread, _ = benchlib.c_checksum() + except benchlib.NativeHelperUnavailable as err: + return {"skipped": "helper_unavailable", "detail": str(err)} + + def fn(): + ok = 0 + for (name, size, want), path in zip(entries, paths, strict=True): + if pread(path, 0, size) != want: + raise AssertionError(f"manifest mismatch: {name}") + ok += 1 + return ok + + else: + return {"error": f"unknown b4 arm: {args.arm}"} + + def summarize(r): + return {"files_verified": r} + + try: + return _measure( + fn, + runs=args.runs, + cache=args.cache, + cold_paths=paths, + guard_mb=150, + summarize=summarize, + ) + except AssertionError as err: + return {"error": "parity", "detail": str(err)} + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--bench", required=True, choices=["b1", "b2", "b3", "b4"]) + ap.add_argument("--arm", default="") + ap.add_argument("--path", default="") + ap.add_argument("--offset", type=int, default=0) + ap.add_argument("--length", type=int, default=0) + ap.add_argument("--runs", type=int, default=5) + ap.add_argument("--cache", choices=["warm", "cold"], default="warm") + args = ap.parse_args() + + dispatch = {"b1": cmd_b1, "b2": cmd_b2, "b3": cmd_b3, "b4": cmd_b4} + result = dispatch[args.bench](args) + result.update( + bench=args.bench, + arm=args.arm, + path=args.path, + offset=args.offset, + length=args.length, + cache=args.cache, + loadavg=benchlib.loadavg(), + ) + print(json.dumps(result)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/bench/native_spike/benchlib.py b/bench/native_spike/benchlib.py new file mode 100644 index 0000000..1e423c2 --- /dev/null +++ b/bench/native_spike/benchlib.py @@ -0,0 +1,342 @@ +"""Shared helpers for the native helper spike (#48) harness. + +Purpose: + - Load native candidate helpers (C .so via ctypes, WASM via wasmtime) + with a single "optional boundary": every loader raises + NativeHelperUnavailable when the helper is missing, fails to build, + or is disabled via HOTMEM_SPIKE_DISABLE_NATIVE=1. This mirrors the + graceful-degradation contract any production helper must satisfy. + - Memory and RSS measurement via /proc (resource.ru_maxrss is broken + on this kernel — it reported ~11x the true VmHWM; see README). + - Unprivileged cold-cache eviction via posix_fadvise(DONTNEED). + +Deps: stdlib only for Python arms; ctypes + wasmtime are lazy imports. +""" + +from __future__ import annotations + +import ctypes +import os +import platform +import subprocess +import time +from pathlib import Path + +SPIKE_DIR = Path(__file__).resolve().parent +CORPUS_DIR = SPIKE_DIR / "corpus" +MANIFEST_PATH = SPIKE_DIR / "manifest.json" + +READ_CHUNK = 1 << 20 +NATIVE_DISABLED = os.environ.get("HOTMEM_SPIKE_DISABLE_NATIVE", "") == "1" + + +class NativeHelperUnavailable(RuntimeError): + """A native helper cannot be used; callers must fall back to Python.""" + + +def _build(subdir: str) -> None: + cmd = ["make", "-s", "-C", str(SPIKE_DIR / subdir)] + res = subprocess.run(cmd, capture_output=True, text=True) + if res.returncode != 0: + raise NativeHelperUnavailable(f"make -C {subdir} failed: {res.stderr.strip()}") + + +def _load_so(subdir: str, libfile: str) -> ctypes.CDLL: + if NATIVE_DISABLED: + raise NativeHelperUnavailable("native helpers disabled (HOTMEM_SPIKE_DISABLE_NATIVE=1)") + so = SPIKE_DIR / subdir / libfile + if not so.exists(): + _build(subdir) + if not so.exists(): + raise NativeHelperUnavailable(f"{so} missing after build") + try: + return ctypes.CDLL(str(so)) + except OSError as err: + raise NativeHelperUnavailable(f"cannot load {so}: {err}") from err + + +_checksum_lib: ctypes.CDLL | None = None + + +def c_checksum(): + """Range-hash helper: range_hash_pread / range_hash_mmap(path, off, len) -> hex. + + Returns (lib, call_pread, call_mmap) where the callables take + (path, offset, length) and return the 64-char hex digest, raising + NativeHelperUnavailable-style errors as ValueError with the C code. + """ + global _checksum_lib + if _checksum_lib is None: + lib = _load_so("c_checksum", "librange_hash.so") + for fn in ("range_hash_pread", "range_hash_mmap"): + getattr(lib, fn).argtypes = [ + ctypes.c_char_p, + ctypes.c_uint64, + ctypes.c_uint64, + ctypes.c_char_p, + ] + getattr(lib, fn).restype = ctypes.c_int + _checksum_lib = lib + lib = _checksum_lib + buf = ctypes.create_string_buffer(65) + + def _call(fn: str, path: str, offset: int, length: int) -> str: + rc = getattr(lib, fn)(path.encode(), offset, length, buf) + if rc == -1: + raise FileNotFoundError(path) + if rc == -2: + raise EOFError(f"truncated range in {path}") + if rc != 0: + raise OSError(f"{fn} failed with code {rc} on {path}") + return buf.value.decode() + + return ( + lambda p, o, n: _call("range_hash_pread", p, o, n), + lambda p, o, n: _call("range_hash_mmap", p, o, n), + ) + + +class _ScanResult(ctypes.Structure): + _fields_ = [ + ("rows", ctypes.c_uint64), + ("first_bad_index", ctypes.c_int64), + ("first_bad_offset", ctypes.c_uint64), + ("lines", ctypes.c_uint64), + ("sample_count", ctypes.c_uint32), + ("_pad", ctypes.c_uint32), + ("sample_offsets", ctypes.c_uint64 * 16), + ("sample_lengths", ctypes.c_uint64 * 16), + ] + + +_scan_lib: ctypes.CDLL | None = None + + +def c_scan(path: str, *, sample_max: int = 5, validate: bool) -> dict: + """C JSONL scanner: mode 0 = scan-only, mode 1 = scan + JSON validation.""" + global _scan_lib + if _scan_lib is None: + lib = _load_so("c_jsonl", "libscan.so") + lib.scan_file.argtypes = [ + ctypes.c_char_p, + ctypes.c_uint32, + ctypes.c_int, + ctypes.POINTER(_ScanResult), + ] + lib.scan_file.restype = ctypes.c_int + _scan_lib = lib + res = _ScanResult() + rc = _scan_lib.scan_file(path.encode(), sample_max, 1 if validate else 0, ctypes.byref(res)) + if rc != 0: + raise OSError(f"scan_file failed with code {rc} on {path}") + return { + "rows": res.rows, + "lines": res.lines, + "first_bad": None + if res.first_bad_index < 0 + else {"index": res.first_bad_index, "offset": res.first_bad_offset}, + "first_n": [ + (res.sample_offsets[i], res.sample_lengths[i]) for i in range(res.sample_count) + ], + } + + +_wasm_scanner = None + + +def wasm_scanner(): + """Lazily compiled WAT scanner (wasmtime). Raises NativeHelperUnavailable.""" + global _wasm_scanner + if _wasm_scanner is not None: + return _wasm_scanner + if NATIVE_DISABLED: + raise NativeHelperUnavailable("native helpers disabled (HOTMEM_SPIKE_DISABLE_NATIVE=1)") + try: + import wasmtime + except ImportError as err: + raise NativeHelperUnavailable(f"wasmtime not installed: {err}") from err + + wat = (SPIKE_DIR / "wasm_parser" / "scanner.wat").read_text() + try: + engine = wasmtime.Engine() + module = wasmtime.Module(engine, wat) + store = wasmtime.Store(engine) + instance = wasmtime.Instance(store, module, []) + exports = instance.exports(store) + scanner = _WasmScanner(store, exports) + except Exception as err: # wasmtime raises assorted config/compile errors + raise NativeHelperUnavailable(f"wasmtime setup failed: {err}") from err + _wasm_scanner = scanner + return scanner + + +class _WasmScanner: + """Host side of the WASM scanner: file I/O + carry, module does the loop.""" + + DATA_BASE = 0x2000 # sample table occupies 0x1000..0x1100 + + def __init__(self, store, exports) -> None: + self._store = store + self._exports = exports + self._mem = exports["memory"] + self._scan = exports["scan"] + self._finish = exports["finish"] + + def scan_file(self, path: str, *, sample_max: int = 5) -> dict: + store = self._store + self._exports["reset"](store) + if sample_max != 5: + self._exports["set_sample_max"](store, sample_max) + carry = b"" + offset = 0 + with open(path, "rb") as f: + while True: + chunk = f.read(READ_CHUNK) + if not chunk: + break + data = carry + chunk + self._ensure_capacity(store, len(data)) + self._mem.write(store, data, self.DATA_BASE) + self._scan(store, self.DATA_BASE, len(data), offset - len(carry)) + last_nl = data.rfind(b"\n") + carry = data[last_nl + 1 :] + offset += len(chunk) + self._finish(store, offset) + count = self._exports["get_sample_count"](store) + samples = [ + ( + self._exports["get_sample_offset"](store, i), + self._exports["get_sample_length"](store, i), + ) + for i in range(count) + ] + return { + "rows": self._exports["get_rows"](store), + "first_n": samples, + "bytes": offset, + } + + def _ensure_capacity(self, store, needed: int) -> None: + """Grow linear memory when carry + chunk outgrows it (long lines).""" + pages = self._mem.size(store) + want = (self.DATA_BASE + needed + 0x10000) // 0x10000 # page-align + headroom + if want > pages: + self._mem.grow(store, want - pages) + + +# ---------------------------------------------------------------------- # +# Measurement helpers (/proc-based; ru_maxrss is unreliable on this box) # +# ---------------------------------------------------------------------- # + + +def _status_field(name: str) -> int | None: + try: + with open("/proc/self/status") as f: + for line in f: + if line.startswith(name): + return int(line.split()[1]) + except OSError: + pass + return None + + +def vm_rss_kb() -> int | None: + return _status_field("VmRSS:") + + +def vm_hwm_kb() -> int | None: + return _status_field("VmHWM:") + + +def try_reset_hwm() -> bool: + """Best-effort VmHWM reset via clear_refs (works when self-writable).""" + try: + with open("/proc/self/clear_refs", "w") as f: + f.write("5") + return True + except OSError: + return False + + +def mem_available_mb() -> int | None: + try: + with open("/proc/meminfo") as f: + for line in f: + if line.startswith("MemAvailable:"): + return int(line.split()[1]) // 1024 + except OSError: + pass + return None + + +def fadvise_dontneed(path: str) -> None: + """Best-effort page-cache eviction for one file (unprivileged).""" + fd = os.open(path, os.O_RDONLY) + try: + os.posix_fadvise(fd, 0, 0, os.POSIX_FADV_DONTNEED) + finally: + os.close(fd) + + +def loadavg() -> str | None: + try: + with open("/proc/loadavg") as f: + return f.read().split("\n")[0] + except OSError: + return None + + +def gcc_version() -> str | None: + try: + res = subprocess.run(["cc", "--version"], capture_output=True, text=True, timeout=10) + return res.stdout.splitlines()[0] if res.returncode == 0 else None + except (OSError, subprocess.TimeoutExpired): + return None + + +def host_info() -> dict: + cpu = "" + try: + with open("/proc/cpuinfo") as f: + for line in f: + if line.startswith("model name"): + cpu = line.split(":", 1)[1].strip() + break + except OSError: + pass + sha_ni = False + try: + with open("/proc/cpuinfo") as f: + sha_ni = " sha_ni" in f.read() + except OSError: + pass + wasmtime_version = None + try: + import wasmtime + + wasmtime_version = getattr(wasmtime, "__version__", "unknown") + except ImportError: + pass + return { + "cpu": cpu, + "sha_ni": sha_ni, + "kernel": platform.release(), + "python": platform.python_version(), + "gcc": gcc_version(), + "wasmtime": wasmtime_version, + "loadavg": loadavg(), + "mem_available_mb": mem_available_mb(), + "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), + } + + +def stats_ms(samples_s: list[float]) -> dict: + """Median / max over per-run wall times (seconds in, ms out).""" + ordered = sorted(samples_s) + n = len(ordered) + mid = ordered[n // 2] if n % 2 else (ordered[n // 2 - 1] + ordered[n // 2]) / 2 + return { + "runs_ms": [round(s * 1000, 3) for s in samples_s], + "median_ms": round(mid * 1000, 3), + "p95_ms": round(ordered[-1] * 1000, 3) if ordered else None, + } diff --git a/bench/native_spike/c_checksum/Makefile b/bench/native_spike/c_checksum/Makefile new file mode 100644 index 0000000..73fcef7 --- /dev/null +++ b/bench/native_spike/c_checksum/Makefile @@ -0,0 +1,23 @@ +# Native helper spike (#48) — C range-checksum candidate. +# +# Self-contained: no OpenSSL/libcrypto dependency (the target box has no +# OpenSSL headers). Emits librange_hash.so for the ctypes loader. + +CC ?= cc +CFLAGS ?= -O2 -Wall -Wextra -std=c11 -fPIC + +LIB := librange_hash.so +OBJS := sha256.o range_hash.o + +all: $(LIB) + +$(LIB): $(OBJS) + $(CC) $(CFLAGS) -shared -o $@ $(OBJS) + +%.o: %.c sha256.h + $(CC) $(CFLAGS) -c -o $@ $< + +clean: + rm -f $(LIB) $(OBJS) + +.PHONY: all clean diff --git a/bench/native_spike/c_checksum/range_hash.c b/bench/native_spike/c_checksum/range_hash.c new file mode 100644 index 0000000..568f2e0 --- /dev/null +++ b/bench/native_spike/c_checksum/range_hash.c @@ -0,0 +1,107 @@ +/* Range SHA-256 hashing for the native helper spike (#48) — ctypes surface. + * + * Two variants over the byte range [offset, offset+length): + * range_hash_pread — open + pread loop over a 1 MiB stack-static buffer + * (O(1 MiB) memory, mirrors the Python streaming shape). + * range_hash_mmap — mmap exactly the page-aligned range window and hash + * in place (zero-copy; resident pages = touched pages). + * + * Return codes: + * 0 success (out_hex holds 64 hex chars + NUL) + * -1 cannot open path + * -2 truncated (file shorter than offset+length) + * -3 I/O or mmap failure (message in err if provided) + */ +#define _POSIX_C_SOURCE 200809L + +#include "sha256.h" + +#include +#include +#include +#include +#include +#include + +#define READ_BUF (1 << 20) + +static uint8_t g_buf[READ_BUF]; + +static int finish_hex(sha256_ctx *ctx, char out_hex[65]) { + uint8_t digest[32]; + + sha256_final(ctx, digest); + sha256_hex(digest, out_hex); + return 0; +} + +int range_hash_pread(const char *path, uint64_t offset, uint64_t length, char out_hex[65]) { + int fd = open(path, O_RDONLY); + sha256_ctx ctx; + uint64_t pos = offset, remaining = length; + + if (fd < 0) { + return -1; + } + sha256_init(&ctx); + while (remaining > 0) { + size_t want = remaining < READ_BUF ? (size_t)remaining : (size_t)READ_BUF; + ssize_t got = pread(fd, g_buf, want, (off_t)pos); + if (got < 0) { + close(fd); + return -3; + } + if (got == 0) { + close(fd); + return -2; + } + sha256_update(&ctx, g_buf, (size_t)got); + pos += (uint64_t)got; + remaining -= (uint64_t)got; + } + close(fd); + return finish_hex(&ctx, out_hex); +} + +int range_hash_mmap(const char *path, uint64_t offset, uint64_t length, char out_hex[65]) { + int fd = open(path, O_RDONLY); + struct stat st; + sha256_ctx ctx; + long page; + uint64_t map_off, map_len, delta; + const uint8_t *map; + + if (fd < 0) { + return -1; + } + if (fstat(fd, &st) != 0) { + close(fd); + return -3; + } + if (length == 0) { + close(fd); + sha256_init(&ctx); + return finish_hex(&ctx, out_hex); + } + if (st.st_size < 0 || (uint64_t)st.st_size < offset || (uint64_t)st.st_size - offset < length) { + close(fd); + return -2; + } + + page = sysconf(_SC_PAGESIZE); + map_off = (offset / (uint64_t)page) * (uint64_t)page; + delta = offset - map_off; + map_len = delta + length; + + map = (const uint8_t *)mmap(NULL, (size_t)map_len, PROT_READ, MAP_PRIVATE, fd, (off_t)map_off); + if (map == MAP_FAILED) { + close(fd); + return -3; + } + + sha256_init(&ctx); + sha256_update(&ctx, map + delta, (size_t)length); + munmap((void *)map, (size_t)map_len); + close(fd); + return finish_hex(&ctx, out_hex); +} diff --git a/bench/native_spike/c_checksum/sha256.c b/bench/native_spike/c_checksum/sha256.c new file mode 100644 index 0000000..21e1719 --- /dev/null +++ b/bench/native_spike/c_checksum/sha256.c @@ -0,0 +1,145 @@ +/* Self-contained SHA-256 implementation (see sha256.h). */ +#include "sha256.h" + +#include + +static const uint32_t K[64] = { + 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, + 0x923f82a4, 0xab1c5ed5, 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, + 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, 0xe49b69c1, 0xefbe4786, + 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, + 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, + 0x06ca6351, 0x14292967, 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, + 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, 0xa2bfe8a1, 0xa81a664b, + 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, + 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, + 0x5b9cca4f, 0x682e6ff3, 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, + 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2, +}; + +#define ROTR(x, n) (((x) >> (n)) | ((x) << (32 - (n)))) + +static void sha256_block(sha256_ctx *ctx, const uint8_t block[64]) { + uint32_t w[64]; + uint32_t a, b, c, d, e, f, g, h; + uint32_t t1, t2; + int i; + + for (i = 0; i < 16; i++) { + w[i] = ((uint32_t)block[i * 4] << 24) | ((uint32_t)block[i * 4 + 1] << 16) | + ((uint32_t)block[i * 4 + 2] << 8) | ((uint32_t)block[i * 4 + 3]); + } + for (i = 16; i < 64; i++) { + uint32_t s0 = ROTR(w[i - 15], 7) ^ ROTR(w[i - 15], 18) ^ (w[i - 15] >> 3); + uint32_t s1 = ROTR(w[i - 2], 17) ^ ROTR(w[i - 2], 19) ^ (w[i - 2] >> 10); + w[i] = w[i - 16] + s0 + w[i - 7] + s1; + } + + a = ctx->state[0]; + b = ctx->state[1]; + c = ctx->state[2]; + d = ctx->state[3]; + e = ctx->state[4]; + f = ctx->state[5]; + g = ctx->state[6]; + h = ctx->state[7]; + + for (i = 0; i < 64; i++) { + uint32_t S1 = ROTR(e, 6) ^ ROTR(e, 11) ^ ROTR(e, 25); + uint32_t ch = (e & f) ^ ((~e) & g); + t1 = h + S1 + ch + K[i] + w[i]; + uint32_t S0 = ROTR(a, 2) ^ ROTR(a, 13) ^ ROTR(a, 22); + uint32_t maj = (a & b) ^ (a & c) ^ (b & c); + t2 = S0 + maj; + h = g; + g = f; + f = e; + e = d + t1; + d = c; + c = b; + b = a; + a = t1 + t2; + } + + ctx->state[0] += a; + ctx->state[1] += b; + ctx->state[2] += c; + ctx->state[3] += d; + ctx->state[4] += e; + ctx->state[5] += f; + ctx->state[6] += g; + ctx->state[7] += h; +} + +void sha256_init(sha256_ctx *ctx) { + ctx->state[0] = 0x6a09e667; + ctx->state[1] = 0xbb67ae85; + ctx->state[2] = 0x3c6ef372; + ctx->state[3] = 0xa54ff53a; + ctx->state[4] = 0x510e527f; + ctx->state[5] = 0x9b05688c; + ctx->state[6] = 0x1f83d9ab; + ctx->state[7] = 0x5be0cd19; + ctx->bitlen = 0; + ctx->buflen = 0; +} + +void sha256_update(sha256_ctx *ctx, const uint8_t *data, size_t len) { + size_t i = 0; + + ctx->bitlen += (uint64_t)len * 8; + + if (ctx->buflen > 0) { + size_t need = 64 - ctx->buflen; + size_t take = len < need ? len : need; + memcpy(ctx->buffer + ctx->buflen, data, take); + ctx->buflen += take; + i = take; + if (ctx->buflen == 64) { + sha256_block(ctx, ctx->buffer); + ctx->buflen = 0; + } + } + + for (; i + 64 <= len; i += 64) { + sha256_block(ctx, data + i); + } + + if (i < len) { + memcpy(ctx->buffer, data + i, len - i); + ctx->buflen = len - i; + } +} + +void sha256_final(sha256_ctx *ctx, uint8_t out[32]) { + uint8_t pad[72]; + size_t padlen; + uint64_t bitlen = ctx->bitlen; + int i; + + padlen = (ctx->buflen < 56) ? (56 - ctx->buflen) : (120 - ctx->buflen); + memset(pad, 0, sizeof pad); + pad[0] = 0x80; + for (i = 0; i < 8; i++) { + pad[padlen + i] = (uint8_t)(bitlen >> (56 - 8 * i)); + } + sha256_update(ctx, pad, padlen + 8); + + for (i = 0; i < 8; i++) { + out[i * 4] = (uint8_t)(ctx->state[i] >> 24); + out[i * 4 + 1] = (uint8_t)(ctx->state[i] >> 16); + out[i * 4 + 2] = (uint8_t)(ctx->state[i] >> 8); + out[i * 4 + 3] = (uint8_t)(ctx->state[i]); + } +} + +void sha256_hex(const uint8_t digest[32], char out_hex[65]) { + static const char hexdig[] = "0123456789abcdef"; + int i; + + for (i = 0; i < 32; i++) { + out_hex[i * 2] = hexdig[digest[i] >> 4]; + out_hex[i * 2 + 1] = hexdig[digest[i] & 0x0f]; + } + out_hex[64] = '\0'; +} diff --git a/bench/native_spike/c_checksum/sha256.h b/bench/native_spike/c_checksum/sha256.h new file mode 100644 index 0000000..c54148d --- /dev/null +++ b/bench/native_spike/c_checksum/sha256.h @@ -0,0 +1,25 @@ +/* Self-contained SHA-256 for the native helper spike (#48). + * + * Standard FIPS 180-4 implementation (public-domain style, no external + * dependencies): the local box has gcc but no OpenSSL headers, and the + * spike must measure native hashing end-to-end without assuming libcrypto. + */ +#ifndef HOTMEM_SPIKE_SHA256_H +#define HOTMEM_SPIKE_SHA256_H + +#include +#include + +typedef struct { + uint32_t state[8]; + uint64_t bitlen; + uint8_t buffer[64]; + size_t buflen; +} sha256_ctx; + +void sha256_init(sha256_ctx *ctx); +void sha256_update(sha256_ctx *ctx, const uint8_t *data, size_t len); +void sha256_final(sha256_ctx *ctx, uint8_t out[32]); +void sha256_hex(const uint8_t digest[32], char out_hex[65]); + +#endif diff --git a/bench/native_spike/c_jsonl/Makefile b/bench/native_spike/c_jsonl/Makefile new file mode 100644 index 0000000..50fa74d --- /dev/null +++ b/bench/native_spike/c_jsonl/Makefile @@ -0,0 +1,21 @@ +# Native helper spike (#48) — C JSONL scanner candidate. +# Emits libscan.so for the ctypes loader. + +CC ?= cc +CFLAGS ?= -O2 -Wall -Wextra -std=c11 -fPIC + +LIB := libscan.so +OBJS := scan.o + +all: $(LIB) + +$(LIB): $(OBJS) + $(CC) $(CFLAGS) -shared -o $@ $(OBJS) + +%.o: %.c scan.h + $(CC) $(CFLAGS) -c -o $@ $< + +clean: + rm -f $(LIB) $(OBJS) + +.PHONY: all clean diff --git a/bench/native_spike/c_jsonl/scan.c b/bench/native_spike/c_jsonl/scan.c new file mode 100644 index 0000000..cf1b57b --- /dev/null +++ b/bench/native_spike/c_jsonl/scan.c @@ -0,0 +1,366 @@ +/* JSONL line scanner + strict JSON validator (see scan.h). */ +#include "scan.h" + +#define _POSIX_C_SOURCE 200809L + +#include +#include +#include +#include + +#define READ_CHUNK (1 << 20) +#define MAX_LINE (16 << 20) +#define MAX_DEPTH 200 + +/* ------------------------------------------------------------------ */ +/* Strict JSON validator (RFC 8259 minus NaN/Infinity, which Python's */ +/* json module accepts; the corpus contains neither). */ +/* ------------------------------------------------------------------ */ + +typedef struct { + const uint8_t *p; + const uint8_t *end; + int depth; +} jval; + +static void j_ws(jval *j) { + while (j->p < j->end) { + uint8_t c = *j->p; + if (c == ' ' || c == '\t' || c == '\n' || c == '\r') { + j->p++; + } else { + break; + } + } +} + +static int j_hex(uint8_t c) { + return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F'); +} + +static int j_string(jval *j) { + j->p++; /* opening quote */ + while (j->p < j->end) { + uint8_t c = *j->p; + if (c == '"') { + j->p++; + return 1; + } + if (c == '\\') { + j->p++; + if (j->p >= j->end) { + return 0; + } + uint8_t e = *j->p; + if (e == '"' || e == '\\' || e == '/' || e == 'b' || e == 'f' || e == 'n' || + e == 'r' || e == 't') { + j->p++; + } else if (e == 'u') { + j->p++; + for (int i = 0; i < 4; i++, j->p++) { + if (j->p >= j->end || !j_hex(*j->p)) { + return 0; + } + } + } else { + return 0; + } + } else if (c < 0x20) { + return 0; + } else { + j->p++; + } + } + return 0; +} + +static int j_digits(jval *j) { + const uint8_t *start = j->p; + while (j->p < j->end && *j->p >= '0' && *j->p <= '9') { + j->p++; + } + return j->p > start; +} + +static int j_number(jval *j) { + if (j->p < j->end && *j->p == '-') { + j->p++; + } + if (j->p < j->end && *j->p == '0') { + j->p++; + } else if (j->p < j->end && *j->p >= '1' && *j->p <= '9') { + j_digits(j); + } else { + return 0; + } + if (j->p < j->end && *j->p == '.') { + j->p++; + if (!j_digits(j)) { + return 0; + } + } + if (j->p < j->end && (*j->p == 'e' || *j->p == 'E')) { + j->p++; + if (j->p < j->end && (*j->p == '+' || *j->p == '-')) { + j->p++; + } + if (!j_digits(j)) { + return 0; + } + } + return 1; +} + +static int j_lit(jval *j, const char *lit, size_t n) { + if ((size_t)(j->end - j->p) < n || memcmp(j->p, lit, n) != 0) { + return 0; + } + j->p += n; + return 1; +} + +static int j_value(jval *j) { + uint8_t c; + + if (j->depth > MAX_DEPTH) { + return 0; + } + j_ws(j); + if (j->p >= j->end) { + return 0; + } + c = *j->p; + if (c == '"') { + return j_string(j); + } + if (c == '{') { + j->p++; + j->depth++; + j_ws(j); + if (j->p < j->end && *j->p == '}') { + j->p++; + j->depth--; + return 1; + } + for (;;) { + j_ws(j); + if (j->p >= j->end || *j->p != '"') { + j->depth--; + return 0; + } + if (!j_string(j)) { + j->depth--; + return 0; + } + j_ws(j); + if (j->p >= j->end || *j->p != ':') { + j->depth--; + return 0; + } + j->p++; + if (!j_value(j)) { + j->depth--; + return 0; + } + j_ws(j); + if (j->p < j->end && *j->p == ',') { + j->p++; + continue; + } + if (j->p < j->end && *j->p == '}') { + j->p++; + j->depth--; + return 1; + } + j->depth--; + return 0; + } + } + if (c == '[') { + j->p++; + j->depth++; + j_ws(j); + if (j->p < j->end && *j->p == ']') { + j->p++; + j->depth--; + return 1; + } + for (;;) { + if (!j_value(j)) { + j->depth--; + return 0; + } + j_ws(j); + if (j->p < j->end && *j->p == ',') { + j->p++; + continue; + } + if (j->p < j->end && *j->p == ']') { + j->p++; + j->depth--; + return 1; + } + j->depth--; + return 0; + } + } + if (c == 't') { + return j_lit(j, "true", 4); + } + if (c == 'f') { + return j_lit(j, "false", 5); + } + if (c == 'n') { + return j_lit(j, "null", 4); + } + return j_number(j); +} + +static int json_valid(const uint8_t *data, size_t len) { + jval j; + + j.p = data; + j.end = data + len; + j.depth = 0; + if (!j_value(&j)) { + return 0; + } + j_ws(&j); + return j.p == j.end; +} + +/* ------------------------------------------------------------------ */ +/* Line scanner */ +/* ------------------------------------------------------------------ */ + +static int is_ws(uint8_t c) { + return c == ' ' || c == '\t' || c == '\r' || c == '\v' || c == '\f'; +} + +static int span_nonblank(const uint8_t *data, size_t len) { + for (size_t i = 0; i < len; i++) { + if (!is_ws(data[i])) { + return 1; + } + } + return 0; +} + +static uint8_t g_chunk[READ_CHUNK]; +static uint8_t *g_pend = NULL; /* line assembly buffer */ +static size_t g_pend_cap = 0; + +static int pend_reserve(size_t need) { + if (need <= g_pend_cap) { + return 1; + } + size_t cap = g_pend_cap ? g_pend_cap : 4096; + while (cap < need) { + cap *= 2; + } + uint8_t *p = (uint8_t *)realloc(g_pend, cap); + if (!p) { + return 0; + } + g_pend = p; + g_pend_cap = cap; + return 1; +} + +static void process_line( + scan_result *res, + const uint8_t *line, /* contiguous, WITHOUT trailing newline */ + size_t len, + uint64_t file_start, + uint64_t len_incl_nl, /* line length including the newline, when present */ + uint32_t sample_max, + int mode /* 0 scan-only, 1 full */ +) { + int nonblank = span_nonblank(line, len); + int valid = 1; + + if (!nonblank) { + return; /* blank: not a row, never bad, never sampled */ + } + res->rows++; + if (mode == 1) { + valid = len <= MAX_LINE ? json_valid(line, len) : 0; + if (!valid && res->first_bad_index < 0) { + res->first_bad_index = (int64_t)res->lines; + res->first_bad_offset = file_start; + } + } + if (res->lines < sample_max && res->sample_count < SCAN_MAX_SAMPLES) { + if (mode == 0 || valid) { + res->sample_offsets[res->sample_count] = file_start; + res->sample_lengths[res->sample_count] = len_incl_nl; + res->sample_count++; + } + } +} + +int scan_file(const char *path, uint32_t sample_max, int mode, scan_result *out) { + int fd = open(path, O_RDONLY); + uint64_t chunk_start = 0; + size_t pend_len = 0; + + if (fd < 0) { + return -1; + } + memset(out, 0, sizeof *out); + out->first_bad_index = -1; + if (!pend_reserve(4096)) { + close(fd); + return -3; + } + + for (;;) { + ssize_t got = read(fd, g_chunk, READ_CHUNK); + if (got < 0) { + close(fd); + return -3; + } + if (got == 0) { + break; + } + size_t len = (size_t)got; + size_t pos = 0; + for (;;) { + const uint8_t *nl = (const uint8_t *)memchr(g_chunk + pos, '\n', len - pos); + if (!nl) { + break; + } + size_t k = (size_t)(nl - g_chunk); /* newline position in chunk */ + size_t prefix = k - pos; /* chunk bytes of this line */ + /* invariant: pend_len > 0 implies pos == 0 (line continues chunk head) */ + if (!pend_reserve(pend_len + prefix)) { + close(fd); + return -3; + } + memcpy(g_pend + pend_len, g_chunk + pos, prefix); + size_t line_len = pend_len + prefix; + uint64_t file_start = chunk_start + pos - pend_len; + process_line(out, g_pend, line_len, file_start, line_len + 1, sample_max, mode); + out->lines++; + pend_len = 0; + pos = k + 1; + } + if (pos < len) { + if (!pend_reserve(pend_len + (len - pos))) { + close(fd); + return -3; + } + memcpy(g_pend + pend_len, g_chunk + pos, len - pos); + pend_len += len - pos; + } + chunk_start += len; + } + close(fd); + + if (pend_len > 0) { + uint64_t file_start = chunk_start - pend_len; + process_line(out, g_pend, pend_len, file_start, pend_len, sample_max, mode); + out->lines++; + } + return 0; +} diff --git a/bench/native_spike/c_jsonl/scan.h b/bench/native_spike/c_jsonl/scan.h new file mode 100644 index 0000000..3f3a94d --- /dev/null +++ b/bench/native_spike/c_jsonl/scan.h @@ -0,0 +1,46 @@ +/* JSONL line scanner for the native helper spike (#48) — ctypes surface. + * + * scan_file() mirrors hotmem JSONLInspector._stream semantics: + * - "row" = non-blank line (bytes outside {space,\t,\r,\v,\f} exist) + * - blank lines do not count as rows, do not become first_bad, and do not + * consume a sample slot — but they DO advance the line index (exactly + * like _handle_line's `line_index >= sample_size` gate). + * - sample = first lines with line_index < sample_max that satisfy the + * mode's capture rule, recorded as (file offset, length + * INCLUDING the trailing newline). + * - mode 0 (scan-only): capture non-blank lines; no JSON validation. + * - mode 1 (full): validate every non-blank line; capture VALID lines; + * report first non-blank INVALID line (index+offset). + * - the final line without a trailing newline counts like any other line + * (length excludes the absent newline). + * + * Line offsets use correct file coordinates. NOTE: the shipped Python + * _stream overstates offsets for lines following a chunk-spanning line + * (line_start = offset + pos in carry+chunk coords) — parity comparisons in + * the spike account for this documented divergence. See README. + * + * Lines longer than 16 MiB are treated as invalid (spike policy; the corpus + * uses ~1.2 KiB lines). + * + * Returns 0 on success, -1 open failure, -3 read failure. + */ +#ifndef HOTMEM_SPIKE_SCAN_H +#define HOTMEM_SPIKE_SCAN_H + +#include + +#define SCAN_MAX_SAMPLES 16 + +typedef struct { + uint64_t rows; /* non-blank lines */ + int64_t first_bad_index; /* -1 when none */ + uint64_t first_bad_offset; /* valid when first_bad_index >= 0 */ + uint64_t lines; /* total completed lines (blank included) */ + uint32_t sample_count; + uint64_t sample_offsets[SCAN_MAX_SAMPLES]; + uint64_t sample_lengths[SCAN_MAX_SAMPLES]; +} scan_result; + +int scan_file(const char *path, uint32_t sample_max, int mode, scan_result *out); + +#endif diff --git a/bench/native_spike/gen_corpus.py b/bench/native_spike/gen_corpus.py new file mode 100644 index 0000000..0b66187 --- /dev/null +++ b/bench/native_spike/gen_corpus.py @@ -0,0 +1,297 @@ +#!/usr/bin/env python3 +"""Deterministic corpus generator for the native helper spike (#48). + +Purpose: + Generate the reproducible benchmark corpus used by run_bench.py: + - binary checksum files (B1: range SHA-256 arms) + - JSONL event files shaped like real hotmem swap records (B2: scanning arms) + - loose markdown bundle trees (B3: bundle parse profile) + +Determinism: + Binary content: SHAKE-256 counter-mode (fast, constant-RSS; random.Random + .randbytes measured pathological on this box — see README methodology). + Text content: seeded random.Random per artifact. + Same seed + same profile => byte-identical corpus (verified via manifest + SHA-256 digests, which are committed in manifest.json). + +Usage: + python gen_corpus.py [--out corpus] [--profile reduced|full] + + reduced: checksum files up to 100MB (~285MB total) — default. + full: adds a 200MB checksum file (~485MB total) for the gated 200MB arm. + +Writes: + /manifest.json — sizes, SHA-256 digests, row counts, bad-line offsets. +""" + +from __future__ import annotations + +import argparse +import base64 +import hashlib +import json +import random +import sys +from datetime import UTC, datetime, timedelta +from pathlib import Path + +SEED = 48 +CHUNK = 8 << 20 # 8 MiB write window — constant RSS regardless of file size. + +CHECKSUM_SIZES_REDUCED = [ + ("bin_1kb.bin", 1024), + ("bin_100kb.bin", 100 * 1024), + ("bin_1mb.bin", 1 << 20), + ("bin_10mb.bin", 10 << 20), + ("bin_100mb.bin", 100 << 20), +] +CHECKSUM_SIZES_FULL = CHECKSUM_SIZES_REDUCED + [("bin_200mb.bin", 200 << 20)] + +JSONL_TARGETS = [ + ("events_10mb.jsonl", 10 << 20), + ("events_50mb.jsonl", 50 << 20), + ("events_100mb.jsonl", 100 << 20), +] + +BUNDLE_COUNTS = [100, 1000, 2500] + +_WORDS = ( # noqa: SIM905 — multi-line split is more maintainable here + "memory agent fact snapshot hydration provenance checksum range offset " + "bundle markdown metadata identifier namespace tier promotion archive " + "invoice contract customer meeting decision preference tool result query " + "search vector embedding index local filesystem sidecar event log append " + "snapshot manifest attachment reference hydrate compact audit full agent " + "risk policy schedule deadline owner status priority review deploy model" +).split() + +_BASE_TIME = datetime(2026, 7, 1, tzinfo=UTC) + + +def shake_ctr(seed_label: str, total: int, sink) -> None: + """Write `total` deterministic bytes to `sink` via SHAKE-256 counter-mode.""" + label = seed_label.encode() + written = 0 + counter = 0 + while written < total: + block = hashlib.shake_256(label + b":" + counter.to_bytes(8, "big")).digest( + min(CHUNK, total - written) + ) + sink.write(block) + written += len(block) + counter += 1 + + +def _sha256_file(path: Path) -> tuple[str, int]: + h = hashlib.sha256() + size = 0 + with open(path, "rb") as f: + while True: + chunk = f.read(1 << 20) + if not chunk: + break + h.update(chunk) + size += len(chunk) + return h.hexdigest(), size + + +def gen_checksum_files(out: Path, profile: str) -> list[dict]: + sizes = CHECKSUM_SIZES_FULL if profile == "full" else CHECKSUM_SIZES_REDUCED + entries = [] + for name, size in sizes: + path = out / name + with open(path, "wb") as f: + shake_ctr(f"checksum:{name}", size, f) + digest, actual = _sha256_file(path) + assert actual == size, f"{name}: wrote {actual}, expected {size}" + entries.append({"name": name, "size": size, "sha256": digest}) + print(f" {name}: {size} bytes sha256={digest[:16]}…") + return entries + + +def _record(rng: random.Random, idx: int) -> dict: + """One swap.jsonl-shaped record (same field set as real snapshot rows).""" + fact = " ".join(rng.choice(_WORDS) for _ in range(rng.randint(10, 22))) + identifier = rng.choice(["user", "project", "session", "agent", "team"]) + f"-{idx % 97}" + created = _BASE_TIME + timedelta(seconds=idx * 37 + rng.randint(0, 36)) + raw = rng.randbytes(256) # 256 bytes -> 344-char b64, same as real embedding blob + importance = round(rng.uniform(0.1, 0.9), 2) + return { + "id": rng.getrandbits(128).to_bytes(16, "big").hex(), + "identifier": identifier, + "fact_text": fact, + "embedding_dim": 64, + "embedding_model": "hotmem-hash-v1", + "source": rng.choice(["", "bundle", "import", "api"]), + "importance": importance, + "metadata_json": "{}", + "content_hash": hashlib.sha256(f"{identifier}:{fact}".encode()).hexdigest(), + "ttl_seconds": None, + "created_at": created.strftime("%Y-%m-%dT%H:%M:%SZ"), + "namespace": rng.choice(["", "work", "personal"]), + "tier": rng.choice(["hot", "warm"]), + "memory_type": "fact", + "source_uri": "", + "source_format": "", + "source_checksum": "", + "byte_offset": None, + "byte_length": None, + "updated_at": None, + "snapshot_id": "", + "promotion_state": rng.choice(["HOT", "READY"]), + "promotion_candidate": rng.randint(0, 1), + "parent_memory": "", + "related_memories": "[]", + "tags": json.dumps([rng.choice(_WORDS) for _ in range(rng.randint(0, 3))]), + "schema_version": 1, + "fact_summary": None, + "provenance_json": None, + "embedding_b64": base64.b64encode(raw).decode(), + } + + +def gen_jsonl_files(out: Path) -> list[dict]: + entries = [] + for name, target in JSONL_TARGETS: + rng = random.Random(f"{SEED}:jsonl:{name}") + path = out / name + rows = 0 + bad_line_index = None + bad_line_offset = None + written = 0 + bad_written = False + with open(path, "wb") as f: + idx = 0 + while written < target: + # Insert one malformed line at ~60% of the target size. + if not bad_written and written >= target * 0.6: + bad_line_offset = written + bad_line_index = rows + f.write(b'{"id": "broken", "fact_text": "truncated') + f.write(b"\n") + written += 38 + rows += 1 + bad_written = True + continue + line = json.dumps(_record(rng, idx), separators=(",", ":")).encode() + f.write(line) + f.write(b"\n") + written += len(line) + 1 + rows += 1 + idx += 1 + digest, size = _sha256_file(path) + assert bad_written, f"{name}: malformed line never inserted" + entries.append( + { + "name": name, + "size": size, + "rows": rows, + "sha256": digest, + "bad_line_index": bad_line_index, + "bad_line_offset": bad_line_offset, + } + ) + print(f" {name}: {size} bytes, {rows} rows, bad line #{bad_line_index}") + return entries + + +def _bundle_dir(tree: Path, i: int) -> Path: + return tree / f"bundle_{i:05d}" + + +def gen_bundle_tree(out: Path, count: int) -> dict: + tree = out / f"bundles_{count}" + tree.mkdir(parents=True, exist_ok=True) + n_files = 0 + for i in range(count): + rng = random.Random(f"{SEED}:bundle:{count}:{i}") + b = _bundle_dir(tree, i) + b.mkdir(exist_ok=True) + (b / "memory.md").write_text( + f"# Bundle {i:05d}\n\n" + + "\n".join( + f"- {' '.join(rng.choice(_WORDS) for _ in range(rng.randint(6, 14)))}" + for _ in range(rng.randint(6, 12)) + ) + + "\n", + encoding="utf-8", + ) + (b / "metadata.json").write_text( + json.dumps( + { + "identifier": f"bundle-{i:05d}", + "importance": round(rng.uniform(0.2, 0.9), 2), + "namespace": rng.choice(["", "work"]), + "tags": [rng.choice(_WORDS) for _ in range(2)], + "source": "spike-corpus", + } + ), + encoding="utf-8", + ) + (b / "facts.json").write_text( + json.dumps( + [ + {"fact": " ".join(rng.choice(_WORDS) for _ in range(rng.randint(8, 18)))} + for _ in range(3) + ] + ), + encoding="utf-8", + ) + with open(b / "events.jsonl", "wb") as f: + for _ in range(5): + text = " ".join(rng.choice(_WORDS) for _ in range(rng.randint(6, 15))) + line = json.dumps({"event": text}, separators=(",", ":")) + "\n" + f.write(line.encode()) + att = b / "attachments" + att.mkdir(exist_ok=True) + (att / "data.bin").write_bytes( + hashlib.shake_256(f"{SEED}:att:{count}:{i}".encode()).digest(rng.randint(200, 500)) + ) + (att / "notes.txt").write_text( + " ".join(rng.choice(_WORDS) for _ in range(20)) + "\n", encoding="utf-8" + ) + n_files += 7 + # Cheap tree fingerprint: sha256 over sorted (relpath, size, digest) triples. + h = hashlib.sha256() + for p in sorted(tree.rglob("*")): + if p.is_file(): + d, s = _sha256_file(p) + h.update(f"{p.relative_to(tree)}:{s}:{d}\n".encode()) + print(f" bundles_{count}: {count} bundles, {n_files} files, digest={h.hexdigest()[:16]}…") + return { + "name": f"bundles_{count}", + "count": count, + "files": n_files, + "tree_digest": h.hexdigest(), + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--out", default="corpus", help="output directory (default: corpus)") + parser.add_argument("--profile", choices=["reduced", "full"], default="reduced") + parser.add_argument("--bundles", default=",".join(str(c) for c in BUNDLE_COUNTS)) + args = parser.parse_args() + + out = Path(__file__).resolve().parent / args.out + out.mkdir(parents=True, exist_ok=True) + + print(f"Generating spike corpus (seed={SEED}, profile={args.profile}) in {out}") + manifest = { + "generator": "gen_corpus.py", + "seed": SEED, + "profile": args.profile, + "created": datetime.now(tz=UTC).strftime("%Y-%m-%dT%H:%M:%SZ"), + "checksum_files": gen_checksum_files(out, args.profile), + "jsonl_files": gen_jsonl_files(out), + "bundle_trees": [ + gen_bundle_tree(out, int(c)) for c in args.bundles.split(",") if c.strip() + ], + } + manifest_path = out.parent / "manifest.json" + manifest_path.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8") + print(f"Manifest written: {manifest_path}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/bench/native_spike/manifest.json b/bench/native_spike/manifest.json new file mode 100644 index 0000000..ed4f40e --- /dev/null +++ b/bench/native_spike/manifest.json @@ -0,0 +1,79 @@ +{ + "generator": "gen_corpus.py", + "seed": 48, + "profile": "reduced", + "created": "2026-08-27T01:41:25Z", + "checksum_files": [ + { + "name": "bin_1kb.bin", + "size": 1024, + "sha256": "13b413ae99f070dc77c5d604c6b902a8af440e6ad817ebb5aeeaf8d85ebef594" + }, + { + "name": "bin_100kb.bin", + "size": 102400, + "sha256": "407f241da530674b6803b9b3fe2bad2e6bb8b6bce10a988d2c062c5ac248e075" + }, + { + "name": "bin_1mb.bin", + "size": 1048576, + "sha256": "da992adb41e2c939a2207b611d8f81012d67e120f2906ac7c81af88dfd8c7df5" + }, + { + "name": "bin_10mb.bin", + "size": 10485760, + "sha256": "931ff5c54c2cf812861679b87335ef2186c523c8a0c87c1c9a21ba7475f05227" + }, + { + "name": "bin_100mb.bin", + "size": 104857600, + "sha256": "124c78b11c2590eb9d5778159b562190ca3bd50d146eb0979abbc5348ca829ba" + } + ], + "jsonl_files": [ + { + "name": "events_10mb.jsonl", + "size": 10485937, + "rows": 8894, + "sha256": "2b540649918a3a84b10ed3509c7bcf5f0a488a7af0735d19878e62c5f3bb8db3", + "bad_line_index": 5339, + "bad_line_offset": 6292320 + }, + { + "name": "events_50mb.jsonl", + "size": 52429550, + "rows": 44464, + "sha256": "8d6b43def019c2c98679cebee538d9e883625b374146b2ee53a5beaf0207adb7", + "bad_line_index": 26678, + "bad_line_offset": 31457732 + }, + { + "name": "events_100mb.jsonl", + "size": 104857784, + "rows": 88914, + "sha256": "15565bfdb04cc4e66ecb9f88590da8224474fca30ba9b3761ccb772d11b4fca5", + "bad_line_index": 53346, + "bad_line_offset": 62915663 + } + ], + "bundle_trees": [ + { + "name": "bundles_100", + "count": 100, + "files": 700, + "tree_digest": "96b38eb510a4ef70124785121817da93d0d1619f6ad565a7eed072778f8761ea" + }, + { + "name": "bundles_1000", + "count": 1000, + "files": 7000, + "tree_digest": "85d63479813bc350183ddb1c45eefd2e3e09b52fd876a4825cd4f5ab058d708d" + }, + { + "name": "bundles_2500", + "count": 2500, + "files": 17500, + "tree_digest": "31523e64e8e68d8f73fb07ec24fea16b082095b4846b10902c038b916bd394da" + } + ] +} diff --git a/bench/native_spike/py_baseline.py b/bench/native_spike/py_baseline.py new file mode 100644 index 0000000..cb5424a --- /dev/null +++ b/bench/native_spike/py_baseline.py @@ -0,0 +1,189 @@ +"""Python baseline arms for the native helper spike (#48). + +Purpose: + Benchmark arms that reproduce the CURRENT hotmem Python paths, plus the + pure-Python improvement candidates they would be compared against. + + B1 (range checksum): + py_current_double_read — the real hydrate-with-verify path: + adapter.read_range() then verify_range() (which re-reads the range). + Uses the REAL hotmem classes, exactly as memory.hydrate_memory_detailed + calls them (src/hotmem/memory.py:244 + src/hotmem/provenance.py:103). + py_single_read — candidate pure-Python fix: one read, hash once. + py_streaming — best-case pure Python: chunked seek+read+hash, no + whole-range bytes object (mirrors storage/local.py _checksum style). + + B2 (JSONL scanning): + py_stream_real — the REAL JSONLInspector._stream (validation included). + py_scan_only — replica of _stream's scan loop with validation removed, + used to isolate scanning cost from json.loads validation cost. + +Note on offsets: + py_scan_only uses corrected line-start arithmetic (base + pos). The + shipped _stream computes line_start = offset + pos in (carry+chunk) + coordinates, which overstates offsets by len(carry) whenever a line spans + a read-chunk boundary. See README ("Discovered: _stream offset bug"). + +Deps: hotmem (real code paths), stdlib only. +""" + +from __future__ import annotations + +import hashlib +import json + +READ_CHUNK = 1 << 20 # 1 MiB — same window as JSONLInspector._stream. +_WS = b" \t\n\r\v\f" + + +def py_current_double_read(uri: str, offset: int, length: int, expected: str) -> str: + """Faithful reproduction of the current verified-hydration path. + + Reads the range (read 1, held while verifying — as memory.py does), then + verify_range() re-reads the same bytes and hashes them (read 2). Raises + ProvenanceError subclasses on failure, exactly like production. + """ + from hotmem.provenance import verify_range + from hotmem.storage.local import LocalFilesystemAdapter + + adapter = LocalFilesystemAdapter() + data = adapter.read_range(uri, offset, length) # read 1 — kept alive, as in memory.py + try: + verify_range(adapter, uri, offset, length, expected) # read 2 + hash + finally: + del data + return expected + + +def py_single_read(uri: str, offset: int, length: int) -> str: + """Candidate pure-Python fix: one read, hash once.""" + from hotmem.storage.local import LocalFilesystemAdapter + + adapter = LocalFilesystemAdapter() + data = adapter.read_range(uri, offset, length) + try: + return hashlib.sha256(data).hexdigest() + finally: + del data + + +def py_streaming(path: str, offset: int, length: int) -> str: + """Best-case pure Python: chunked read + incremental hash, O(1MB) memory.""" + h = hashlib.sha256() + remaining = length + with open(path, "rb") as f: + f.seek(offset) + while remaining > 0: + chunk = f.read(min(READ_CHUNK, remaining)) + if not chunk: + raise EOFError(f"short read: {length - remaining} bytes missing") + h.update(chunk) + remaining -= len(chunk) + return h.hexdigest() + + +def _nonblank(line: bytes) -> bool: + return bool(line.strip(_WS)) + + +def py_scan_only(path: str, *, sample_size: int = 5) -> dict: + """Scan-only contract: rows + first-N line boundaries, no JSON validation. + + Mirrors JSONLInspector._stream's chunk/carry structure and its sampling + eligibility gate (line_index < sample_size), but validates nothing. + Line offsets use corrected arithmetic (see module docstring). + """ + rows = 0 + first_n: list[tuple[int, int]] = [] + line_index = 0 + carry = b"" + + with open(path, "rb") as f: + offset = 0 + while True: + chunk = f.read(READ_CHUNK) + if not chunk: + if carry and _nonblank(carry): + rows += 1 + if line_index < sample_size: + first_n.append((offset - len(carry), len(carry))) + break + data = carry + chunk + base = offset - len(carry) # file offset of data[0] + pos = 0 + nl = data.find(b"\n", pos) + while nl != -1: + line = data[pos : nl + 1] + if _nonblank(line): + rows += 1 + if line_index < sample_size: + first_n.append((base + pos, nl + 1 - pos)) + line_index += 1 + pos = nl + 1 + nl = data.find(b"\n", pos) + carry = data[pos:] + offset += len(chunk) + + return {"rows": rows, "first_n": first_n, "bytes": offset} + + +def py_stream_real(path: str, *, sample_size: int = 5) -> dict: + """Run the REAL JSONLInspector._stream and normalize its outputs. + + Returns row_count, sample rows, byte_ranges, and the first-bad-line + (index, offset) parsed out of the unsupported_reason message (the real + function only reports it as a human string). + """ + from hotmem.inspectors.jsonl_inspector import _stream # noqa: PLC2701 — bench parity target + + row_count, sample_rows, byte_ranges, unsupported_reason = _stream( + path, count_rows=True, sample_size=sample_size + ) + bad = None + if unsupported_reason: + prefix = "line " + rest = unsupported_reason[len(prefix) :] + index_s, rest = rest.split(" (offset ", 1) + offset_s = rest.split(")", 1)[0] + bad = {"index": int(index_s), "offset": int(offset_s), "reason": unsupported_reason} + return { + "row_count": row_count, + "sample_rows": sample_rows, + "byte_ranges": byte_ranges, + "first_bad": bad, + } + + +def reference_range_digest(path: str, offset: int, length: int) -> str: + """Independent reference digest for parity assertions (plain hashlib).""" + with open(path, "rb") as f: + f.seek(offset) + data = f.read(length) + if len(data) != length: + raise EOFError(f"short read: got {len(data)}, want {length}") + return hashlib.sha256(data).hexdigest() + + +def main() -> None: + """Quick smoke check on the real repo swap.jsonl + one corpus file.""" + from pathlib import Path + + corpus = Path(__file__).resolve().parent / "corpus" + swap = Path(__file__).resolve().parents[2] / "swap.jsonl" + if swap.exists(): + r = py_stream_real(str(swap)) + s = py_scan_only(str(swap)) + assert r["row_count"] == s["rows"], (r["row_count"], s["rows"]) + print(f"swap.jsonl: rows={r['row_count']} scan_first5={s['first_n'][:2]}") + binf = corpus / "bin_1mb.bin" + if binf.exists(): + ref = reference_range_digest(str(binf), 0, binf.stat().st_size) + assert py_single_read(str(binf), 0, binf.stat().st_size) == ref + assert py_streaming(str(binf), 0, binf.stat().st_size) == ref + assert py_current_double_read(str(binf), 0, binf.stat().st_size, ref) == ref + print(f"bin_1mb.bin: B1 arms agree with reference {ref[:16]}…") + print(json.dumps({"ok": True})) + + +if __name__ == "__main__": + main() diff --git a/bench/native_spike/results.json b/bench/native_spike/results.json new file mode 100644 index 0000000..21e583f --- /dev/null +++ b/bench/native_spike/results.json @@ -0,0 +1,4205 @@ +{ + "spike": "#48 native helper spike", + "profile": "reduced", + "quick": false, + "corpus_seed": 48, + "host": { + "cpu": "Intel(R) Core(TM) i7-10610U CPU @ 1.80GHz", + "sha_ni": false, + "kernel": "6.18.33.1-microsoft-standard-WSL2", + "python": "3.12.3", + "gcc": "cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0", + "wasmtime": "unknown", + "loadavg": "3.77 3.09 2.22 4/409 11309", + "mem_available_mb": 580, + "timestamp": "2026-08-27T06:50:39Z" + }, + "config": { + "runs": 5, + "cache_modes": [ + "warm", + "cold" + ] + }, + "b1_range_checksum": { + "cases": [ + { + "file": "bin_1kb.bin", + "offset": 0, + "length": 1024, + "context": { + "loadavg": "3.77 3.09 2.22 4/409 11309", + "mem_available_mb": 580, + "ts": "07:50:39" + }, + "arms": { + "py_current_double_read": { + "warm": { + "runs_ms": [ + 0.294, + 0.445, + 0.279, + 0.187, + 0.245 + ], + "median_ms": 0.279, + "p95_ms": 0.445, + "rss_baseline_kb": 18556, + "rss_peak_kb": 20060, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 569, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "warm", + "loadavg": "3.77 3.09 2.22 4/410 11310", + "spawn_s": 0.2 + }, + "cold": { + "runs_ms": [ + 38.527, + 0.635, + 0.562, + 0.58, + 0.467 + ], + "median_ms": 0.58, + "p95_ms": 38.527, + "rss_baseline_kb": 18552, + "rss_peak_kb": 20052, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 565, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "cold", + "loadavg": "3.77 3.09 2.22 4/410 11311", + "spawn_s": 0.2 + } + }, + "py_single_read": { + "warm": { + "runs_ms": [ + 0.112, + 0.09, + 0.096, + 0.095, + 0.114 + ], + "median_ms": 0.096, + "p95_ms": 0.114, + "rss_baseline_kb": 18564, + "rss_peak_kb": 19300, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 552, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "warm", + "loadavg": "3.77 3.09 2.22 5/410 11312", + "spawn_s": 0.2 + }, + "cold": { + "runs_ms": [ + 12.842, + 0.44, + 4.983, + 0.48, + 0.433 + ], + "median_ms": 0.48, + "p95_ms": 12.842, + "rss_baseline_kb": 18564, + "rss_peak_kb": 19300, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 612, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 3/408 11316", + "spawn_s": 0.2 + } + }, + "py_streaming": { + "warm": { + "runs_ms": [ + 0.041, + 0.033, + 0.028, + 0.028, + 0.027 + ], + "median_ms": 0.028, + "p95_ms": 0.041, + "rss_baseline_kb": 18576, + "rss_peak_kb": 18576, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 611, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 1/410 11319", + "spawn_s": 0.1 + }, + "cold": { + "runs_ms": [ + 0.57, + 0.453, + 0.399, + 0.392, + 0.39 + ], + "median_ms": 0.399, + "p95_ms": 0.57, + "rss_baseline_kb": 18568, + "rss_peak_kb": 18568, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 600, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 2/411 11331", + "spawn_s": 0.1 + } + }, + "c_pread": { + "warm": { + "runs_ms": [ + 0.027, + 0.022, + 0.02, + 0.02, + 0.019 + ], + "median_ms": 0.02, + "p95_ms": 0.027, + "rss_baseline_kb": 18600, + "rss_peak_kb": 18668, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 600, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 2/411 11332", + "spawn_s": 0.1 + }, + "cold": { + "runs_ms": [ + 3.704, + 0.625, + 1.464, + 8.486, + 3.765 + ], + "median_ms": 3.704, + "p95_ms": 8.486, + "rss_baseline_kb": 18620, + "rss_peak_kb": 18688, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 594, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 3/413 11335", + "spawn_s": 0.1 + } + }, + "c_mmap": { + "warm": { + "runs_ms": [ + 0.053, + 0.048, + 0.045, + 0.045, + 0.044 + ], + "median_ms": 0.045, + "p95_ms": 0.053, + "rss_baseline_kb": 18600, + "rss_peak_kb": 18604, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 581, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 3/413 11336", + "spawn_s": 0.1 + }, + "cold": { + "runs_ms": [ + 13.662, + 7.151, + 1.017, + 7.292, + 5.105 + ], + "median_ms": 7.151, + "p95_ms": 13.662, + "rss_baseline_kb": 18592, + "rss_peak_kb": 18596, + "hwm_reset": true, + "output": { + "digest": "13b413ae99f0\u2026" + }, + "mem_available_mb": 568, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1kb.bin", + "offset": 0, + "length": 1024, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 4/413 11337", + "spawn_s": 0.2 + } + } + }, + "parity_all_arms_agree": true + }, + { + "file": "bin_100kb.bin", + "offset": 0, + "length": 102400, + "context": { + "loadavg": "3.95 3.14 2.24 4/412 11337", + "mem_available_mb": 565, + "ts": "07:50:41" + }, + "arms": { + "py_current_double_read": { + "warm": { + "runs_ms": [ + 0.629, + 0.7, + 0.736, + 0.749, + 0.821 + ], + "median_ms": 0.736, + "p95_ms": 0.821, + "rss_baseline_kb": 18664, + "rss_peak_kb": 20252, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 572, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 5/414 11341", + "spawn_s": 0.2 + }, + "cold": { + "runs_ms": [ + 33.7, + 1.466, + 1.311, + 1.678, + 1.832 + ], + "median_ms": 1.678, + "p95_ms": 33.7, + "rss_baseline_kb": 18656, + "rss_peak_kb": 20248, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 573, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 4/413 11346", + "spawn_s": 0.2 + } + }, + "py_single_read": { + "warm": { + "runs_ms": [ + 0.53, + 0.619, + 0.55, + 0.507, + 0.527 + ], + "median_ms": 0.53, + "p95_ms": 0.619, + "rss_baseline_kb": 18776, + "rss_peak_kb": 19504, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 570, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 2/410 11355", + "spawn_s": 0.1 + }, + "cold": { + "runs_ms": [ + 16.058, + 1.323, + 1.256, + 1.149, + 1.181 + ], + "median_ms": 1.256, + "p95_ms": 16.058, + "rss_baseline_kb": 18620, + "rss_peak_kb": 19344, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 567, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 2/410 11356", + "spawn_s": 0.2 + } + }, + "py_streaming": { + "warm": { + "runs_ms": [ + 0.837, + 0.787, + 0.791, + 0.871, + 0.766 + ], + "median_ms": 0.791, + "p95_ms": 0.871, + "rss_baseline_kb": 18776, + "rss_peak_kb": 18776, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 559, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 3/410 11357", + "spawn_s": 0.1 + }, + "cold": { + "runs_ms": [ + 1.579, + 1.551, + 1.175, + 1.466, + 1.589 + ], + "median_ms": 1.551, + "p95_ms": 1.589, + "rss_baseline_kb": 18680, + "rss_peak_kb": 18692, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 555, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 2/407 11358", + "spawn_s": 0.2 + } + }, + "c_pread": { + "warm": { + "runs_ms": [ + 1.072, + 1.15, + 1.326, + 1.38, + 2.013 + ], + "median_ms": 1.326, + "p95_ms": 2.013, + "rss_baseline_kb": 18684, + "rss_peak_kb": 18852, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 555, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 3/407 11359", + "spawn_s": 0.1 + }, + "cold": { + "runs_ms": [ + 2.795, + 1.946, + 2.472, + 1.663, + 1.855 + ], + "median_ms": 1.946, + "p95_ms": 2.795, + "rss_baseline_kb": 18680, + "rss_peak_kb": 18848, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 583, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 2/407 11363", + "spawn_s": 0.2 + } + }, + "c_mmap": { + "warm": { + "runs_ms": [ + 1.391, + 1.442, + 1.064, + 1.08, + 1.106 + ], + "median_ms": 1.106, + "p95_ms": 1.442, + "rss_baseline_kb": 18688, + "rss_peak_kb": 18692, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 577, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 2/407 11364", + "spawn_s": 0.1 + }, + "cold": { + "runs_ms": [ + 1.84, + 1.91, + 1.835, + 1.834, + 2.028 + ], + "median_ms": 1.84, + "p95_ms": 2.028, + "rss_baseline_kb": 18692, + "rss_peak_kb": 18696, + "hwm_reset": true, + "output": { + "digest": "407f241da530\u2026" + }, + "mem_available_mb": 573, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100kb.bin", + "offset": 0, + "length": 102400, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 4/407 11365", + "spawn_s": 0.2 + } + } + }, + "parity_all_arms_agree": true + }, + { + "file": "bin_1mb.bin", + "offset": 0, + "length": 1048576, + "context": { + "loadavg": "3.95 3.14 2.24 4/406 11365", + "mem_available_mb": 573, + "ts": "07:50:43" + }, + "arms": { + "py_current_double_read": { + "warm": { + "runs_ms": [ + 10.011, + 10.47, + 8.393, + 9.55, + 12.347 + ], + "median_ms": 10.011, + "p95_ms": 12.347, + "rss_baseline_kb": 18580, + "rss_peak_kb": 22076, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 561, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 5/409 11368", + "spawn_s": 0.2 + }, + "cold": { + "runs_ms": [ + 58.693, + 11.939, + 15.502, + 13.412, + 13.297 + ], + "median_ms": 13.412, + "p95_ms": 58.693, + "rss_baseline_kb": 18580, + "rss_peak_kb": 22096, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 558, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 6/409 11369", + "spawn_s": 0.3 + } + }, + "py_single_read": { + "warm": { + "runs_ms": [ + 7.583, + 6.053, + 5.183, + 5.854, + 7.089 + ], + "median_ms": 6.053, + "p95_ms": 7.583, + "rss_baseline_kb": 18568, + "rss_peak_kb": 20316, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 563, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 3/409 11370", + "spawn_s": 0.2 + }, + "cold": { + "runs_ms": [ + 21.009, + 7.009, + 6.805, + 7.398, + 7.141 + ], + "median_ms": 7.141, + "p95_ms": 21.009, + "rss_baseline_kb": 18608, + "rss_peak_kb": 20356, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 551, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 2/409 11374", + "spawn_s": 0.2 + } + }, + "py_streaming": { + "warm": { + "runs_ms": [ + 5.943, + 6.162, + 7.541, + 5.604, + 5.164 + ], + "median_ms": 5.943, + "p95_ms": 7.541, + "rss_baseline_kb": 18556, + "rss_peak_kb": 19576, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 552, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 3/409 11375", + "spawn_s": 0.2 + }, + "cold": { + "runs_ms": [ + 8.23, + 7.048, + 7.313, + 8.226, + 7.916 + ], + "median_ms": 7.916, + "p95_ms": 8.23, + "rss_baseline_kb": 18584, + "rss_peak_kb": 19604, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 551, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 3/409 11376", + "spawn_s": 0.2 + } + }, + "c_pread": { + "warm": { + "runs_ms": [ + 13.372, + 11.948, + 11.965, + 12.181, + 12.506 + ], + "median_ms": 12.181, + "p95_ms": 13.372, + "rss_baseline_kb": 18580, + "rss_peak_kb": 19672, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 547, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 3/407 11377", + "spawn_s": 0.2 + }, + "cold": { + "runs_ms": [ + 16.008, + 13.952, + 15.166, + 13.242, + 14.379 + ], + "median_ms": 14.379, + "p95_ms": 16.008, + "rss_baseline_kb": 18576, + "rss_peak_kb": 19668, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 582, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 4/407 11378", + "spawn_s": 0.2 + } + }, + "c_mmap": { + "warm": { + "runs_ms": [ + 13.806, + 11.119, + 13.914, + 13.172, + 12.186 + ], + "median_ms": 13.172, + "p95_ms": 13.914, + "rss_baseline_kb": 18596, + "rss_peak_kb": 19508, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 584, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "warm", + "loadavg": "3.95 3.14 2.24 2/407 11382", + "spawn_s": 0.2 + }, + "cold": { + "runs_ms": [ + 16.781, + 15.202, + 13.333, + 14.974, + 13.23 + ], + "median_ms": 14.974, + "p95_ms": 16.781, + "rss_baseline_kb": 18596, + "rss_peak_kb": 19508, + "hwm_reset": true, + "output": { + "digest": "da992adb41e2\u2026" + }, + "mem_available_mb": 590, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "cold", + "loadavg": "3.95 3.14 2.24 4/409 11385", + "spawn_s": 0.2 + } + } + }, + "parity_all_arms_agree": true + }, + { + "file": "bin_10mb.bin", + "offset": 0, + "length": 10485760, + "context": { + "loadavg": "3.95 3.14 2.24 4/408 11385", + "mem_available_mb": 585, + "ts": "07:50:45" + }, + "arms": { + "py_current_double_read": { + "warm": { + "runs_ms": [ + 99.668, + 90.692, + 90.572, + 93.882, + 96.478 + ], + "median_ms": 93.882, + "p95_ms": 99.668, + "rss_baseline_kb": 18552, + "rss_peak_kb": 40492, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 562, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "warm", + "loadavg": "3.87 3.13 2.24 4/413 11401", + "spawn_s": 0.8 + }, + "cold": { + "runs_ms": [ + 134.785, + 99.74, + 90.012, + 90.518, + 79.692 + ], + "median_ms": 90.518, + "p95_ms": 134.785, + "rss_baseline_kb": 18564, + "rss_peak_kb": 40388, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 562, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "cold", + "loadavg": "3.87 3.13 2.24 1/410 11404", + "spawn_s": 0.7 + } + }, + "py_single_read": { + "warm": { + "runs_ms": [ + 55.479, + 60.87, + 62.215, + 60.931, + 59.78 + ], + "median_ms": 60.87, + "p95_ms": 62.215, + "rss_baseline_kb": 18656, + "rss_peak_kb": 29616, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 550, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "warm", + "loadavg": "3.87 3.13 2.24 4/409 11422", + "spawn_s": 0.6 + }, + "cold": { + "runs_ms": [ + 118.938, + 95.781, + 88.855, + 107.275, + 90.705 + ], + "median_ms": 95.781, + "p95_ms": 118.938, + "rss_baseline_kb": 18580, + "rss_peak_kb": 29540, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 593, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "cold", + "loadavg": "3.87 3.13 2.24 5/409 11429", + "spawn_s": 0.7 + } + }, + "py_streaming": { + "warm": { + "runs_ms": [ + 89.498, + 98.977, + 84.645, + 78.68, + 88.438 + ], + "median_ms": 88.438, + "p95_ms": 98.977, + "rss_baseline_kb": 18580, + "rss_peak_kb": 20624, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 552, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "warm", + "loadavg": "3.87 3.13 2.24 2/407 11433", + "spawn_s": 0.8 + }, + "cold": { + "runs_ms": [ + 127.991, + 149.291, + 110.86, + 75.711, + 73.113 + ], + "median_ms": 110.86, + "p95_ms": 149.291, + "rss_baseline_kb": 18580, + "rss_peak_kb": 20624, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 605, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "cold", + "loadavg": "3.87 3.13 2.24 4/406 11439", + "spawn_s": 0.8 + } + }, + "c_pread": { + "warm": { + "runs_ms": [ + 145.078, + 141.439, + 156.413, + 179.584, + 141.848 + ], + "median_ms": 145.078, + "p95_ms": 179.584, + "rss_baseline_kb": 18572, + "rss_peak_kb": 19664, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 578, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "warm", + "loadavg": "4.44 3.26 2.29 4/406 11443", + "spawn_s": 1.1 + }, + "cold": { + "runs_ms": [ + 170.917, + 159.767, + 153.184, + 147.358, + 193.756 + ], + "median_ms": 159.767, + "p95_ms": 193.756, + "rss_baseline_kb": 18584, + "rss_peak_kb": 19676, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 632, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "cold", + "loadavg": "4.44 3.26 2.29 2/409 11463", + "spawn_s": 1.0 + } + }, + "c_mmap": { + "warm": { + "runs_ms": [ + 156.408, + 139.362, + 130.552, + 128.523, + 131.394 + ], + "median_ms": 131.394, + "p95_ms": 156.408, + "rss_baseline_kb": 18572, + "rss_peak_kb": 28808, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 601, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "warm", + "loadavg": "4.44 3.26 2.29 3/405 11480", + "spawn_s": 1.1 + }, + "cold": { + "runs_ms": [ + 137.676, + 146.008, + 168.373, + 156.672, + 136.687 + ], + "median_ms": 146.008, + "p95_ms": 168.373, + "rss_baseline_kb": 18576, + "rss_peak_kb": 28812, + "hwm_reset": true, + "output": { + "digest": "931ff5c54c2c\u2026" + }, + "mem_available_mb": 669, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 0, + "length": 10485760, + "cache": "cold", + "loadavg": "4.44 3.26 2.29 3/406 11502", + "spawn_s": 1.0 + } + } + }, + "parity_all_arms_agree": true + }, + { + "file": "bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "context": { + "loadavg": "4.44 3.26 2.29 4/405 11502", + "mem_available_mb": 654, + "ts": "07:50:53" + }, + "arms": { + "py_current_double_read": { + "warm": { + "runs_ms": [ + 21.342, + 23.661, + 20.168, + 23.032, + 25.976 + ], + "median_ms": 23.032, + "p95_ms": 25.976, + "rss_baseline_kb": 18576, + "rss_peak_kb": 25160, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 654, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "warm", + "loadavg": "4.44 3.26 2.29 8/406 11506", + "spawn_s": 0.4 + }, + "cold": { + "runs_ms": [ + 62.291, + 22.345, + 25.187, + 23.736, + 26.56 + ], + "median_ms": 25.187, + "p95_ms": 62.291, + "rss_baseline_kb": 18612, + "rss_peak_kb": 25200, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 638, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "cold", + "loadavg": "4.44 3.26 2.29 7/406 11507", + "spawn_s": 0.3 + } + }, + "py_single_read": { + "warm": { + "runs_ms": [ + 16.052, + 14.263, + 18.239, + 18.365, + 14.992 + ], + "median_ms": 16.052, + "p95_ms": 18.365, + "rss_baseline_kb": 18580, + "rss_peak_kb": 21864, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 652, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "warm", + "loadavg": "4.44 3.26 2.29 2/404 11508", + "spawn_s": 0.3 + }, + "cold": { + "runs_ms": [ + 34.61, + 18.371, + 19.464, + 20.033, + 18.126 + ], + "median_ms": 19.464, + "p95_ms": 34.61, + "rss_baseline_kb": 18564, + "rss_peak_kb": 21848, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 652, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "cold", + "loadavg": "4.44 3.26 2.29 4/404 11512", + "spawn_s": 0.3 + } + }, + "py_streaming": { + "warm": { + "runs_ms": [ + 19.132, + 19.15, + 17.044, + 15.399, + 17.579 + ], + "median_ms": 17.579, + "p95_ms": 19.15, + "rss_baseline_kb": 18580, + "rss_peak_kb": 20624, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 651, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "warm", + "loadavg": "4.44 3.26 2.29 3/406 11515", + "spawn_s": 0.3 + }, + "cold": { + "runs_ms": [ + 21.918, + 21.413, + 24.627, + 24.329, + 19.384 + ], + "median_ms": 21.918, + "p95_ms": 24.627, + "rss_baseline_kb": 18580, + "rss_peak_kb": 20624, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 640, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "cold", + "loadavg": "4.44 3.26 2.29 7/406 11516", + "spawn_s": 0.3 + } + }, + "c_pread": { + "warm": { + "runs_ms": [ + 36.623, + 42.756, + 38.556, + 30.69, + 33.649 + ], + "median_ms": 36.623, + "p95_ms": 42.756, + "rss_baseline_kb": 18628, + "rss_peak_kb": 19720, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 651, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "warm", + "loadavg": "4.33 3.26 2.29 3/409 11530", + "spawn_s": 0.4 + }, + "cold": { + "runs_ms": [ + 34.026, + 39.515, + 47.226, + 48.17, + 50.182 + ], + "median_ms": 47.226, + "p95_ms": 50.182, + "rss_baseline_kb": 18628, + "rss_peak_kb": 19720, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 656, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "cold", + "loadavg": "4.33 3.26 2.29 3/409 11534", + "spawn_s": 0.4 + } + }, + "c_mmap": { + "warm": { + "runs_ms": [ + 31.494, + 30.678, + 31.272, + 29.971, + 31.192 + ], + "median_ms": 31.192, + "p95_ms": 31.494, + "rss_baseline_kb": 18596, + "rss_peak_kb": 21060, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 634, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "warm", + "loadavg": "4.33 3.26 2.29 6/407 11535", + "spawn_s": 0.3 + }, + "cold": { + "runs_ms": [ + 43.727, + 47.774, + 42.626, + 43.942, + 40.783 + ], + "median_ms": 43.727, + "p95_ms": 47.774, + "rss_baseline_kb": 18592, + "rss_peak_kb": 21148, + "hwm_reset": true, + "output": { + "digest": "80094fad4cb1\u2026" + }, + "mem_available_mb": 656, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_10mb.bin", + "offset": 5243003, + "length": 2621440, + "cache": "cold", + "loadavg": "4.33 3.26 2.29 2/404 11547", + "spawn_s": 0.4 + } + } + }, + "parity_all_arms_agree": true + }, + { + "file": "bin_100mb.bin", + "offset": 0, + "length": 104857600, + "context": { + "loadavg": "4.33 3.26 2.29 1/403 11547", + "mem_available_mb": 651, + "ts": "07:50:57" + }, + "arms": { + "py_current_double_read": { + "warm": { + "runs_ms": [ + 825.94, + 850.238, + 864.238, + 865.248, + 757.353 + ], + "median_ms": 850.238, + "p95_ms": 865.248, + "rss_baseline_kb": 18580, + "rss_peak_kb": 224564, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 714, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "warm", + "loadavg": "4.66 3.37 2.34 3/406 11633", + "spawn_s": 9.9 + }, + "cold": { + "runs_ms": [ + 1066.366, + 864.362, + 848.142, + 1074.401, + 985.812 + ], + "median_ms": 985.812, + "p95_ms": 1074.401, + "rss_baseline_kb": 18688, + "rss_peak_kb": 224864, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 688, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "cold", + "loadavg": "4.45 3.34 2.34 4/406 11675", + "spawn_s": 5.7 + } + }, + "py_single_read": { + "warm": { + "runs_ms": [ + 757.101, + 658.07, + 757.508, + 705.885, + 735.785 + ], + "median_ms": 735.785, + "p95_ms": 757.508, + "rss_baseline_kb": 18608, + "rss_peak_kb": 121692, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 766, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "warm", + "loadavg": "4.25 3.32 2.33 5/406 11722", + "spawn_s": 5.2 + }, + "cold": { + "runs_ms": [ + 874.851, + 771.273, + 787.703, + 741.416, + 708.523 + ], + "median_ms": 771.273, + "p95_ms": 874.851, + "rss_baseline_kb": 18572, + "rss_peak_kb": 121684, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 679, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "cold", + "loadavg": "4.15 3.32 2.34 2/406 11769", + "spawn_s": 4.8 + } + }, + "py_streaming": { + "warm": { + "runs_ms": [ + 676.248, + 629.999, + 649.502, + 673.198, + 733.593 + ], + "median_ms": 673.198, + "p95_ms": 733.593, + "rss_baseline_kb": 18580, + "rss_peak_kb": 20604, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 735, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "warm", + "loadavg": "3.90 3.28 2.33 3/403 11813", + "spawn_s": 4.9 + }, + "cold": { + "runs_ms": [ + 719.566, + 835.057, + 1034.632, + 869.96, + 820.248 + ], + "median_ms": 835.057, + "p95_ms": 1034.632, + "rss_baseline_kb": 18564, + "rss_peak_kb": 20624, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 691, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "cold", + "loadavg": "3.83 3.27 2.34 5/406 11869", + "spawn_s": 5.2 + } + }, + "c_pread": { + "warm": { + "runs_ms": [ + 1554.383, + 2028.179, + 1691.747, + 1743.623, + 1760.749 + ], + "median_ms": 1743.623, + "p95_ms": 2028.179, + "rss_baseline_kb": 18528, + "rss_peak_kb": 19620, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 729, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "warm", + "loadavg": "3.62 3.25 2.34 3/408 11964", + "spawn_s": 11.8 + }, + "cold": { + "runs_ms": [ + 1640.432, + 1504.216, + 1792.041, + 1790.018, + 1715.502 + ], + "median_ms": 1715.502, + "p95_ms": 1792.041, + "rss_baseline_kb": 18600, + "rss_peak_kb": 19692, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 776, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "cold", + "loadavg": "4.42 3.44 2.41 3/407 12059", + "spawn_s": 9.6 + } + }, + "c_mmap": { + "warm": { + "runs_ms": [ + 1624.465, + 1784.645, + 1371.871, + 1422.173, + 1669.946 + ], + "median_ms": 1624.465, + "p95_ms": 1784.645, + "rss_baseline_kb": 18596, + "rss_peak_kb": 120968, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 706, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "warm", + "loadavg": "4.36 3.45 2.43 4/405 12148", + "spawn_s": 10.7 + }, + "cold": { + "runs_ms": [ + 1783.063, + 1814.961, + 1781.894, + 1646.384, + 1979.154 + ], + "median_ms": 1783.063, + "p95_ms": 1979.154, + "rss_baseline_kb": 18592, + "rss_peak_kb": 120964, + "hwm_reset": true, + "output": { + "digest": "124c78b11c25\u2026" + }, + "mem_available_mb": 710, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 0, + "length": 104857600, + "cache": "cold", + "loadavg": "4.31 3.47 2.44 4/406 12238", + "spawn_s": 10.1 + } + } + }, + "parity_all_arms_agree": true + }, + { + "file": "bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "context": { + "loadavg": "4.31 3.47 2.44 4/407 12240", + "mem_available_mb": 717, + "ts": "07:52:14" + }, + "arms": { + "py_current_double_read": { + "warm": { + "runs_ms": [ + 329.638, + 219.537, + 238.602, + 213.939, + 196.271 + ], + "median_ms": 219.537, + "p95_ms": 329.638, + "rss_baseline_kb": 18604, + "rss_peak_kb": 71272, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 686, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "warm", + "loadavg": "4.37 3.50 2.46 6/405 12279", + "spawn_s": 2.4 + }, + "cold": { + "runs_ms": [ + 278.553, + 235.919, + 224.186, + 230.722, + 254.206 + ], + "median_ms": 235.919, + "p95_ms": 278.553, + "rss_baseline_kb": 18580, + "rss_peak_kb": 71244, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 643, + "bench": "b1", + "arm": "py_current_double_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "cold", + "loadavg": "4.37 3.50 2.46 3/405 12288", + "spawn_s": 1.6 + } + }, + "py_single_read": { + "warm": { + "runs_ms": [ + 152.306, + 157.558, + 169.132, + 157.038, + 146.355 + ], + "median_ms": 157.038, + "p95_ms": 169.132, + "rss_baseline_kb": 18556, + "rss_peak_kb": 44876, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 687, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "warm", + "loadavg": "4.37 3.50 2.46 4/403 12292", + "spawn_s": 1.3 + }, + "cold": { + "runs_ms": [ + 217.599, + 232.446, + 237.3, + 248.373, + 220.077 + ], + "median_ms": 232.446, + "p95_ms": 248.373, + "rss_baseline_kb": 18576, + "rss_peak_kb": 44896, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 701, + "bench": "b1", + "arm": "py_single_read", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "cold", + "loadavg": "4.34 3.51 2.47 4/407 12312", + "spawn_s": 1.5 + } + }, + "py_streaming": { + "warm": { + "runs_ms": [ + 145.578, + 146.757, + 138.387, + 181.37, + 335.665 + ], + "median_ms": 146.757, + "p95_ms": 335.665, + "rss_baseline_kb": 18568, + "rss_peak_kb": 20612, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 647, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "warm", + "loadavg": "4.34 3.51 2.47 3/405 12333", + "spawn_s": 1.5 + }, + "cold": { + "runs_ms": [ + 190.61, + 184.736, + 180.44, + 179.82, + 169.044 + ], + "median_ms": 180.44, + "p95_ms": 190.61, + "rss_baseline_kb": 18568, + "rss_peak_kb": 20612, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 635, + "bench": "b1", + "arm": "py_streaming", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "cold", + "loadavg": "4.34 3.51 2.47 5/402 12337", + "spawn_s": 1.2 + } + }, + "c_pread": { + "warm": { + "runs_ms": [ + 449.592, + 324.69, + 401.974, + 356.331, + 345.504 + ], + "median_ms": 356.331, + "p95_ms": 449.592, + "rss_baseline_kb": 18592, + "rss_peak_kb": 19684, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 669, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "warm", + "loadavg": "4.31 3.52 2.47 4/407 12374", + "spawn_s": 2.6 + }, + "cold": { + "runs_ms": [ + 410.924, + 389.582, + 353.353, + 346.969, + 383.154 + ], + "median_ms": 383.154, + "p95_ms": 410.924, + "rss_baseline_kb": 18552, + "rss_peak_kb": 19644, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 669, + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "cold", + "loadavg": "4.31 3.52 2.47 7/405 12385", + "spawn_s": 2.3 + } + }, + "c_mmap": { + "warm": { + "runs_ms": [ + 371.84, + 423.919, + 394.926, + 503.12, + 359.835 + ], + "median_ms": 394.926, + "p95_ms": 503.12, + "rss_baseline_kb": 18596, + "rss_peak_kb": 44176, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 657, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "warm", + "loadavg": "4.44 3.56 2.49 3/406 12419", + "spawn_s": 2.7 + }, + "cold": { + "runs_ms": [ + 436.556, + 453.656, + 428.2, + 474.656, + 432.742 + ], + "median_ms": 436.556, + "p95_ms": 474.656, + "rss_baseline_kb": 18584, + "rss_peak_kb": 44160, + "hwm_reset": true, + "output": { + "digest": "0e08f73acfac\u2026" + }, + "mem_available_mb": 660, + "bench": "b1", + "arm": "c_mmap", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_100mb.bin", + "offset": 52428923, + "length": 26214400, + "cache": "cold", + "loadavg": "4.44 3.56 2.49 2/403 12430", + "spawn_s": 2.8 + } + } + }, + "parity_all_arms_agree": true + } + ], + "parity": { + "all_arms_agree": true + } + }, + "b2_jsonl_scan": { + "files": [ + { + "file": "events_10mb.jsonl", + "context": { + "loadavg": "4.44 3.56 2.49 1/402 12430", + "mem_available_mb": 716, + "ts": "07:52:34" + }, + "arms": { + "py_stream_real": { + "warm": { + "runs_ms": [ + 204.002, + 122.199, + 146.909, + 116.543, + 117.641 + ], + "median_ms": 122.199, + "p95_ms": 204.002, + "rss_baseline_kb": 18576, + "rss_peak_kb": 25792, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": 5339, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 715, + "bench": "b2", + "arm": "py_stream_real", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.25 3.53 2.49 2/405 12436", + "spawn_s": 1.3 + }, + "cold": { + "runs_ms": [ + 188.112, + 118.23, + 128.735, + 122.726, + 122.718 + ], + "median_ms": 122.726, + "p95_ms": 188.112, + "rss_baseline_kb": 18580, + "rss_peak_kb": 25792, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": 5339, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 740, + "bench": "b2", + "arm": "py_stream_real", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.25 3.53 2.49 2/404 12447", + "spawn_s": 0.8 + } + }, + "py_inspect_full": { + "warm": { + "runs_ms": [ + 176.446, + 190.875, + 178.629, + 185.046, + 175.437 + ], + "median_ms": 178.629, + "p95_ms": 190.875, + "rss_baseline_kb": 20752, + "rss_peak_kb": 25788, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": 5339, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ], + "checksum": "2b540649918a\u2026" + }, + "mem_available_mb": 729, + "bench": "b2", + "arm": "py_inspect_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.25 3.53 2.49 1/403 12452", + "spawn_s": 1.3 + }, + "cold": { + "runs_ms": [ + 178.77, + 176.908, + 187.25, + 180.545, + 180.861 + ], + "median_ms": 180.545, + "p95_ms": 187.25, + "rss_baseline_kb": 20756, + "rss_peak_kb": 25792, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": 5339, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ], + "checksum": "2b540649918a\u2026" + }, + "mem_available_mb": 691, + "bench": "b2", + "arm": "py_inspect_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.25 3.53 2.49 2/405 12458", + "spawn_s": 1.1 + } + }, + "py_scan_only": { + "warm": { + "runs_ms": [ + 22.577, + 24.419, + 22.224, + 22.563, + 23.447 + ], + "median_ms": 22.577, + "p95_ms": 24.419, + "rss_baseline_kb": 18560, + "rss_peak_kb": 23692, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 676, + "bench": "b2", + "arm": "py_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.25 3.53 2.49 2/405 12459", + "spawn_s": 0.3 + }, + "cold": { + "runs_ms": [ + 34.37, + 32.466, + 32.324, + 29.156, + 29.175 + ], + "median_ms": 32.324, + "p95_ms": 34.37, + "rss_baseline_kb": 18584, + "rss_peak_kb": 23716, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 657, + "bench": "b2", + "arm": "py_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.25 3.53 2.49 2/406 12461", + "spawn_s": 0.3 + } + }, + "c_scan_full": { + "warm": { + "runs_ms": [ + 33.828, + 34.059, + 33.597, + 33.003, + 33.778 + ], + "median_ms": 33.778, + "p95_ms": 34.059, + "rss_baseline_kb": 18560, + "rss_peak_kb": 19608, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": 5339, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 648, + "bench": "b2", + "arm": "c_scan_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.25 3.53 2.49 1/403 12464", + "spawn_s": 0.3 + }, + "cold": { + "runs_ms": [ + 40.454, + 39.659, + 39.304, + 40.441, + 39.166 + ], + "median_ms": 39.659, + "p95_ms": 40.454, + "rss_baseline_kb": 18576, + "rss_peak_kb": 19624, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": 5339, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 659, + "bench": "b2", + "arm": "c_scan_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.25 3.53 2.49 1/403 12465", + "spawn_s": 0.3 + } + }, + "c_scan_only": { + "warm": { + "runs_ms": [ + 2.947, + 2.711, + 2.641, + 2.66, + 2.831 + ], + "median_ms": 2.711, + "p95_ms": 2.947, + "rss_baseline_kb": 18580, + "rss_peak_kb": 19624, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 656, + "bench": "b2", + "arm": "c_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.25 3.53 2.49 1/403 12466", + "spawn_s": 0.1 + }, + "cold": { + "runs_ms": [ + 18.03, + 16.81, + 17.358, + 16.809, + 17.282 + ], + "median_ms": 17.282, + "p95_ms": 18.03, + "rss_baseline_kb": 18560, + "rss_peak_kb": 19604, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 656, + "bench": "b2", + "arm": "c_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.25 3.53 2.49 1/403 12467", + "spawn_s": 0.2 + } + }, + "wasm_scan": { + "warm": { + "runs_ms": [ + 100.991, + 96.695, + 112.885, + 118.804, + 136.818 + ], + "median_ms": 112.885, + "p95_ms": 136.818, + "rss_baseline_kb": 37056, + "rss_peak_kb": 42828, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 692, + "bench": "b2", + "arm": "wasm_scan", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.99 3.49 2.48 2/413 12481", + "spawn_s": 0.9 + }, + "cold": { + "runs_ms": [ + 178.657, + 103.778, + 100.88, + 99.346, + 99.332 + ], + "median_ms": 100.88, + "p95_ms": 178.657, + "rss_baseline_kb": 37172, + "rss_peak_kb": 44416, + "hwm_reset": true, + "output": { + "rows": 8894, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1183 + ], + [ + 1183, + 1132 + ], + [ + 2315, + 1141 + ], + [ + 3456, + 1122 + ], + [ + 4578, + 1179 + ] + ] + }, + "mem_available_mb": 688, + "bench": "b2", + "arm": "wasm_scan", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "3.99 3.49 2.48 1/410 12493", + "spawn_s": 0.8 + } + } + }, + "parity_rows_all_arms": true, + "parity_first_bad_index": true, + "parity_scan_samples": true + }, + { + "file": "events_50mb.jsonl", + "context": { + "loadavg": "3.99 3.49 2.48 1/401 12493", + "mem_available_mb": 711, + "ts": "07:52:42" + }, + "arms": { + "py_stream_real": { + "warm": { + "runs_ms": [ + 907.678, + 1137.77, + 1508.824, + 1184.668, + 1208.806 + ], + "median_ms": 1184.668, + "p95_ms": 1508.824, + "rss_baseline_kb": 18580, + "rss_peak_kb": 25800, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": 26678, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 704, + "bench": "b2", + "arm": "py_stream_real", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.23 3.55 2.51 7/404 12518", + "spawn_s": 6.8 + }, + "cold": { + "runs_ms": [ + 1189.921, + 792.446, + 1137.25, + 702.261, + 745.154 + ], + "median_ms": 792.446, + "p95_ms": 1189.921, + "rss_baseline_kb": 18580, + "rss_peak_kb": 25680, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": 26678, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 670, + "bench": "b2", + "arm": "py_stream_real", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.05 3.52 2.50 2/404 12527", + "spawn_s": 4.8 + } + }, + "py_inspect_full": { + "warm": { + "runs_ms": [ + 885.535, + 902.179, + 833.517, + 1038.728, + 860.167 + ], + "median_ms": 885.535, + "p95_ms": 1038.728, + "rss_baseline_kb": 20740, + "rss_peak_kb": 25784, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": 26678, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ], + "checksum": "8d6b43def019\u2026" + }, + "mem_available_mb": 707, + "bench": "b2", + "arm": "py_inspect_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.97 3.51 2.51 2/404 12555", + "spawn_s": 5.6 + }, + "cold": { + "runs_ms": [ + 928.868, + 970.601, + 1067.055, + 921.061, + 1669.372 + ], + "median_ms": 970.601, + "p95_ms": 1669.372, + "rss_baseline_kb": 20788, + "rss_peak_kb": 25832, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": 26678, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ], + "checksum": "8d6b43def019\u2026" + }, + "mem_available_mb": 662, + "bench": "b2", + "arm": "py_inspect_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "3.81 3.49 2.50 8/407 12589", + "spawn_s": 5.7 + } + }, + "py_scan_only": { + "warm": { + "runs_ms": [ + 195.428, + 240.277, + 158.955, + 154.615, + 207.345 + ], + "median_ms": 195.428, + "p95_ms": 240.277, + "rss_baseline_kb": 18564, + "rss_peak_kb": 23696, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 672, + "bench": "b2", + "arm": "py_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.66 3.46 2.50 2/404 12607", + "spawn_s": 1.2 + }, + "cold": { + "runs_ms": [ + 192.962, + 146.879, + 143.983, + 145.0, + 176.288 + ], + "median_ms": 146.879, + "p95_ms": 192.962, + "rss_baseline_kb": 18572, + "rss_peak_kb": 23704, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 680, + "bench": "b2", + "arm": "py_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "3.66 3.46 2.50 2/402 12611", + "spawn_s": 1.0 + } + }, + "c_scan_full": { + "warm": { + "runs_ms": [ + 262.416, + 278.607, + 205.247, + 251.008, + 318.527 + ], + "median_ms": 262.416, + "p95_ms": 318.527, + "rss_baseline_kb": 18588, + "rss_peak_kb": 19636, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": 26678, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 691, + "bench": "b2", + "arm": "c_scan_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.66 3.46 2.50 3/405 12621", + "spawn_s": 1.7 + }, + "cold": { + "runs_ms": [ + 209.38, + 250.31, + 303.174, + 243.406, + 218.201 + ], + "median_ms": 243.406, + "p95_ms": 303.174, + "rss_baseline_kb": 18684, + "rss_peak_kb": 19732, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": 26678, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 719, + "bench": "b2", + "arm": "c_scan_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "3.66 3.46 2.50 2/408 12641", + "spawn_s": 1.4 + } + }, + "c_scan_only": { + "warm": { + "runs_ms": [ + 17.219, + 21.456, + 15.767, + 30.011, + 35.814 + ], + "median_ms": 21.456, + "p95_ms": 35.814, + "rss_baseline_kb": 18568, + "rss_peak_kb": 19612, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 699, + "bench": "b2", + "arm": "c_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.53 3.44 2.50 9/405 12654", + "spawn_s": 0.3 + }, + "cold": { + "runs_ms": [ + 64.499, + 58.761, + 53.875, + 57.688, + 74.934 + ], + "median_ms": 58.761, + "p95_ms": 74.934, + "rss_baseline_kb": 18684, + "rss_peak_kb": 19728, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 688, + "bench": "b2", + "arm": "c_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "3.53 3.44 2.50 3/405 12656", + "spawn_s": 0.5 + } + }, + "wasm_scan": { + "warm": { + "runs_ms": [ + 664.09, + 678.488, + 748.93, + 671.955, + 615.295 + ], + "median_ms": 671.955, + "p95_ms": 748.93, + "rss_baseline_kb": 36848, + "rss_peak_kb": 45340, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 710, + "bench": "b2", + "arm": "wasm_scan", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.41 3.41 2.50 3/414 12709", + "spawn_s": 4.5 + }, + "cold": { + "runs_ms": [ + 669.809, + 628.638, + 655.347, + 761.348, + 666.808 + ], + "median_ms": 666.808, + "p95_ms": 761.348, + "rss_baseline_kb": 36976, + "rss_peak_kb": 46268, + "hwm_reset": true, + "output": { + "rows": 44464, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1139 + ], + [ + 1139, + 1184 + ], + [ + 2323, + 1164 + ], + [ + 3487, + 1175 + ], + [ + 4662, + 1200 + ] + ] + }, + "mem_available_mb": 671, + "bench": "b2", + "arm": "wasm_scan", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_50mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "3.41 3.41 2.50 1/411 12735", + "spawn_s": 3.7 + } + } + }, + "parity_rows_all_arms": true, + "parity_first_bad_index": true, + "parity_scan_samples": true + }, + { + "file": "events_100mb.jsonl", + "context": { + "loadavg": "3.41 3.41 2.50 1/402 12735", + "mem_available_mb": 671, + "ts": "07:53:19" + }, + "arms": { + "py_stream_real": { + "warm": { + "runs_ms": [ + 1694.74, + 1749.287, + 1737.561, + 1752.181, + 1739.853 + ], + "median_ms": 1739.853, + "p95_ms": 1752.181, + "rss_baseline_kb": 18556, + "rss_peak_kb": 25772, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": 53346, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 672, + "bench": "b2", + "arm": "py_stream_real", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.34 3.40 2.50 3/408 12842", + "spawn_s": 10.8 + }, + "cold": { + "runs_ms": [ + 2686.488, + 1647.186, + 1630.501, + 2258.604, + 3490.135 + ], + "median_ms": 2258.604, + "p95_ms": 3490.135, + "rss_baseline_kb": 18540, + "rss_peak_kb": 25756, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": 53346, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 654, + "bench": "b2", + "arm": "py_stream_real", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "3.18 3.36 2.50 3/405 12900", + "spawn_s": 11.9 + } + }, + "py_inspect_full": { + "warm": { + "runs_ms": [ + 2644.5, + 2501.683, + 1965.872, + 1924.581, + 2118.742 + ], + "median_ms": 2118.742, + "p95_ms": 2644.5, + "rss_baseline_kb": 20792, + "rss_peak_kb": 25828, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": 53346, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ], + "checksum": "15565bfdb04c\u2026" + }, + "mem_available_mb": 699, + "bench": "b2", + "arm": "py_inspect_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "2.85 3.28 2.49 1/403 12952", + "spawn_s": 14.3 + }, + "cold": { + "runs_ms": [ + 2550.029, + 2338.268, + 3783.562, + 2716.934, + 2695.347 + ], + "median_ms": 2695.347, + "p95_ms": 3783.562, + "rss_baseline_kb": 20792, + "rss_peak_kb": 25828, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": 53346, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ], + "checksum": "15565bfdb04c\u2026" + }, + "mem_available_mb": 697, + "bench": "b2", + "arm": "py_inspect_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "2.81 3.25 2.49 1/404 13065", + "spawn_s": 14.3 + } + }, + "py_scan_only": { + "warm": { + "runs_ms": [ + 362.855, + 366.743, + 304.244, + 318.567, + 299.821 + ], + "median_ms": 318.567, + "p95_ms": 366.743, + "rss_baseline_kb": 18516, + "rss_peak_kb": 23644, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 681, + "bench": "b2", + "arm": "py_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "2.81 3.25 2.49 3/405 13075", + "spawn_s": 2.2 + }, + "cold": { + "runs_ms": [ + 394.692, + 459.312, + 464.979, + 391.477, + 377.139 + ], + "median_ms": 394.692, + "p95_ms": 464.979, + "rss_baseline_kb": 18568, + "rss_peak_kb": 23696, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 701, + "bench": "b2", + "arm": "py_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "2.81 3.25 2.49 3/409 13099", + "spawn_s": 2.3 + } + }, + "c_scan_full": { + "warm": { + "runs_ms": [ + 678.138, + 464.114, + 538.163, + 482.132, + 489.469 + ], + "median_ms": 489.469, + "p95_ms": 678.138, + "rss_baseline_kb": 18580, + "rss_peak_kb": 19628, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": 53346, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 740, + "bench": "b2", + "arm": "c_scan_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.39 3.36 2.53 3/406 13123", + "spawn_s": 3.3 + }, + "cold": { + "runs_ms": [ + 843.813, + 567.419, + 524.18, + 784.323, + 542.218 + ], + "median_ms": 567.419, + "p95_ms": 843.813, + "rss_baseline_kb": 18560, + "rss_peak_kb": 19608, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": 53346, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 715, + "bench": "b2", + "arm": "c_scan_full", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.00 3.49 2.58 9/405 13160", + "spawn_s": 3.4 + } + }, + "c_scan_only": { + "warm": { + "runs_ms": [ + 33.242, + 36.328, + 31.62, + 37.842, + 35.977 + ], + "median_ms": 35.977, + "p95_ms": 37.842, + "rss_baseline_kb": 18536, + "rss_peak_kb": 19580, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 767, + "bench": "b2", + "arm": "c_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.00 3.49 2.58 9/406 13165", + "spawn_s": 0.4 + }, + "cold": { + "runs_ms": [ + 168.698, + 224.36, + 150.506, + 143.081, + 176.543 + ], + "median_ms": 168.698, + "p95_ms": 224.36, + "rss_baseline_kb": 18688, + "rss_peak_kb": 19732, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 743, + "bench": "b2", + "arm": "c_scan_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.00 3.49 2.58 4/409 13172", + "spawn_s": 1.0 + } + }, + "wasm_scan": { + "warm": { + "runs_ms": [ + 1325.031, + 1247.178, + 1959.962, + 1944.966, + 1667.011 + ], + "median_ms": 1667.011, + "p95_ms": 1959.962, + "rss_baseline_kb": 36992, + "rss_peak_kb": 45408, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 709, + "bench": "b2", + "arm": "wasm_scan", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.53 3.62 2.63 2/414 13264", + "spawn_s": 10.1 + }, + "cold": { + "runs_ms": [ + 1146.075, + 1111.321, + 1089.311, + 1118.918, + 1502.447 + ], + "median_ms": 1118.918, + "p95_ms": 1502.447, + "rss_baseline_kb": 37136, + "rss_peak_kb": 46528, + "hwm_reset": true, + "output": { + "rows": 88914, + "first_bad_index": null, + "first_n": [ + [ + 0, + 1160 + ], + [ + 1160, + 1183 + ], + [ + 2343, + 1121 + ], + [ + 3464, + 1228 + ], + [ + 4692, + 1236 + ] + ] + }, + "mem_available_mb": 673, + "bench": "b2", + "arm": "wasm_scan", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_100mb.jsonl", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.25 3.58 2.62 2/414 13321", + "spawn_s": 6.2 + } + } + }, + "parity_rows_all_arms": true, + "parity_first_bad_index": true, + "parity_scan_samples": true + } + ], + "parity": { + "rows_all_arms": true, + "first_bad_index": true, + "scan_samples": true + } + }, + "b3_bundle_profile": { + "trees": [ + { + "tree": "bundles_100", + "context": { + "loadavg": "4.25 3.58 2.62 3/405 13321", + "mem_available_mb": 652, + "ts": "07:54:39" + }, + "arms": { + "parse_e2e": { + "warm": { + "runs_ms": [ + 1174.006, + 1011.766 + ], + "median_ms": 1092.886, + "p95_ms": 1174.006, + "rss_baseline_kb": 22764, + "rss_peak_kb": 22884, + "hwm_reset": true, + "output": { + "records": 1100, + "warnings": 0, + "bundles": 100 + }, + "mem_available_mb": 664, + "bench": "b3", + "arm": "parse_e2e", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bundles_100", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.23 3.58 2.63 6/405 13336", + "spawn_s": 5.3 + } + }, + "embed_only": { + "warm": { + "runs_ms": [ + 908.965, + 803.417 + ], + "median_ms": 856.191, + "p95_ms": 908.965, + "rss_baseline_kb": 19736, + "rss_peak_kb": 19828, + "hwm_reset": true, + "output": { + "vectors": 57600, + "texts": 900 + }, + "mem_available_mb": 682, + "bench": "b3", + "arm": "embed_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bundles_100", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.37 3.62 2.65 2/406 13351", + "spawn_s": 2.6 + } + } + } + }, + { + "tree": "bundles_1000", + "context": { + "loadavg": "4.37 3.62 2.65 2/405 13351", + "mem_available_mb": 688, + "ts": "07:54:47" + }, + "arms": { + "parse_e2e": { + "warm": { + "runs_ms": [ + 23439.647, + 16637.06 + ], + "median_ms": 20038.353, + "p95_ms": 23439.647, + "rss_baseline_kb": 23820, + "rss_peak_kb": 24080, + "hwm_reset": true, + "output": { + "records": 11000, + "warnings": 0, + "bundles": 1000 + }, + "mem_available_mb": 691, + "bench": "b3", + "arm": "parse_e2e", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bundles_1000", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.43 3.82 2.80 5/406 14064", + "spawn_s": 80.9 + } + }, + "embed_only": { + "warm": { + "runs_ms": [ + 10555.024, + 9260.901 + ], + "median_ms": 9907.962, + "p95_ms": 10555.024, + "rss_baseline_kb": 22576, + "rss_peak_kb": 22672, + "hwm_reset": true, + "output": { + "vectors": 576000, + "texts": 9000 + }, + "mem_available_mb": 662, + "bench": "b3", + "arm": "embed_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bundles_1000", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "5.92 4.25 2.98 7/408 14421", + "spawn_s": 30.0 + } + } + } + }, + { + "tree": "bundles_2500", + "context": { + "loadavg": "5.92 4.25 2.98 10/407 14421", + "mem_available_mb": 672, + "ts": "07:56:38" + }, + "arms": { + "parse_e2e": { + "warm": { + "runs_ms": [ + 36231.033, + 38817.077 + ], + "median_ms": 37524.055, + "p95_ms": 38817.077, + "rss_baseline_kb": 25616, + "rss_peak_kb": 26280, + "hwm_reset": true, + "output": { + "records": 27500, + "warnings": 0, + "bundles": 2500 + }, + "mem_available_mb": 683, + "bench": "b3", + "arm": "parse_e2e", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bundles_2500", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.77 4.50 3.28 6/403 15840", + "spawn_s": 156.1 + } + }, + "embed_only": { + "warm": { + "runs_ms": [ + 14050.829, + 13979.705 + ], + "median_ms": 14015.267, + "p95_ms": 14050.829, + "rss_baseline_kb": 27504, + "rss_peak_kb": 27596, + "hwm_reset": true, + "output": { + "vectors": 1440000, + "texts": 22500 + }, + "mem_available_mb": 686, + "bench": "b3", + "arm": "embed_only", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bundles_2500", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "3.33 4.15 3.22 5/404 16166", + "spawn_s": 49.2 + } + } + } + } + ] + }, + "b4_manifest_verify": { + "arms": { + "py_hashlib": { + "warm": { + "runs_ms": [ + 1965.489, + 2003.521, + 1906.243, + 1867.164, + 2010.8 + ], + "median_ms": 1965.489, + "p95_ms": 2010.8, + "rss_baseline_kb": 18572, + "rss_peak_kb": 18700, + "hwm_reset": true, + "output": { + "files_verified": 8 + }, + "mem_available_mb": 681, + "bench": "b4", + "arm": "py_hashlib", + "path": "", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.31 4.31 3.29 10/407 16371", + "spawn_s": 20.9 + }, + "cold": { + "runs_ms": [ + 9563.411, + 5660.383, + 3149.601, + 3496.666, + 2566.368 + ], + "median_ms": 3496.666, + "p95_ms": 9563.411, + "rss_baseline_kb": 18564, + "rss_peak_kb": 18520, + "hwm_reset": true, + "output": { + "files_verified": 8 + }, + "mem_available_mb": 708, + "bench": "b4", + "arm": "py_hashlib", + "path": "", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "4.99 4.48 3.38 3/405 16508", + "spawn_s": 24.8 + } + }, + "c_pread": { + "warm": { + "runs_ms": [ + 3255.805, + 3018.624, + 3277.69, + 3938.122, + 4311.656 + ], + "median_ms": 3277.69, + "p95_ms": 4311.656, + "rss_baseline_kb": 18604, + "rss_peak_kb": 19696, + "hwm_reset": true, + "output": { + "files_verified": 8 + }, + "mem_available_mb": 789, + "bench": "b4", + "arm": "c_pread", + "path": "", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "4.96 4.49 3.41 6/405 16648", + "spawn_s": 21.7 + }, + "cold": { + "runs_ms": [ + 4727.602, + 4652.961, + 4519.944, + 4153.205, + 4218.029 + ], + "median_ms": 4519.944, + "p95_ms": 4727.602, + "rss_baseline_kb": 18580, + "rss_peak_kb": 19672, + "hwm_reset": true, + "output": { + "files_verified": 8 + }, + "mem_available_mb": 704, + "bench": "b4", + "arm": "c_pread", + "path": "", + "offset": 0, + "length": 0, + "cache": "cold", + "loadavg": "5.00 4.53 3.44 2/408 16855", + "spawn_s": 22.5 + } + } + }, + "parity_files_verified": true + }, + "optional_boundary_check": { + "ok": true, + "c_pread_disabled": { + "skipped": "helper_unavailable", + "detail": "native helpers disabled (HOTMEM_SPIKE_DISABLE_NATIVE=1)", + "bench": "b1", + "arm": "c_pread", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/bin_1mb.bin", + "offset": 0, + "length": 1048576, + "cache": "warm", + "loadavg": "5.00 4.53 3.44 1/408 16856", + "spawn_s": 0.1 + }, + "wasm_scan_disabled": { + "skipped": "helper_unavailable", + "detail": "native helpers disabled (HOTMEM_SPIKE_DISABLE_NATIVE=1)", + "bench": "b2", + "arm": "wasm_scan", + "path": "/home/kenneth/projects/HotMem/bench/native_spike/corpus/events_10mb.jsonl", + "offset": 0, + "length": 0, + "cache": "warm", + "loadavg": "5.00 4.53 3.44 2/410 16859", + "spawn_s": 0.1 + } + }, + "total_s": 654.2, + "final_context": { + "loadavg": "5.00 4.53 3.44 2/409 16859", + "mem_available_mb": 705, + "ts": "08:01:34" + } +} diff --git a/bench/native_spike/run_bench.py b/bench/native_spike/run_bench.py new file mode 100644 index 0000000..3236baf --- /dev/null +++ b/bench/native_spike/run_bench.py @@ -0,0 +1,419 @@ +#!/usr/bin/env python3 +"""Benchmark orchestrator for the native helper spike (#48). + +Runs the full benchmark matrix serially, each cell in a fresh subprocess +(bench_worker.py), with memory guards, warm/cold cache modes, per-run +output self-verification, and cross-arm parity assertions. Writes +results.json and can re-render the markdown report from it. + +Usage: + python run_bench.py [--profile reduced|full] [--quick] [--out results.json] + python run_bench.py --report [--out results.json] + +Benchmarks: + b1 range SHA-256 checksum — Python current/candidate paths vs C (pread, mmap) + b2 JSONL scanning — real inspector paths vs C scanner vs WASM scanner + b3 bundle parse profile — where parse_bundle() time actually goes + b4 manifest verification — hashing all corpus files, py vs C +""" + +from __future__ import annotations + +import argparse +import json +import os +import subprocess +import sys +import time +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +import benchlib # noqa: E402 + +SPIKE_DIR = Path(__file__).resolve().parent + +B1_ARMS = ["py_current_double_read", "py_single_read", "py_streaming", "c_pread", "c_mmap"] +B2_ARMS = [ + "py_stream_real", + "py_inspect_full", + "py_scan_only", + "c_scan_full", + "c_scan_only", + "wasm_scan", +] +B3_ARMS = ["parse_e2e", "embed_only"] +B4_ARMS = ["py_hashlib", "c_pread"] + +CACHE_MODES = ["warm", "cold"] + + +def spawn(bench: str, arm: str, *, path: str = "", offset: int = 0, length: int = 0, + runs: int, cache: str, env: dict | None = None, timeout: int = 1800) -> dict: + cmd = [ + sys.executable, + str(SPIKE_DIR / "bench_worker.py"), + "--bench", bench, + "--arm", arm, + "--runs", str(runs), + "--cache", cache, + ] + if path: + cmd += ["--path", path, "--offset", str(offset), "--length", str(length)] + child_env = {**os.environ, **(env or {})} + t0 = time.time() + res = subprocess.run( + cmd, capture_output=True, text=True, timeout=timeout, env=child_env, cwd=str(SPIKE_DIR) + ) + wall = round(time.time() - t0, 1) + if res.returncode != 0: + return {"error": (res.stderr or res.stdout)[-1500:], "spawn_s": wall} + try: + out = json.loads(res.stdout.strip().splitlines()[-1]) + except (json.JSONDecodeError, IndexError): + return {"error": f"bad worker output: {res.stdout[:400]}", "spawn_s": wall} + out["spawn_s"] = wall + return out + + +def _ctx() -> dict: + return { + "loadavg": benchlib.loadavg(), + "mem_available_mb": benchlib.mem_available_mb(), + "ts": time.strftime("%H:%M:%S"), + } + + +# --------------------------------------------------------------------- # +# b1 — range checksum # +# --------------------------------------------------------------------- # + +def run_b1(manifest: dict, runs: int, quick: bool) -> dict: + cases = [] + for e in manifest["checksum_files"]: + if quick and e["size"] > (10 << 20): + continue + cases.append((e["name"], 0, e["size"])) + if e["size"] >= (10 << 20) and not quick: + # Mid-file, deliberately non-page-aligned range. + cases.append((e["name"], e["size"] // 2 + 123, e["size"] // 4)) + + out_cases = [] + all_ok = True + for name, off, ln in cases: + path = str(benchlib.CORPUS_DIR / name) + print(f" [b1] {name} [{off}, +{ln}] …", flush=True) + cell = {"file": name, "offset": off, "length": ln, "context": _ctx(), "arms": {}} + digests = {} + for arm in B1_ARMS: + modes = ["warm"] if quick else CACHE_MODES + for mode in modes: + r = spawn("b1", arm, path=path, offset=off, length=ln, runs=runs, cache=mode) + cell["arms"].setdefault(arm, {})[mode] = r + if "output" in r: + digests[arm] = r["output"].get("digest") + vals = {a: d for a, d in digests.items() if d} + cell["parity_all_arms_agree"] = len(set(vals.values())) <= 1 and len(vals) == len(digests) + all_ok &= cell["parity_all_arms_agree"] + out_cases.append(cell) + return {"cases": out_cases, "parity": {"all_arms_agree": all_ok}} + + +# --------------------------------------------------------------------- # +# b2 — JSONL scanning # +# --------------------------------------------------------------------- # + +def run_b2(manifest: dict, runs: int, quick: bool) -> dict: + files = [e["name"] for e in manifest["jsonl_files"]] + if quick: + files = [f for f in files if "10mb" in f] + + out_files = [] + rows_ok = True + bad_ok = True + scan_ok = True + for name in files: + path = str(benchlib.CORPUS_DIR / name) + print(f" [b2] {name} …", flush=True) + cell = {"file": name, "context": _ctx(), "arms": {}} + rows = {} + bads = {} + firsts = {} + for arm in B2_ARMS: + modes = ["warm"] if quick else CACHE_MODES + for mode in modes: + r = spawn("b2", arm, path=path, runs=runs, cache=mode) + cell["arms"].setdefault(arm, {})[mode] = r + if "output" in r: + rows[arm] = r["output"].get("rows") + bads[arm] = r["output"].get("first_bad_index") + firsts[arm] = tuple(map(tuple, r["output"].get("first_n") or [])) + # All arms must agree on row count. + cell["parity_rows_all_arms"] = len(set(rows.values())) <= 1 + # Arms that validate JSON must agree on the first bad line INDEX. + validator_arms = ("py_stream_real", "py_inspect_full", "c_scan_full") + validators = {a: bads[a] for a in validator_arms if a in bads} + cell["parity_first_bad_index"] = len(set(validators.values())) <= 1 + # Scan-only arms must agree on the sampled line boundaries. + scanner_arms = ("py_scan_only", "c_scan_only", "wasm_scan") + scanners = {a: firsts[a] for a in scanner_arms if a in firsts} + cell["parity_scan_samples"] = len(set(scanners.values())) <= 1 + rows_ok &= cell["parity_rows_all_arms"] + bad_ok &= cell["parity_first_bad_index"] + scan_ok &= cell["parity_scan_samples"] + out_files.append(cell) + return { + "files": out_files, + "parity": {"rows_all_arms": rows_ok, "first_bad_index": bad_ok, "scan_samples": scan_ok}, + } + + +# --------------------------------------------------------------------- # +# b3 — bundle parse profile # +# --------------------------------------------------------------------- # + +def run_b3(manifest: dict, runs: int, quick: bool) -> dict: + trees = [t["name"] for t in manifest["bundle_trees"]] + if quick: + trees = [t for t in trees if t.endswith("100")] + + out_trees = [] + for name in trees: + path = str(benchlib.CORPUS_DIR / name) + print(f" [b3] {name} …", flush=True) + cell = {"tree": name, "context": _ctx(), "arms": {}} + for arm in B3_ARMS: + r = spawn("b3", arm, path=path, runs=max(runs - 3, 2), cache="warm") + cell["arms"][arm] = {"warm": r} + out_trees.append(cell) + return {"trees": out_trees} + + +# --------------------------------------------------------------------- # +# b4 — manifest verification # +# --------------------------------------------------------------------- # + +def run_b4(runs: int, quick: bool) -> dict: + print(" [b4] manifest verification …", flush=True) + cell = {"context": _ctx(), "arms": {}} + verified = {} + for arm in B4_ARMS: + modes = ["warm"] if quick else CACHE_MODES + for mode in modes: + r = spawn("b4", arm, runs=runs, cache=mode) + cell["arms"].setdefault(arm, {})[mode] = r + if "output" in r: + verified[arm] = r["output"].get("files_verified") + cell["parity_files_verified"] = len(set(verified.values())) <= 1 and len(verified) == len( + [a for a in B4_ARMS] + ) + return {"arms": cell["arms"], "parity_files_verified": cell["parity_files_verified"]} + + +# --------------------------------------------------------------------- # +# optional-boundary check (graceful degradation) # +# --------------------------------------------------------------------- # + +def run_boundary_check() -> dict: + """Prove helpers degrade gracefully: run a c arm and a wasm arm with + HOTMEM_SPIKE_DISABLE_NATIVE=1; both must report skipped=helper_unavailable + rather than crash — the same contract a production optional helper needs.""" + print(" [boundary] native disabled fallback check …", flush=True) + env = {"HOTMEM_SPIKE_DISABLE_NATIVE": "1"} + binf = benchlib.CORPUS_DIR / "bin_1mb.bin" + jf = benchlib.CORPUS_DIR / "events_10mb.jsonl" + c1 = spawn("b1", "c_pread", path=str(binf), length=binf.stat().st_size, + runs=1, cache="warm", env=env) + c2 = spawn("b2", "wasm_scan", path=str(jf), runs=1, cache="warm", env=env) + ok = ( + c1.get("skipped") == "helper_unavailable" + and c2.get("skipped") == "helper_unavailable" + ) + return {"ok": ok, "c_pread_disabled": c1, "wasm_scan_disabled": c2} + + +# --------------------------------------------------------------------- # +# report # +# --------------------------------------------------------------------- # + +def _fmt_ms(v) -> str: + return "—" if v is None else f"{v:.1f}" + + +def _arm_cell(cell: dict, arm: str, mode: str) -> dict: + return cell["arms"].get(arm, {}).get(mode, {}) + + +def _med(cell: dict, arm: str, mode: str): + return _arm_cell(cell, arm, mode).get("median_ms") + + +def _peak(cell: dict, arm: str, mode: str): + return _arm_cell(cell, arm, mode).get("rss_peak_kb") + + +def _ratio(a, b) -> str: + if a and b: + return f"{a / b:.2f}x" + return "—" + + +def report(results: dict) -> str: + lines: list[str] = [] + + host = results.get("host", {}) + lines.append(f"Host: {host.get('cpu', '?')} | sha_ni={host.get('sha_ni')} | " + f"kernel {host.get('kernel', '?')} | Python {host.get('python', '?')}") + lines.append(f"Profile: {results.get('profile')} | runs={results.get('config', {}).get('runs')}" + f" | corpus seed={results.get('corpus_seed')}") + lines.append("") + + # ---- B1 ---- + b1 = results.get("b1_range_checksum", {}) + lines.append("## B1 — Range SHA-256 checksum (median ms, warm / cold; peak RSS MB in parens)") + lines.append("") + header = ("| file (range) | py double-read | py single | py stream | c pread | c mmap " + "| py-single gains | c_mmap vs py_single |") + lines.append(header) + lines.append("|" + "---|" * 8) + for case in b1.get("cases", []): + size = case["length"] + label = f"{case['file']} (+{size >> 20}MB @{case['offset']})" + if size < (1 << 20): + label = f"{case['file']} (+{size >> 10}KB @{case['offset']})" + cells = [] + for arm in B1_ARMS: + w = _med(case, arm, "warm") + c = _med(case, arm, "cold") + rss = _peak(case, arm, "warm") + rss_mb = f" ({rss / 1024:.0f})" if rss else "" + cells.append(f"{_fmt_ms(w)} / {_fmt_ms(c)}{rss_mb}") + single_gains = _ratio( + _med(case, "py_current_double_read", "warm"), _med(case, "py_single_read", "warm") + ) + c_vs_py = _ratio(_med(case, "c_mmap", "warm"), _med(case, "py_single_read", "warm")) + lines.append(f"| {label} | " + " | ".join(cells) + f" | {single_gains} | {c_vs_py} |") + parity = b1.get("parity", {}).get("all_arms_agree") + lines.append(f"\nParity (all arms agree on digest): **{parity}**") + lines.append("") + + # ---- B2 ---- + b2 = results.get("b2_jsonl_scan", {}) + lines.append("## B2 — JSONL scanning (median ms, warm / cold)") + lines.append("") + b2_header = ( + "| file | py real _stream | py inspect() e2e | py scan-only " + "| c scan+valid | c scan-only | wasm scan | rows |" + ) + lines.append(b2_header) + lines.append("|---|---|---|---|---|---|---|---|") + for cell in b2.get("files", []): + vals = [] + for arm in B2_ARMS: + vals.append(f"{_fmt_ms(_med(cell, arm, 'warm'))} / {_fmt_ms(_med(cell, arm, 'cold'))}") + rows = next( + (_arm_cell(cell, a, "warm").get("output", {}).get("rows") for a in B2_ARMS), + None, + ) + lines.append(f"| {cell['file']} | " + " | ".join(vals) + f" | {rows} |") + p = b2.get("parity", {}) + lines.append( + f"\nParity: rows all arms **{p.get('rows_all_arms')}** " + f"| first-bad index **{p.get('first_bad_index')}** " + f"| scan samples **{p.get('scan_samples')}**" + ) + lines.append("") + + # ---- B3 ---- + b3 = results.get("b3_bundle_profile", {}) + lines.append("## B3 — Bundle parse profile (median ms, warm)") + lines.append("") + lines.append("| tree | parse e2e | embed only | embed share |") + lines.append("|---|---|---|---|") + for tree in b3.get("trees", []): + pe = _med(tree, "parse_e2e", "warm") + eo = _med(tree, "embed_only", "warm") + share = f"{eo / pe:.0%}" if pe and eo else "—" + lines.append(f"| {tree['tree']} | {_fmt_ms(pe)} | {_fmt_ms(eo)} | {share} |") + lines.append("") + + # ---- B4 ---- + b4 = results.get("b4_manifest_verify", {}) + lines.append("## B4 — Manifest verification (median ms, warm / cold)") + lines.append("") + lines.append("| arm | warm | cold | files verified |") + lines.append("|---|---|---|---|") + for arm in B4_ARMS: + w = b4.get("arms", {}).get(arm, {}).get("warm", {}).get("median_ms") + c = b4.get("arms", {}).get(arm, {}).get("cold", {}).get("median_ms") + fv = b4.get("arms", {}).get(arm, {}).get("warm", {}).get("output", {}).get("files_verified") + lines.append(f"| {arm} | {_fmt_ms(w)} | {_fmt_ms(c)} | {fv} |") + lines.append(f"\nParity (same file set verified): **{b4.get('parity_files_verified')}**") + lines.append("") + + # ---- boundary ---- + bc = results.get("optional_boundary_check", {}) + if bc: + lines.append("## Optional-boundary check (HOTMEM_SPIKE_DISABLE_NATIVE=1): " + f"**{bc.get('ok')}**") + lines.append("") + + return "\n".join(lines) + + +# --------------------------------------------------------------------- # + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--profile", choices=["reduced", "full"], default="reduced") + ap.add_argument("--quick", action="store_true", + help="smoke the harness: 1 run, small inputs") + ap.add_argument("--out", default="results.json") + ap.add_argument("--report", action="store_true", + help="print markdown report from --out and exit") + args = ap.parse_args() + + out_path = SPIKE_DIR / args.out + + if args.report: + if not out_path.exists(): + print(f"no results at {out_path}", file=sys.stderr) + return 1 + print(report(json.loads(out_path.read_text(encoding="utf-8")))) + return 0 + + if not benchlib.MANIFEST_PATH.exists() or not benchlib.CORPUS_DIR.exists(): + hint = f"corpus missing — run: python gen_corpus.py --profile {args.profile}" + print(hint, file=sys.stderr) + return 1 + + manifest = json.loads(benchlib.MANIFEST_PATH.read_text(encoding="utf-8")) + runs = 1 if args.quick else 5 + + print(f"native helper spike benchmark — profile={args.profile}, runs={runs}") + results = { + "spike": "#48 native helper spike", + "profile": args.profile, + "quick": args.quick, + "corpus_seed": manifest.get("seed"), + "host": benchlib.host_info(), + "config": {"runs": runs, "cache_modes": ["warm"] if args.quick else CACHE_MODES}, + } + + t0 = time.time() + results["b1_range_checksum"] = run_b1(manifest, runs, args.quick) + results["b2_jsonl_scan"] = run_b2(manifest, runs, args.quick) + results["b3_bundle_profile"] = run_b3(manifest, runs, args.quick) + results["b4_manifest_verify"] = run_b4(runs, args.quick) + results["optional_boundary_check"] = run_boundary_check() + results["total_s"] = round(time.time() - t0, 1) + results["final_context"] = _ctx() + + out_path.write_text(json.dumps(results, indent=2) + "\n", encoding="utf-8") + print(f"\nWrote {out_path} in {results['total_s']}s") + print(report(results)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/bench/native_spike/wasm_parser/scanner.wat b/bench/native_spike/wasm_parser/scanner.wat new file mode 100644 index 0000000..0f9feaa --- /dev/null +++ b/bench/native_spike/wasm_parser/scanner.wat @@ -0,0 +1,122 @@ +;; WebAssembly line scanner for the native helper spike (#48). +;; +;; Scan-only contract (matches py_baseline.py_scan_only and the C scanner's +;; mode 0): count non-blank lines, capture the first N line boundaries +;; (file offset, length INCLUDING trailing newline), where eligibility is +;; line_index < sample_max and blank lines burn line_index but never +;; produce a sample. The final line without a trailing newline counts with +;; length excluding the absent newline. +;; +;; Architecture mirrors the real integration shape: the HOST (Python) owns +;; file I/O and carry management — it prepends leftover bytes to the next +;; chunk so every scan() call starts exactly at a line boundary — and the +;; module does the per-byte CPU work over linear memory. +;; +;; Layout: sample table at 0x1000 (16 slots of i64 offset/length pairs, +;; 256 bytes); host writes (carry + chunk) at 0x2000 upward — up to ~1 MiB +;; of fresh chunk bytes plus carried partial-line bytes, with the host +;; growing memory when long lines need more (initial 33 pages = 2.125 MiB). +;; +;; Offsets use correct file coordinates (see py_baseline.py note about the +;; shipped _stream offset bug). +(module + (memory (export "memory") 33) + + (global $rows (mut i64) (i64.const 0)) + (global $lines (mut i64) (i64.const 0)) + (global $sample_count (mut i32) (i32.const 0)) + (global $sample_max (mut i32) (i32.const 5)) + (global $pend_start (mut i64) (i64.const 0)) + (global $pend_nonblank (mut i32) (i32.const 0)) + (global $have_pend (mut i32) (i32.const 0)) + + ;; true when byte is line whitespace: space, \t, \r, \v, \f (not \n) + (func $ws (param $b i32) (result i32) + (i32.or (i32.eq (local.get $b) (i32.const 32)) + (i32.or (i32.eq (local.get $b) (i32.const 9)) + (i32.or (i32.eq (local.get $b) (i32.const 13)) + (i32.or (i32.eq (local.get $b) (i32.const 11)) + (i32.eq (local.get $b) (i32.const 12))))))) + + (func $eligible (result i32) + (i32.and + (i64.lt_u (global.get $lines) (i64.extend_i32_u (global.get $sample_max))) + (i32.lt_u (global.get $sample_count) (global.get $sample_max)))) + + (func $record_sample (param $start i64) (param $len i64) + (if (i32.lt_u (global.get $sample_count) (i32.const 16)) + (then + (i64.store + (i32.add (i32.const 0x1000) (i32.mul (global.get $sample_count) (i32.const 16))) + (local.get $start)) + (i64.store + (i32.add (i32.const 0x1008) (i32.mul (global.get $sample_count) (i32.const 16))) + (local.get $len)) + (global.set $sample_count (i32.add (global.get $sample_count) (i32.const 1)))))) + + ;; Scan bytes at [ptr, ptr+len), which live at absolute file offset `abs`. + ;; The first byte is always a line boundary (host carry guarantee). + (func (export "scan") (param $ptr i32) (param $len i32) (param $abs i64) + (local $i i32) + (local $b i32) + (local $nl_abs i64) + (if (i32.eqz (global.get $have_pend)) + (then + (global.set $have_pend (i32.const 1)) + (global.set $pend_start (local.get $abs)) + (global.set $pend_nonblank (i32.const 0)))) + (loop $bytes + (if (i32.lt_u (local.get $i) (local.get $len)) + (then + (local.set $b (i32.load8_u (i32.add (local.get $ptr) (local.get $i)))) + (if (i32.eq (local.get $b) (i32.const 10)) + (then + (local.set $nl_abs + (i64.add (local.get $abs) (i64.extend_i32_u (local.get $i)))) + (if (global.get $pend_nonblank) + (then + (global.set $rows (i64.add (global.get $rows) (i64.const 1))) + (if (call $eligible) + (then + (call $record_sample + (global.get $pend_start) + (i64.add (i64.sub (local.get $nl_abs) (global.get $pend_start)) + (i64.const 1))))))) + (global.set $lines (i64.add (global.get $lines) (i64.const 1))) + (global.set $pend_start (i64.add (local.get $nl_abs) (i64.const 1))) + (global.set $pend_nonblank (i32.const 0))) + (else + (if (i32.eqz (call $ws (local.get $b))) + (then (global.set $pend_nonblank (i32.const 1)))))) + (local.set $i (i32.add (local.get $i) (i32.const 1))) + (br $bytes))))) + + ;; Finalize with the total file size: the last line may lack a newline. + (func (export "finish") (param $total i64) + (if (i32.and (global.get $have_pend) (global.get $pend_nonblank)) + (then + (global.set $rows (i64.add (global.get $rows) (i64.const 1))) + (if (call $eligible) + (then + (call $record_sample + (global.get $pend_start) + (i64.sub (local.get $total) (global.get $pend_start)))))))) + + (func (export "reset") + (global.set $rows (i64.const 0)) + (global.set $lines (i64.const 0)) + (global.set $sample_count (i32.const 0)) + (global.set $pend_nonblank (i32.const 0)) + (global.set $have_pend (i32.const 0)) + (global.set $pend_start (i64.const 0))) + + (func (export "set_sample_max") (param $n i32) + (global.set $sample_max (local.get $n))) + + (func (export "get_rows") (result i64) (global.get $rows)) + (func (export "get_lines") (result i64) (global.get $lines)) + (func (export "get_sample_count") (result i32) (global.get $sample_count)) + (func (export "get_sample_offset") (param $i i32) (result i64) + (i64.load (i32.add (i32.const 0x1000) (i32.mul (local.get $i) (i32.const 16))))) + (func (export "get_sample_length") (param $i i32) (result i64) + (i64.load (i32.add (i32.const 0x1008) (i32.mul (local.get $i) (i32.const 16))))))