Summary
Fix the line-offset arithmetic bug in JSONLInspector._stream discovered by the native helper spike (#48, bench in #84).
Bug
When a line spans the 1 MiB read-chunk boundary, _stream computes line_start = offset + pos in (carry+chunk) coordinates (src/hotmem/inspectors/jsonl_inspector.py:129), overstating file offsets by len(carry) for every subsequent line.
Evidence on the spike corpus (events_10mb.jsonl): the malformed line sits at byte 6,292,320 (manifest-committed); the real _stream reports 6,292,650 — off by +330, exactly the carry length at the crossing. The spike's C scanner reports the correct offset.
Affected outputs: unsupported_reason offsets and byte_ranges (both feed FileInspection → API/MCP/CLI).
Fix
line_start = base + pos where base = offset - len(carry) — the replica arithmetic in bench/native_spike/py_baseline.py::py_scan_only, which achieves full parity with the C scanner.
Acceptance criteria
- One-line arithmetic fix in
_stream.
- Regression test with a fixture whose early line spans the 1 MiB chunk boundary (the spike's fixtures are reusable), asserting
byte_ranges and unsupported_reason offsets against a plain full-file read.
- All existing inspector tests pass.
Context
Summary
Fix the line-offset arithmetic bug in
JSONLInspector._streamdiscovered by the native helper spike (#48, bench in #84).Bug
When a line spans the 1 MiB read-chunk boundary,
_streamcomputesline_start = offset + posin(carry+chunk)coordinates (src/hotmem/inspectors/jsonl_inspector.py:129), overstating file offsets bylen(carry)for every subsequent line.Evidence on the spike corpus (
events_10mb.jsonl): the malformed line sits at byte 6,292,320 (manifest-committed); the real_streamreports 6,292,650 — off by +330, exactly the carry length at the crossing. The spike's C scanner reports the correct offset.Affected outputs:
unsupported_reasonoffsets andbyte_ranges(both feedFileInspection→ API/MCP/CLI).Fix
line_start = base + poswherebase = offset - len(carry)— the replica arithmetic inbench/native_spike/py_baseline.py::py_scan_only, which achieves full parity with the C scanner.Acceptance criteria
_stream.byte_rangesandunsupported_reasonoffsets against a plain full-file read.Context
bench/native_spike/README.md("Discovered:_streamline-offset bug")