Skip to content

Fix _stream line-offset overstatement for chunk-spanning lines #86

Description

@valiantone

Summary

Fix the line-offset arithmetic bug in JSONLInspector._stream discovered by the native helper spike (#48, bench in #84).

Bug

When a line spans the 1 MiB read-chunk boundary, _stream computes line_start = offset + pos in (carry+chunk) coordinates (src/hotmem/inspectors/jsonl_inspector.py:129), overstating file offsets by len(carry) for every subsequent line.

Evidence on the spike corpus (events_10mb.jsonl): the malformed line sits at byte 6,292,320 (manifest-committed); the real _stream reports 6,292,650 — off by +330, exactly the carry length at the crossing. The spike's C scanner reports the correct offset.

Affected outputs: unsupported_reason offsets and byte_ranges (both feed FileInspection → API/MCP/CLI).

Fix

line_start = base + pos where base = offset - len(carry) — the replica arithmetic in bench/native_spike/py_baseline.py::py_scan_only, which achieves full parity with the C scanner.

Acceptance criteria

  • One-line arithmetic fix in _stream.
  • Regression test with a fixture whose early line spans the 1 MiB chunk boundary (the spike's fixtures are reusable), asserting byte_ranges and unsupported_reason offsets against a plain full-file read.
  • All existing inspector tests pass.

Context

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:storageStorage adapter abstraction + filesystem adaptersbugSomething isn't workingpriority:p1Do next after P0; proves and extends the company-brain moatv0.2.xHotMem post-v0.2 follow-up work

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions