feat: bggrep tool + bgtail delta tailing - #2
Closed
lloydsk wants to merge 5 commits into
Closed
Conversation
Collaborator
Author
|
Addressed the fresh-eyes review in f7808cf (56/56 tests, tsc clean):
Declined by design (from the review's request-changes on #1): ancestor walk-up for subdirectory cwds — an adjudicated design decision; the README's Rules section documents the actual behavior (cwd must itself be a project root, else global fallback). Path traversal in |
lloydsk
added this pull request to stack #3
September 10, 2026 16:35
…migration-safe) (#1) * feat: project-local jobs dir via relative jobsDir (auto git-exclude, config-change-safe log lookup) A relative jobsDir (any config layer or PI_BGRUN_DIR) now resolves against the session's project root, putting logs inside the workspace — reachable for project-sandboxed analysis tools (ctx_execute_file/ctx_index) and scoped per checkout. Absolute paths behave exactly as before (migration-safe). With no recognizable project root a relative path falls back to the global dir instead of scattering logs. Project-local dirs are auto-added to .git/info/exclude (local-only, linked-worktree aware) so logs never pollute git status. bgtail now prefers the session record's logPath, so jobs stay readable across mid-session config changes (e.g. switching to project-local after upgrade). * chore: formatter pass on new tests; language-tag SKILL.md example fence (MD040) * fix: exclude-write retries on transient failure; gitdir pointers with spaces ensureGitExcluded now memoizes only on success: a failed attempt (unwritable exclude file, unparseable .git pointer) is retried on the next bgrun instead of being permanently skipped. The gitdir pointer regex accepts paths containing spaces (was \S+, which truncated at the first space). * chore: formatter reflow in spaces-path test
bggrep: capped in-extension regex search over job logs — line-numbered matches, optional context with gap markers, 50-match + ~8KB condenser caps; works on any jobs dir (native fs), closing the gap where project-sandboxed tools cannot reach global logs; generic failure-pattern default is a convenience only, always overridable. bgtail delta tailing: first read = full last-N tail (unchanged); repeat reads return only lines appended since the last read (high-water bookmark of content lines + bytes); shrunken/replaced logs reset to a full tail; raw:true keeps the verbatim window but advances the bookmark. Exit marker now filtered before slicing so last-N means last N content lines.
… sandbox gap Rationale captured in the three places that steer behavior: the run-bg skill (rules + why-not section), the README read-model section, and the bggrep tool's prompt guidelines. Never bash-grep a bgrun log; plain grep only for one-off searches known to be tiny.
…ty-log
bgtail delta: store the first content line in the bookmark — append-only
logs never mutate line 0, so a changed first line means the log was replaced
or rotated; catches same-line-count/same-size replacements the shrink checks
cannot see. Also normalize CRLF (stray \r broke nothing but leaked into
output), clamp lines to >= 1 (slice(-0) pitfall), and drop an unreachable
body branch.
bggrep: empty log now reports 'in 0 lines' (was 'in 1 lines' via
''.split('\n') === ['']); CRLF normalized so $-anchored patterns match;
context clamped to >= 0 (negative context dropped the match lines
themselves); schema minimums added for lines/context.
appendExcludePattern: skip ../-prefixed patterns (unreachable via the
walk-up today, defense-in-depth for future callers/symlinks).
Docs: ~2KB/line cap now documented alongside ~8KB in README + SKILL.md.
Tests: +6 (replacement reset, CRLF, empty+notFound, context+cap, clamps) — 57 total.
lloydsk
force-pushed
the
lloydsk/bggrep-delta
branch
from
September 10, 2026 16:40
f7808cf to
237e706
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1 (project-local jobs dir) — review/merge that first; this branch contains its commits plus the new work below.
What
Two context-efficiency features for reading job logs:
bggrep— capped in-extension pattern searchcontextlines with…[N lines skipped]…gap markers, capped at 50 matches + the ~8KB condenser budget.ctx_execute_file) cannot reach. That gap repeatedly forced agents into uncapped raw greps; bggrep closes it.pattern, a generic failure-signature default is used — deliberately a convenience only (overridden by passing your own pattern): a general-audience tool across languages/task types misses more than it catches, so heuristics are never load-bearing.bgtail.bgtaildelta tailing+N new linesheader — polling a running job never re-pays context for lines already seen; a no-change poll returns a single tiny line.raw: truekeeps the verbatim last-N window but still advances the bookmark; bookmarks are in-memory only — a session restart starts fresh./bgtailslash command inherits everything via the shared core.Testing
tsc clean; 51/51 node:test — 8 new: bggrep (line numbers + explicit/default/no-match, context windows + gap markers, invalid-pattern error, 50-match cap, record preference) and delta tailing (full→new→none sequence, raw-advances-bookmark, byte-level shrink reset). The delta rewrite also fixed a regression caught by existing tests: the exit marker is now filtered before the window is sliced, so "last N lines" keeps meaning the last N content lines.
Docs
README tools table + two-tier read model; SKILL.md tools table + reading guidance.