feat(dist): pure-Go releases, a cache preset, --idle-exit, and gateway conformance - #141
Conversation
92631bf to
6f10c82
Compare
|
Reviewed against main The headline claim holds, and the compaction core is sound. Almost everything below is in the plugin shell, not the core. 1. Blocking —
|
|
Two additional findings that landed after my main review, both in 11. Blocking — the checksum verification is fail-open, so an unverified binary installs and runs
An unverified binary landed on a PATH directory and executed — and this binary then handles all of Three things make it worse than a missing check:
Fix: 12. Blocking — there is no upgrade path, on the PR that creates the release channel
( For the PR whose purpose is a release channel, "installs once, never upgradable" is the gap that Addendum on the
|
| Zero-value condition | Detected | Documented |
|---|---|---|
| Non-Anthropic backend | no | yes (install-plugin.md:67, status/SKILL.md:71) |
| System prompt under the 1,024-token floor | no | no |
| Not a git repository | no | nowhere |
Row 3 is the common case for a casual trial, and it reproduces live: non-git cwd →
cachesplit mutated=0 verdict=skipped; the same task inside a git repo → mutated=2 verdict=moved.
status/SKILL.md:65-73 lists four honest reasons the numbers may be flat and the one that actually
applies is not among them — so a first-run user outside a git repo is shown a zero and told the cache
warms on later turns. True in general, wrong there. A fifth bullet plus
git rev-parse --is-inside-work-tree in the status skill closes it.
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. ## 1. The toolchain gate was our own documentation `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. ## 2. A `cache` preset: cachesplit alone The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. ## 3. `--idle-exit`: the proxy cleans itself up Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. ## 4. Gateway conformance All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. ## 5. Review of #141: the `cache` preset advertised a tool whose every call must fail Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. ## 6. The rest of the review - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. ## Verification Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
6f10c82 to
5e344d4
Compare
|
Thank you — this was an unusually useful review, and three of the findings were things no test of mine would have caught. Split as you asked, into this PR (core) and #160 (the plugin): every blocking finding was in the plugin, so nothing clean is held behind it. I went with two rather than five because Where each finding went:
I verified finding 4 before fixing it rather than taking it on faith, and got your result exactly:
Two places I did not do what you suggested, both with reasons:
On the −34.1% figure — you were right and it mattered. Both numbers are now stated with their regimes, along with the three zero cases; the old citation pointed at Not fixed, deliberately: Still unverified, and the thing I would not paper over: the plugin has never been installed into a real Claude Code session, because Also from your review: the stale preset tables are #142 (now guarded by a set-equality drift test), and the prose instances the guard cannot reach are #143. |
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. ## 1. The toolchain gate was our own documentation `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. ## 2. A `cache` preset: cachesplit alone The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. ## 3. `--idle-exit`: the proxy cleans itself up Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. ## 4. Gateway conformance All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. ## 5. Review of #141: the `cache` preset advertised a tool whose every call must fail Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. ## 6. The rest of the review - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. ## Verification Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
5e344d4 to
eadc015
Compare
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. ## The six blocking findings **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. ## Smaller review items - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). ## Verification The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. ## The six blocking findings **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. ## Smaller review items - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). ## Verification The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
eadc015 to
e624e0c
Compare
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
e624e0c to
61827f6
Compare
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. ## The six blocking findings **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. ## Smaller review items - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). ## Verification The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
|
Restructured per your ask: the pre-existing defects are now #161, and this PR keeps only distribution work. #161 carries the expand-tool gate (including It also adds a Two things from doing the split that you should know, both mine:
Also filed #162: #143 duplicates #145 and should be closed in its favour — I filed it without checking the tracker. |
cache preset, --idle-exit, a Claude Code plugin, and gateway conformancecache preset, --idle-exit, and gateway conformance
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
left a comment
There was a problem hiding this comment.
Second pass, over 69cb2be..92a04e5 (3 commits, 20 files). No Go toolchain in the review worktree, so nothing was executed — static read of every hunk plus proxy/keepalive.go, proxy/proxy.go, config/docdrift_test.go, Makefile, store/store.go.
None of the ten findings from the first pass resurfaced — f2b3d6e addresses all of them. But three of the five new findings are consequences of those fixes, so they are worth reading as follow-ups rather than as fresh territory: the envDuration fatal now catches --version/--help (the highest-severity item here), the trailing-slash hole in probeRoutes, and the floor being skipped entirely rather than reduced when the store is off.
Verified correct this pass: pendingPings reads stopped/pol/pings under k.mu, matching markStopped/record1, so no race; why has capacity 2 for exactly its two senders and stopWatch is closed only by the single consumer, so no double-close or blocked send; checkIdleExit still runs after --store/--config resolution, so cfg.Store is the same Options that reaches NewMemory (config.go:670); IdleExitFloor's > at the 1h boundary is value-equivalent; the cache preset survives TestPresetDocsDoNotDrift in both doc formats and the other range presets tests; and set -e does not abort the CI readiness loops.
| return d | ||
| d, err := parseEnvDuration(os.Getenv(key), def) | ||
| if err != nil { | ||
| log.Fatalf("context-guru: %s=%q is not a duration (%v). Use a unit — 24h, 30m, 1500ms — "+ |
There was a problem hiding this comment.
This fatal fires before flag.Parse(), so it breaks --version and --help. Every call site is a default expression inside main's var (...) block, which Go evaluates before flag.Parse() and therefore before the new --version short-circuit at main.go:217. So IDLE_EXIT=86400 context-guru-proxy --version — precisely the unitless mistake this fatal was added to catch — exits 1 with a parse error instead of printing the version. That defeats the stated reason for adding the flag ("an installer asks a binary what it is, and it must be able to ask without starting a server or needing a config"), and the release workflow's own --version | grep -q context-guru-proxy gate fails the same way in a shell that exports a bad value.
The blast radius is wider than IDLE_EXIT: this helper also backs UPSTREAM_HEADER_TIMEOUT, DASHBOARD_RETENTION, ARCHIVE_CONTENT_AFTER, ARCHIVE_SESSION_AFTER and ARCHIVE_INTERVAL, so an existing deployment carrying a malformed value for any of those now refuses to start where it previously fell back to the default — and it dies before logging.Setup(), so the message never reaches CG_LOG_FILE.
Suggest making the parse lazy (validate after flag.Parse(), past the --version return) or scoping the fatal to IDLE_EXIT.
| // Everything else still counts, including the dashboard's own polling: a person with the | ||
| // dashboard open is using this process, and exiting under them is a worse failure than a | ||
| // process left running. That is a deliberate asymmetry — a probe is not a viewer. | ||
| var probeRoutes = map[string]bool{ |
There was a problem hiding this comment.
/healthz/ is not exempt, so a trailing-slash probe keeps the process alive forever. This is an exact r.URL.Path match, and http.ServeMux answers /healthz/ with a 301 to /healthz — which a Kubernetes httpGet probe (and most monitoring loops) treats as healthy. Since stampActivity stamps on entry, before the mux ever sees the request, a probe configured with a trailing slash refreshes the clock and --idle-exit never fires, silently, with idle-exit armed as the only log line. That is exactly the failure the comment above says was measured and fixed.
Same for any 404: a port scanner or a stray /health probe counts as use. Matching on the cleaned path, or on a small prefix set, would close both.
| // A function rather than two inline `if`s in main so both refusals are testable: they are | ||
| // startup-fatal, which is the one class of check where "it looked right" is the only evidence | ||
| // anyone ever gathers. | ||
| func checkIdleExit(d time.Duration, upstreamsPath string, o store.Options) error { |
There was a problem hiding this comment.
Skipping the floor when the store is off removed it entirely, and one consequence is a startup panic. With --store=false / STORE=false, ValidateIdleExit is never called, so no floor is enforced at all — not even the 1h term.
(a) idleCheckInterval (idleexit.go:175) now has no lower clamp, and its comment reasons that "checkIdleExit refuses any threshold below an hour, so threshold/20 is at least three minutes for every value that reaches here" — untrue on this path. STORE=false --idle-exit=10ns yields threshold/20 == 0, and time.NewTicker(0) panics, converting an intended startup refusal into a crash.
(b) README.md:150 and docs/reference/config.md:83 both state the floor unconditionally ("Refused at startup below max(2 × store.ttl_seconds, 1h)") with no mention of the store-off exemption, so the documented invariant does not hold.
Keeping the bare 1h term when the store is off — or at minimum clamping the interval to a positive value — plus a sentence in the two tables would fix both halves.
| - name: Test the shipped configuration (CGO off, no race detector) | ||
| env: | ||
| CGO_ENABLED: "0" | ||
| run: go test ./config/... ./components/... ./apply/... ./proxy/... ./store/... |
There was a problem hiding this comment.
Missing -p 1. ci.yaml's purego job runs this identical package set and carries a comment explaining that omitting -p 1 on a 2-core runner provoked a real flake (TestCtlGetCampaignAggregatesPredictedAndRealPerTenant, #163) and that serialising "costs about a minute and removes the contention this job added". This workflow additionally runs go test ./... a few steps down, so the contention is at least as bad — a tag push can flake and block the publish for a cause already diagnosed and mitigated one file over.
| } | ||
|
|
||
| // envDuration reads a Go duration environment variable (e.g. "72h"). | ||
| // envDuration reads a duration from the environment, falling back to def. |
There was a problem hiding this comment.
Leftover from the edit: the old doc line above (// envDuration reads a Go duration environment variable (e.g. "72h").) was not removed, so the function now has two consecutive doc comments. Cosmetic, but it is a stale first line on a function whose semantics this PR changed, and some doc linters flag the duplicated identifier prefix.
|
|
||
| // pendingPings counts entries with a ping still ahead of them. Nil-safe: a keeper whose | ||
| // sweep never launched (the CONTEXT_GURU_KEEPALIVE kill switch) has nothing pending, which | ||
| // correctly lets an idle proxy exit. |
There was a problem hiding this comment.
Not filed as a defect, but the doc claim is slightly stronger than the code: pendingPings reproduces due() minus the timing term, but not due's companion gate — sweep also drops entries whose pingable() has gone false, and only once now >= startedAt + Idle. In that window pendingPings counts an entry that will never be pinged, which resets the activity clock and delays exit by up to one Idle. So "a live entry is by construction one we intend to ping" holds outside that window, not inside it. Bounded by the retire timer and immaterial at a 24h threshold.
| Route [Claude Code](https://docs.claude.com/en/docs/claude-code) through context-guru with | ||
| one environment variable — no changes to Claude Code itself. | ||
|
|
||
| ## You do not need an API key |
There was a problem hiding this comment.
Docs judgement call rather than a defect: this section correctly notes the savings land in usage limits, but not that routing subscription-authenticated Claude Code through the proxy means the proxy receives and (for keep-alive) retains the claude.ai OAuth credential, and that pings spend against those same limits. Worth a caveat given the section directly below is titled "Keep the API key out of Claude Code" — a reader can come away thinking the no-API-key path hands the proxy less.
… fixes' blast radius
Five findings plus two notes. The reviewer's framing is the useful one: three are consequences of
the FIRST review's fixes, so they read as follow-ups rather than fresh ground. All three were mine,
and in each case the fix was right and its blast radius was not measured.
## The `envDuration` fatal broke `--version` and `--help`
Every call site is a default expression in main's `var (...)` block, which Go evaluates BEFORE
`flag.Parse` — so exiting from inside that helper ran before the `--version` short-circuit existed to
be reached. `IDLE_EXIT=86400 context-guru-proxy --version` exited 1 with a parse error: precisely the
unitless mistake the fatal was added to catch, defeating the reason `--version` was added in the same
round ("an installer must be able to ask what it is"), and failing the release workflow's own
`--version | grep` gate in any shell exporting a bad value. It also died before `logging.Setup()`, so
the message never reached CG_LOG_FILE.
The blast radius was wider than IDLE_EXIT: the helper also backs UPSTREAM_HEADER_TIMEOUT,
DASHBOARD_RETENTION and the three ARCHIVE_* windows, so a deployment carrying a malformed value for
any of them refused to start where it had previously fallen back.
Keep the loudness, move the moment: `envDuration` records the bad value and returns the default;
`checkEnvDurations` refuses after `flag.Parse`, past the `--version` return, once the log sink exists
and before anything is opened. All of them at once — an operator fixing one typo should not restart
to discover the next.
## `/healthz/` was not exempt, so a trailing-slash probe kept the process alive forever
An exact `r.URL.Path` match. `http.ServeMux` answers `/healthz/` with a 301 to `/healthz`, which a
Kubernetes httpGet probe and most monitoring loops read as healthy — and because the stamp is taken
before the mux sees the request, such a probe refreshed the activity clock indefinitely. Silently,
with `idle-exit armed` as the only log line: the exact failure the previous round's fix was written
for, reachable by adding one character to a probe URL.
`stampActivity` now asks the mux which pattern matches rather than comparing the path.
`ServeMux.Handler` reports, for an internally-generated redirect, the pattern that will match after
following it — so `/healthz/` resolves to `/healthz` — and reports the EMPTY pattern for an unmatched
path, so a 404 (a port scanner, a stray `/health`, a typo) is no longer mistaken for use. One
question to the same matcher that will route the request, instead of a second copy of the rules.
## Skipping the floor with the store off removed it entirely, and could panic
`ValidateIdleExit` was not called at all with `--store=false`, so no floor applied — not even the
bare 1h term. `STORE=false --idle-exit=10ns` then reached `time.NewTicker` with `threshold/20 == 0`,
which PANICS: an intended startup refusal became a crash. It also broke the invariant README and
docs/reference/config.md state unconditionally.
Only the `2 x store.ttl_seconds` term is about the store. The 1h minimum now always applies, and
`idleCheckInterval` cannot return a non-positive value regardless of caller — a crash is the wrong
failure mode for a helper, and the previous version of that function reasoned "checkIdleExit refuses
anything under an hour" and was then reached with 10ns through the path I had just opened.
## And
- `release.yaml` was missing `-p 1`, which `ci.yaml` one file over documents as the mitigation for a
real flake (#163) on a 2-core runner. This workflow runs the full suite too, so the contention is
at least as bad, and a tag push must not fail to publish for a diagnosed cause.
- A stale duplicate doc line above `envDuration`, left by the previous edit.
- `pendingPings`'s doc said "a live entry is by construction one we intend to ping". Nearly true:
`sweep` also drops entries whose `pingable()` has gone false and only looks once `Idle` has
elapsed, so inside that window this counts an entry that will never be pinged. Narrowed to
"outside that window", with why erring toward counting is still right — over-counting delays an
exit by minutes, under-counting kills the process in the gap the keep-alive exists to work in.
- `docs/how-to/use-with-claude-code.md` now says what the no-API-key path actually hands over: the
proxy receives the claude.ai OAuth credential on every request, retains it in memory for a tracked
session when keep-alive is on, and those pings spend the same usage limits. The section below it is
titled "Keep the API key out of Claude Code", so a reader could otherwise conclude the
subscription path gives the proxy less.
## A test of mine argued for the defect
The first attempt at this round failed on `TestCheckIdleExitSkipsTheFloorWithNoStore` — my own test
from the previous round, whose NAME asserts the behaviour this review showed was wrong. It was
deleted rather than edited into shape, and its still-valid cases folded into the test named for the
corrected behaviour. A test named for a defect is worse than no test: it argues for the defect on
every future read.
## Verification
Three new tests, each revert-verified against the pre-fix file:
exact-path probe match -> TestProbeExemptionSurvivesATrailingSlash FAIL on all five:
/healthz/, /metrics/, //healthz, /health (404), /nope (404) all counted as activity
floor skipped entirely -> TestCheckIdleExitKeepsTheOneHourMinimumWithNoStore FAIL
accepted --idle-exit=10ns, 1ms and 30m with the store disabled
no panic guard -> TestIdleCheckIntervalIsAlwaysPositive FAIL
idleCheckInterval(0s) = 0s; time.NewTicker would panic
Finding 1 is verified by reading rather than by test: asserting it needs a subprocess that runs the
built binary with a bad env var and `--version`, which felt disproportionate — the fix is that the
refusal now sits after `flag.Parse` and after the `--version` return, which is a position, not a
behaviour a unit test can observe.
`go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
|
All five plus both notes. Your framing is the part I want to acknowledge first: three of these are the first review's fixes' blast radius, and all three were mine. In each case the fix was right and the reach was not measured.
|
amiddavid
left a comment
There was a problem hiding this comment.
Third pass, over 69cb2be..6590033. Read in full: cmd/context-guru-proxy/idleexit.go, cmd/context-guru-proxy/main.go, store/store.go, proxy/keepalive.go, config/config.go, .goreleaser.yaml, both workflows, the new tests and the docs. No go on PATH in the review worktree, so nothing was compiled — findings are from reading, cross-checked against net/http.ServeMux's actual matching rules and the surrounding call sites.
The two envDuration findings and the -p 1/floor items from the last pass are resolved. What is left is one live hole (the Bob-mode catch-all re-opens the trailing-slash gap the mux.Handler fix was written to close), two messages/comments that misstate their own mechanism, and two packaging/doc items.
Checked and found correct: checkIdleExit runs after the --store override folds into cfg.Store (main.go:244 → 256) and before any database is opened, so the "no WAL left behind" claim holds, and hosted mode is unreachable without --upstreams so the gateway refusal does cover it; the monotonic reasoning in activityClock is sound (CLOCK_MONOTONIC/mach_absolute_time both stop across suspend, so a lid-open does not read as idleness); idleCheckInterval cannot return <= 0 for any value checkIdleExit admits, and the guard is still correct defensively; why is buffered 2 for exactly two senders and stopWatch is closed exactly once (a second signal during shutdown being ignored is pre-existing, not a regression); PendingPings/pendingPings are nil-safe against a real *keeper field, take k.mu, and their documented divergence from sweep's pingable() gate errs toward not exiting; Options.EffectiveTTL is genuinely the single source since NewMemory calls it; the cache preset satisfies TestPresetDocsDoNotDrift in both tables (the regex skips the "Your workload" row because its first cell is not a backticked name), PresetNames() derives from the map so the settings page picks it up, and the gofmt de-alignment of "off": is correct; and the release workflow's health loop is set -e-safe with the trailing curl … | grep -q ok failing the step if the server never came up.
| return fmt.Errorf("idle-exit %s is below the floor of %s (2x the store's %s entry "+ | ||
| "lifetime): exiting wipes the in-memory store, so a shorter threshold drops live "+ | ||
| "frozen decisions and re-bills their prefix as cache creation instead of a cache "+ | ||
| "read. Raise --idle-exit, or raise store.ttl_seconds if the short lifetime is "+ |
There was a problem hiding this comment.
The remediation is backwards, and the derivation is false whenever the 1h term binds. The floor is max(2*ttl, 1h), so "raise store.ttl_seconds if the short lifetime is deliberate" tells the operator to do the one thing that raises the floor — they follow the advice and get the same refusal with a larger number. It should be to lower ttl_seconds, or raise --idle-exit.
Separately, the message unconditionally attributes the floor to 2x the store's %s entry lifetime: with store.ttl_seconds: 30 and --idle-exit=30m it prints floor of 1h0m0s (2x the store's 30s entry lifetime) — 2x30s is 1m, not 1h. The single number the operator has to act on is credited to arithmetic that does not produce it. Worth getting exactly right because this path is startup-fatal, so the message is the only evidence anyone gathers.
| // This asks the same matcher that will route the request, which is the point: no second | ||
| // copy of the routing rules to drift. | ||
| _, pattern := mux.Handler(r) | ||
| use := pattern != "" && !probeRoutes[patternPath(pattern)] |
There was a problem hiding this comment.
Bob mode has a / catch-all, so every unmatched path counts as activity and --idle-exit silently never fires. proxy.Mux() registers m.HandleFunc("/", h.passthrough(...)) whenever BobUpstream != "" || Tenants != nil (proxy/proxy.go:408), and --bob-upstream does not trigger checkIdleExit's gateway refusal — that only fires on upstreamsPath != "" — so --bob-upstream=… --idle-exit=24h is an accepted configuration.
In it, mux.Handler(r) returns pattern "/" for /healthz/, /health, /nope and every port-scan path. "/" is not in probeRoutes, so use is true and the clock is stamped. Concretely: a monitoring loop doing GET /healthz/ every 60s against a Bob-mode proxy holds the clock up forever, and the only log line is idle-exit armed at startup — exactly the silent failure lines 60-68 say was measured and fixed.
probeMux() (idleexit_test.go:448) registers no catch-all, so no test covers this shape.
| // Ask the mux which pattern this request resolves to, rather than comparing r.URL.Path. | ||
| // | ||
| // An exact path compare had two holes, and the first is the one that mattered: ServeMux | ||
| // answers `/healthz/` with a 301 to `/healthz`, which a Kubernetes httpGet probe treats as |
There was a problem hiding this comment.
The stated mechanism is wrong: ServeMux does not 301 /healthz/ to /healthz. cleanPath re-appends a trailing slash after path.Clean, and matchOrRedirect only ever adds a slash (/tree → /tree/), never strips one. With the default route table /healthz/ is a plain 404 with an empty pattern — so the exemption does hold, but through the 404 branch described below, not the redirect branch.
The same inaccuracy is in idleexit_test.go:461 and the "301 to /healthz" / "301 to /metrics" case labels at :470 and :472 — those rows pass for a different reason than they document.
Worth correcting because this comment is the justification for choosing mux.Handler(r) over a path compare, and because the redirect Go does generate (subtree root) returns an empty pattern from Handler — the opposite of "the pattern that will match after following it". Anything built on that claim for an added-slash redirect would be wrong.
| goos: [linux, darwin] | ||
| goarch: [amd64, arm64] | ||
|
|
||
| archives: |
There was a problem hiding this comment.
No wrap_in_directory, so the documented untar command clobbers the user's files. archives[0].files puts LICENSE, README.md and THIRD-PARTY-NOTICES at the archive root (GoReleaser defaults wrap_in_directory to false), and both the release footer (:95) and docs/get-started/quickstart-proxy.md tell the evaluator to run tar xzf context-guru_*_darwin_arm64.tar.gz && install -m 755 context-guru-proxy ~/.local/bin/ — with no -C.
Run in a project directory, which is exactly where someone evaluating a proxy for their agent is standing, that silently overwrites their own README.md and LICENSE. Either set wrap_in_directory: true, or make the documented command extract into a temp dir.
| |---|---|---| | ||
| | `--preset` / `PRESET` | `house` | pipeline preset when no `--config` | | ||
| | `--idle-exit` / `IDLE_EXIT` | `0` (never) | exit after this long unused; floor `max(2 × store.ttl_seconds, 1h)`, refused with `--upstreams` | | ||
| | `--version` | — | print version and commit, then exit | |
There was a problem hiding this comment.
Two lines below this, the table still documents only LISTEN_ADDR — the --listen flag this PR adds is missing from the row. docs/reference/config.md:81 was updated to --listen / LISTEN_ADDR; the README row was not, in the same table where these --idle-exit and --version rows landed. A README reader cannot discover the flag whose stated reason for existing is discoverability in ps.
… of mine that were false
Five findings. One is a hole I left open, two are corrections to statements I wrote, and one would
have destroyed a user's files.
## Bob mode defeated the probe exemption entirely
`proxy.Mux` registers a `/` catch-all whenever `BobUpstream` is set, and `--bob-upstream` did NOT
trigger the gateway refusal — only `--upstreams` did. So `--bob-upstream=… --idle-exit=24h` was an
accepted configuration, and in it `mux.Handler` answers pattern `"/"` for every unmatched path:
`/healthz/`, `/nope`, every port-scan path. `"/"` is not a probe route, so the clock was stamped and
`--idle-exit` never fired — the same silent failure the previous round's `mux.Handler` fix was written
to close, reachable through a different flag.
Closed by construction: the gateway refusal now covers `--bob-upstream`, so a proxy with a catch-all
cannot also have a watchdog, and the message names whichever flag the operator actually passed. Plus
a catch-all exclusion in the stamp decision as belt to that braces — the two rules live in different
files, and if they drift, over-counting `/` as "not use" errs toward exiting a laptop proxy rather
than toward a gateway that never exits.
## The mechanism I documented is not what ServeMux does
I wrote that ServeMux answers `/healthz/` with a 301 to `/healthz`, and that `Handler` reports "the
pattern that will match after following the redirect". Both false: `cleanPath` re-appends a trailing
slash and `matchOrRedirect` only ever ADDS one, so with this route table `/healthz/` is a plain 404
and `Handler` reports the EMPTY pattern — which is what exempts it, through the same branch that
exempts `/nope`. The redirect Go does generate, for a subtree root, also reports an empty pattern.
The behaviour was right; the justification was invented, and it was the justification for choosing
`mux.Handler` over a path compare — so anything built on it later would have been wrong. Corrected in
the comment and in three test case labels that were passing for a reason they did not document.
## The refusal's advice was backwards
`ValidateIdleExit` told the operator to "raise store.ttl_seconds if the short lifetime is deliberate".
The floor is `max(2*ttl, 1h)`, so raising the TTL RAISES the floor: follow the advice, get the same
refusal with a larger number. It also credited the floor to `2x the store's %s entry lifetime`
unconditionally — with `ttl_seconds: 30` it announced `floor of 1h0m0s (2x the store's 30s entry
lifetime)`, and 2x30s is 1m. The single number the operator must act on was attributed to arithmetic
that does not produce it.
The message now says which of the two terms binds, offers only levers that lower the floor, and when
the absolute term binds it says what the 2x-TTL floor would have been so the arithmetic is checkable.
This path is startup-fatal, so the message is the only evidence anyone gathers.
## The documented untar would overwrite the user's own files
`archives[0].files` puts LICENSE, README.md and THIRD-PARTY-NOTICES at the archive ROOT (GoReleaser
defaults `wrap_in_directory` to false), and both the release footer and the quickstart tell the
evaluator to run `tar xzf …` with no `-C`. In a project directory — where somebody evaluating a proxy
for their coding agent is standing — that silently overwrites their README.md and LICENSE.
`wrap_in_directory: true`, and both documented commands updated to
`install -m 755 context-guru_*/context-guru-proxy`. Note for the plugin PR: its install.sh locates the
binary at the archive root, so it needs the same change; handled there rather than left to fail at a
stranger's install.
## And
The README flag table never got the `--listen` row that landed in `docs/reference/config.md`, in the
same table where `--idle-exit` and `--version` were added — so a README reader could not discover the
flag whose stated reason for existing is discoverability.
## Verification
Two new tests, plus label corrections:
TestCheckIdleExitRefusesBobModeToo — the four flag combinations, and that the message names the
flag actually passed
TestCatchAllRouteIsNotActivity — with a `/` route registered, /healthz/ and /nope must still not
count, while the explicit Bob and Anthropic routes must
TestValidateIdleExitMessageNamesTheBindingTerm — asserts the message says "2x the store's" only
when that term binds, offers LOWER rather than raise, and shows the 1m0s figure when the absolute
term is doing the work
`go test ./...` and `gofmt -l` clean. Findings 4 and 5 are documentation and packaging, verified by
reading the rendered command and the table.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
… path #141 sets goreleaser's `wrap_in_directory: true`, so the release archive unpacks into a directory and the binary is no longer at the root — which is exactly what `[ -f "$TMP/$BIN" ]` assumed. It is wrapped for a reason worth not undoing: a flat archive plus the documented `tar xzf` with no `-C` overwrites the README.md and LICENSE of whatever directory the user is standing in, which for somebody evaluating a proxy for their agent is their own project. So find the binary rather than assume it. That works for either layout, and the failure it avoids is the worst-placed one available: a stranger's very first install, reporting `binary_not_in_tarball` — which reads as a broken release rather than a file that moved. Found by checking #141's packaging change against this script rather than by waiting for it to fail. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
|
All five. Two of them are corrections to things I wrote, one was a hole I left open, and one would have destroyed a user's files — so this round is mostly me being wrong rather than incomplete, and it is worth saying that plainly. The Bob catch-all — you found the hole my own fix leftYou are right, and the mechanism is worse than "one flag was missed": Fixed by construction: the refusal now covers
The ServeMux mechanism — I invented itThis is the finding I am most glad you wrote down. I claimed a 301 from Corrected in the comment and in the three test labels, which were passing for a reason they did not document. The comment now states what actually exempts each row and warns explicitly that nothing may assume a redirect resolves to its post-redirect pattern. The refusal messageBoth halves taken. It now branches on which term binds: when the store term does, it says "2x the store's %s entry lifetime" and offers LOWER Your point about it being startup-fatal is the reason I went further than a wording fix: the message is the entire diagnostic surface for that path. The archive
It has a consequence you did not have to tell me about, and I checked rather than assumed: #160's README
|
|
@copilot Review the code |
…way conformance Implements the local-distribution proposal (#130) minus the Claude Code plugin, which ships as its own PR because all six blocking findings from the review of #141 live in it. Nothing here is held behind that. `docs/get-started/quickstart-proxy.md`, `docs/setup.md` and `docs/hosted.md` all told evaluators to install a C toolchain and set `CGO_ENABLED=1`. That is true only for a `cg_skeleton` build. setup.md went further and named bifrost's tokenizer as a cgo dependency, which it is not — o200k_base is embedded (`internal/tokens/tokens.go`). Verified directly on go 1.26.4 rather than taken from the proposal: `CGO_ENABLED=0` with default tags builds all four release targets (linux/darwin x amd64/arm64, 27.1-33.5 MB stripped), `file` reports "statically linked", `ldd` reports "not a dynamic executable", the binary starts and answers /healthz, and `-tags cg_skeleton` fails under CGO_ENABLED=0 with the build-constraints signature — confirming tree-sitter is the only C dependency. - `.goreleaser.yaml`: a plain GOOS/GOARCH matrix, no cross-toolchains, no `brews:` block (the tap repo and release signing are an unowned question, and nothing may depend on a repo that does not exist). - `.github/workflows/release.yaml`: a tag publishes, `workflow_dispatch` builds the same matrix as a snapshot. It asserts the pure-Go claim with `CC=/nonexistent-c-compiler`. - `make build-static`. The Makefile keeps `CGO_ENABLED=1` because `go test -race` needs it, and the comment now says that is a test-time requirement — reading it as a shipping requirement is how the wrong claim reached the docs. The funnel's default, chosen so a stranger can verify the claim by reading one line rather than trusting four components. Not `safe`, whose extra components are lossless in meaning but still rewrite the JSON. Off by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the SAME teardown, so the self-killing path cannot drift from the one known to work. Two properties are load-bearing: - **The keep-alive inverts "idle."** Pinging is what the proxy does precisely while no client traffic arrives — the quiet gap after `end_turn`, where 83.7% of the recoverable dollars sit. A pending ping both vetoes the exit and RESETS the clock, so retiring the last ping buys a full fresh threshold rather than exiting moments later. - **Exit wipes the in-memory store.** `store.ValidateIdleExit` refuses anything below `max(2 x store.ttl_seconds, 1h)` at startup — ~5h34m at the default. 2x because the TTL is a sliding window. `NewMemory` now calls the same `Options.EffectiveTTL` the floor is computed from, so the two cannot drift. All five items from the proposal, under the `cache` preset. Four were already correct and are now pinned by tests; `POST /anthropic/v1/messages/count_tokens` was missing entirely — without it a client counts context by issuing INFERENCE requests, billed calls added by a proxy sold on removing them. Five places promised it did not: `config/config.go`, `docs/reference/presets.md`, `docs/how-to/choose-a-preset.md`, the plugin doc, and the install skill. Verified before fixing — `[Read Bash]` in, `[Read Bash context_guru_expand]` out on the real gateway route. Root cause was a code-vs-comment contradiction. `Options.InjectExpand` documented the gate as requiring "an expandable marker"; `expand/inject.go` says "No marker condition, deliberately" and the real conditions were mode, store-persists and has-tools. Nothing asked whether the pipeline could produce a marker at all. `components.Pipeline.HasOffload()` answers that by type assertion rather than a name list (a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one). Under `auto`, injection now requires it. `always` still injects unconditionally — an operator who asks for it by name gets it. This also fixes `off`, the A/B control arm, which was carrying an extra tool declaration. Marker independence is preserved, which is the invariant that matters for cache stability: a pipeline does not change turn to turn, so the tools array stays byte-stable across a session. **Ten existing expand tests changed fixture, and that is worth reading.** They hand-seed the Store to simulate an offload, but built their handler with `pipeline: []` — a pipeline that cannot offload anything. That was harmless only while injection ignored the pipeline. They now use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies), so each fixture matches its own premise. No assertion was weakened. - **`--idle-exit` was defeated by any health probe** (finding 7). `/healthz` and `/metrics` no longer count as activity: a probe on a schedule shorter than the threshold meant the exit NEVER fired and logged nothing to say so — measured, a 1h-threshold proxy reporting "idle for 1h3m0s" after 2h03m. A dashboard poll still counts, deliberately: a probe is not a viewer, and exiting under somebody who is watching is the worse failure. - **A gateway may no longer self-terminate.** `--idle-exit` with `--upstreams` is refused at startup. That safety was previously accidental — it held only because hosted deployments run a liveness probe, which the change above stops counting. - **The floor's refusal was logged after "listening"**, so a rejected configuration read as a crash. Both refusals moved earlier and into one testable `checkIdleExit`. - **`--listen` and `--version` flags** (findings 2 and 12, which are the plugin's, but the flags are the core's). The address reached the process only through the environment, so no supervisor or `ps` could tell which port an instance held; and `buildinfo.Version` was reachable only via `/stats` on a running proxy, so an installer asking `--help` recorded "Usage of context-guru-proxy:" as the installed version. - **Nothing tested the shipped configuration** (finding 9). A tag published without running any tests, and CI runs the suite only with `CGO_ENABLED=1` — so `TestEveryPresetBuilds`, which guards exactly the CGO-free artifact, was never executed in that configuration. The release workflow now runs a CGO-off suite over the packages whose behaviour depends on which components are compiled in, plus the full suite, before publishing. It also asserts `--version` answers. - **`scripts/gate-a-purego.sh` was cited as proof in four places and is not in this PR** (finding 8). Those now cite the release workflow's own assert step, which exists here and fails the release if a cgo dependency escapes the `cg_skeleton` tag. - **The savings claim was measured in the wrong regime.** −34.1% / 96.7% comes from a harness running tasks back-to-back inside the provider's 5-minute TTL, and is one task measured three times; this project's own interactive figure is $0.0298 across 1,127 sessions, with 1,105 of 1,127 session starts reading zero from cache. Both are now stated, with the zero cases (outside a git repo, under the 1,024-token floor, non-Anthropic backend). The old citation pointed at `docs/results/context-guru.md`, which contains neither number. - **`count_tokens` behaviour kept, consequence documented** (finding 10). It answers about the ORIGINAL body — over-reporting is recoverable, under-reporting costs a failed turn when a fail-open component reverts. What was undocumented is the cost: the client self-compacts earlier than needed (115,933 reported vs 32,802 forwarded). Now in `docs/reference/routes.md`, where the route was absent entirely. Its **hosted branch was wholly untested** — that branch is all that stands between the multi-tenant service and an unmetered open forwarder — so it now has one. - `--idle-exit` and `--version` added to both flag tables; `bytes.NewReader` in counttokens. Six mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated (the defect) -> TestCachePresetAdvertisesNoExtraTool FAIL cache: sent [Read Bash], forwarded [Read Bash context_guru_expand] off: sent [Read Bash], forwarded [Read Bash context_guru_expand] HasOffload always true -> same test FAIL, same two subcases HasOffload always false -> FAIL on the offloader subcase: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" probes count as activity again -> TestProbesDoNotDeferIdleExit FAIL ("two hours of nothing but liveness probes: idle past the threshold, but watchIdle never exited") gateway guard disabled -> TestCheckIdleExitRefusesAGatewaySelfTerminating FAIL count_tokens hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The third mutation is the one worth noting: it proves the fix did not trade a silent defect for its mirror image, an offloader whose output nothing can expand. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
CI validated on Go 1.25 while `go.mod` declared 1.26.4 and this PR's release workflow built on 1.26 — three numbers that have to agree, with nothing making them. That is a distribution bug, not a housekeeping one. `purego` asserts that the SHIPPED artifact builds with cgo off, and `release.yaml` builds the artifact people download; when those run different toolchains from each other and from the module, the assertion describes a build nobody ships. The `purego` job inherited the wrong pin from the job it was copied from, which is exactly how the drift spread in the first place. Fixed as a class rather than an instance: `go-version-file: go.mod` in all three places, so the module file is the single source of truth and the next toolchain bump moves CI, the release build and the module together or not at all. Typing `1.26` in three files would have fixed today's symptom and left tomorrow's. `check-latest` is dropped with the literals — it existed to pick up patch releases of a pinned minor, and go.mod names an exact version. Both workflows re-validated as YAML. The change can only really be proven by CI itself, which is where the previous mismatch was invisible. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…usal, honest comments
Ten findings, all addressed. The pattern across most of them is the one this branch keeps
rediscovering: a comment that describes something the code does not do.
## Behaviour
**The idle clock lost its monotonic reading.** It stored `now.UnixNano()` and rebuilt the instant
with `time.Unix(0, ns)`, which carries no monotonic reading — so `now.Sub(act.last())` was
wall-clock arithmetic. A laptop suspend/resume or an NTP step counted as idleness, and the watchdog
could fire on its first tick after a lid-open, racing the user's first request. On the laptop this
feature exists for, suspend is the normal case. Now stores the `time.Time` itself.
Fixing it broke the unstamped-clock backstop, and the existing test caught that: `time.Time{}`'s
`UnixNano()` is a large NEGATIVE number, not zero, so the guard stopped firing and the watchdog
exited immediately reporting "idle for 2562047h47m16s". The predicate is `IsZero()` now.
**The refusal ran too late to be free.** `checkIdleExit` sat after the dashboard and control SQLite
files are opened, and `log.Fatalf` calls `os.Exit`, which runs no defers — so `--idle-exit 30m`
created and migrated both databases and then exited with WAL/-shm left behind. Moved to immediately
after the config resolves, where everything it reads is already known.
**The floor protected a store that need not exist.** `--store=false` resolves to `store.Nop`, which
holds no frozen decisions, yet a short threshold was refused with a message about dropping them.
Skipped when `Enabled` is explicitly false; `nil` (unconfigured) still means on.
**`IDLE_EXIT=86400` silently meant "never".** `envDuration` discarded the parse error, and the
`idle-exit armed` line is only logged above zero — so the operator's evidence was the ABSENCE of a
line. A non-empty unparseable duration is now fatal, for every caller: each one is a timeout, a
retention window or a process lifetime, and a typo in any of them changes behaviour nobody chose.
**Requests are stamped on completion as well as entry.** Entry-only meant a long request looked
like a gap in use the moment it finished.
**A clamp that could never fire is gone.** `checkIdleExit` refuses anything under an hour, so
`threshold/20` is always at least three minutes and the `< 30s` branch was unreachable — while the
comment above it claimed the clamps prevented "a 1h floor meaning a check every three minutes",
which is exactly what an hour yields.
## Comments that were not true
- `stampActivity` claimed a long streaming response "cannot age out while it is still being served".
It can: the clock is not refreshed DURING a request, so a lone SSE consumer past the threshold is
severed by the shutdown `armShutdown` performs. Now states the residual and why it is not worth
machinery (the dashboard polls every 30s, and the floor is an hour).
- The `--idle-exit` flag comment said it and the resurrection hook "ship together". They do not: the
hook is in the plugin PR. Anyone setting this by hand today gets a proxy that exits and stays
exited, and the comment says so.
- `.goreleaser.yaml` cited `scripts/install.sh` as the consumer of `checksums.txt`. That installer
ships with the plugin; today the file is what a human curling a release should check by hand.
## Tests
`recordedRequest` (mutex-guarded) replaces three unsynchronised captures the reviewer flagged in
`conformance_test.go` and `counttokens_test.go`: a handler goroutine wrote them, the test goroutine
read them, and an HTTP round trip is not a happens-before edge the memory model guarantees.
Worth stating precisely: `-race` is clean before AND after, so this is a fix by INSPECTION, not one
the detector demonstrated. A parallel sweep of the whole suite (28 more instances, its own PR) built
a control proving the detector was live and that this shape is invisible to it — an in-process
loopback round trip manufactures an edge through net/http's internals that the spec does not promise
and a Go release can remove.
The `10m -> 30s` interval case was labelled "clamped low" and asserted exactly `10m/20`, so it
passed whether or not the clamp existed. Replaced with cases that pin the rule and the cap.
New tests, each revert-verified against the pre-fix file with the mutation proven to have landed:
Unix-nanos clock -> TestActivityClockKeepsItsMonotonicReading FAIL
"the stored instant has no monotonic reading, so idleness is measured against the wall clock"
entry-only stamp -> TestStampActivityRefreshesOnCompletion FAIL
clock reads the moment the request STARTED, 20 minutes behind its completion
floor always enforced -> TestCheckIdleExitSkipsTheFloorWithNoStore FAIL
refused 30m with the store disabled, citing frozen decisions that cannot exist
silent duration parse -> TestParseEnvDurationRefusesAUnitlessValue FAIL
parseEnvDuration("86400") returned the default and no error
`parseEnvDuration` is split out of `envDuration` so the decision is testable without a process that
calls `os.Exit`.
Two findings are verified by inspection only, and neither is testable without a subprocess harness:
the refusal's new POSITION (it precedes every `Close`-deferring construction in main) and the three
comment corrections.
`go build ./...`, `go vet ./...`, `gofmt -l`, the full `go test ./...`, and `go test -race` over
./proxy/ and ./cmd/context-guru-proxy/ are all clean.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
#191 landed a mutex-guarded `upstreamCapture` in package proxy_test while this branch carried `recordedRequest`, added for the same reason on the same review. Two near-identical helpers in one package is duplication a reader has to reconcile, so this drops mine. Theirs is the better of the two: it records every round with method, path, headers and body — so a test needing the second round's headers has somewhere to read them — and `record` returns the 1-based round number, which removes the captured counter a per-round handler would otherwise need. Mapping: rec.forwarded() -> up.body(1), rec.requestPath() -> up.round(1).path. The reason both existed is unchanged and worth keeping in view: a fixture handler runs on the test server's goroutine, and the HTTP round trip that follows is not a happens-before edge. Note that `-race` reports none of these, before or after — #191 established that with a control, so this is synchronisation by the memory model rather than by anything the detector demanded. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
… fixes' blast radius
Five findings plus two notes. The reviewer's framing is the useful one: three are consequences of
the FIRST review's fixes, so they read as follow-ups rather than fresh ground. All three were mine,
and in each case the fix was right and its blast radius was not measured.
## The `envDuration` fatal broke `--version` and `--help`
Every call site is a default expression in main's `var (...)` block, which Go evaluates BEFORE
`flag.Parse` — so exiting from inside that helper ran before the `--version` short-circuit existed to
be reached. `IDLE_EXIT=86400 context-guru-proxy --version` exited 1 with a parse error: precisely the
unitless mistake the fatal was added to catch, defeating the reason `--version` was added in the same
round ("an installer must be able to ask what it is"), and failing the release workflow's own
`--version | grep` gate in any shell exporting a bad value. It also died before `logging.Setup()`, so
the message never reached CG_LOG_FILE.
The blast radius was wider than IDLE_EXIT: the helper also backs UPSTREAM_HEADER_TIMEOUT,
DASHBOARD_RETENTION and the three ARCHIVE_* windows, so a deployment carrying a malformed value for
any of them refused to start where it had previously fallen back.
Keep the loudness, move the moment: `envDuration` records the bad value and returns the default;
`checkEnvDurations` refuses after `flag.Parse`, past the `--version` return, once the log sink exists
and before anything is opened. All of them at once — an operator fixing one typo should not restart
to discover the next.
## `/healthz/` was not exempt, so a trailing-slash probe kept the process alive forever
An exact `r.URL.Path` match. `http.ServeMux` answers `/healthz/` with a 301 to `/healthz`, which a
Kubernetes httpGet probe and most monitoring loops read as healthy — and because the stamp is taken
before the mux sees the request, such a probe refreshed the activity clock indefinitely. Silently,
with `idle-exit armed` as the only log line: the exact failure the previous round's fix was written
for, reachable by adding one character to a probe URL.
`stampActivity` now asks the mux which pattern matches rather than comparing the path.
`ServeMux.Handler` reports, for an internally-generated redirect, the pattern that will match after
following it — so `/healthz/` resolves to `/healthz` — and reports the EMPTY pattern for an unmatched
path, so a 404 (a port scanner, a stray `/health`, a typo) is no longer mistaken for use. One
question to the same matcher that will route the request, instead of a second copy of the rules.
## Skipping the floor with the store off removed it entirely, and could panic
`ValidateIdleExit` was not called at all with `--store=false`, so no floor applied — not even the
bare 1h term. `STORE=false --idle-exit=10ns` then reached `time.NewTicker` with `threshold/20 == 0`,
which PANICS: an intended startup refusal became a crash. It also broke the invariant README and
docs/reference/config.md state unconditionally.
Only the `2 x store.ttl_seconds` term is about the store. The 1h minimum now always applies, and
`idleCheckInterval` cannot return a non-positive value regardless of caller — a crash is the wrong
failure mode for a helper, and the previous version of that function reasoned "checkIdleExit refuses
anything under an hour" and was then reached with 10ns through the path I had just opened.
## And
- `release.yaml` was missing `-p 1`, which `ci.yaml` one file over documents as the mitigation for a
real flake (#163) on a 2-core runner. This workflow runs the full suite too, so the contention is
at least as bad, and a tag push must not fail to publish for a diagnosed cause.
- A stale duplicate doc line above `envDuration`, left by the previous edit.
- `pendingPings`'s doc said "a live entry is by construction one we intend to ping". Nearly true:
`sweep` also drops entries whose `pingable()` has gone false and only looks once `Idle` has
elapsed, so inside that window this counts an entry that will never be pinged. Narrowed to
"outside that window", with why erring toward counting is still right — over-counting delays an
exit by minutes, under-counting kills the process in the gap the keep-alive exists to work in.
- `docs/how-to/use-with-claude-code.md` now says what the no-API-key path actually hands over: the
proxy receives the claude.ai OAuth credential on every request, retains it in memory for a tracked
session when keep-alive is on, and those pings spend the same usage limits. The section below it is
titled "Keep the API key out of Claude Code", so a reader could otherwise conclude the
subscription path gives the proxy less.
## A test of mine argued for the defect
The first attempt at this round failed on `TestCheckIdleExitSkipsTheFloorWithNoStore` — my own test
from the previous round, whose NAME asserts the behaviour this review showed was wrong. It was
deleted rather than edited into shape, and its still-valid cases folded into the test named for the
corrected behaviour. A test named for a defect is worse than no test: it argues for the defect on
every future read.
## Verification
Three new tests, each revert-verified against the pre-fix file:
exact-path probe match -> TestProbeExemptionSurvivesATrailingSlash FAIL on all five:
/healthz/, /metrics/, //healthz, /health (404), /nope (404) all counted as activity
floor skipped entirely -> TestCheckIdleExitKeepsTheOneHourMinimumWithNoStore FAIL
accepted --idle-exit=10ns, 1ms and 30m with the store disabled
no panic guard -> TestIdleCheckIntervalIsAlwaysPositive FAIL
idleCheckInterval(0s) = 0s; time.NewTicker would panic
Finding 1 is verified by reading rather than by test: asserting it needs a subprocess that runs the
built binary with a bad env var and `--version`, which felt disproportionate — the fix is that the
refusal now sits after `flag.Parse` and after the `--version` return, which is a position, not a
behaviour a unit test can observe.
`go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
… of mine that were false
Five findings. One is a hole I left open, two are corrections to statements I wrote, and one would
have destroyed a user's files.
## Bob mode defeated the probe exemption entirely
`proxy.Mux` registers a `/` catch-all whenever `BobUpstream` is set, and `--bob-upstream` did NOT
trigger the gateway refusal — only `--upstreams` did. So `--bob-upstream=… --idle-exit=24h` was an
accepted configuration, and in it `mux.Handler` answers pattern `"/"` for every unmatched path:
`/healthz/`, `/nope`, every port-scan path. `"/"` is not a probe route, so the clock was stamped and
`--idle-exit` never fired — the same silent failure the previous round's `mux.Handler` fix was written
to close, reachable through a different flag.
Closed by construction: the gateway refusal now covers `--bob-upstream`, so a proxy with a catch-all
cannot also have a watchdog, and the message names whichever flag the operator actually passed. Plus
a catch-all exclusion in the stamp decision as belt to that braces — the two rules live in different
files, and if they drift, over-counting `/` as "not use" errs toward exiting a laptop proxy rather
than toward a gateway that never exits.
## The mechanism I documented is not what ServeMux does
I wrote that ServeMux answers `/healthz/` with a 301 to `/healthz`, and that `Handler` reports "the
pattern that will match after following the redirect". Both false: `cleanPath` re-appends a trailing
slash and `matchOrRedirect` only ever ADDS one, so with this route table `/healthz/` is a plain 404
and `Handler` reports the EMPTY pattern — which is what exempts it, through the same branch that
exempts `/nope`. The redirect Go does generate, for a subtree root, also reports an empty pattern.
The behaviour was right; the justification was invented, and it was the justification for choosing
`mux.Handler` over a path compare — so anything built on it later would have been wrong. Corrected in
the comment and in three test case labels that were passing for a reason they did not document.
## The refusal's advice was backwards
`ValidateIdleExit` told the operator to "raise store.ttl_seconds if the short lifetime is deliberate".
The floor is `max(2*ttl, 1h)`, so raising the TTL RAISES the floor: follow the advice, get the same
refusal with a larger number. It also credited the floor to `2x the store's %s entry lifetime`
unconditionally — with `ttl_seconds: 30` it announced `floor of 1h0m0s (2x the store's 30s entry
lifetime)`, and 2x30s is 1m. The single number the operator must act on was attributed to arithmetic
that does not produce it.
The message now says which of the two terms binds, offers only levers that lower the floor, and when
the absolute term binds it says what the 2x-TTL floor would have been so the arithmetic is checkable.
This path is startup-fatal, so the message is the only evidence anyone gathers.
## The documented untar would overwrite the user's own files
`archives[0].files` puts LICENSE, README.md and THIRD-PARTY-NOTICES at the archive ROOT (GoReleaser
defaults `wrap_in_directory` to false), and both the release footer and the quickstart tell the
evaluator to run `tar xzf …` with no `-C`. In a project directory — where somebody evaluating a proxy
for their coding agent is standing — that silently overwrites their README.md and LICENSE.
`wrap_in_directory: true`, and both documented commands updated to
`install -m 755 context-guru_*/context-guru-proxy`. Note for the plugin PR: its install.sh locates the
binary at the archive root, so it needs the same change; handled there rather than left to fail at a
stranger's install.
## And
The README flag table never got the `--listen` row that landed in `docs/reference/config.md`, in the
same table where `--idle-exit` and `--version` were added — so a README reader could not discover the
flag whose stated reason for existing is discoverability.
## Verification
Two new tests, plus label corrections:
TestCheckIdleExitRefusesBobModeToo — the four flag combinations, and that the message names the
flag actually passed
TestCatchAllRouteIsNotActivity — with a `/` route registered, /healthz/ and /nope must still not
count, while the explicit Bob and Anthropic routes must
TestValidateIdleExitMessageNamesTheBindingTerm — asserts the message says "2x the store's" only
when that term binds, offers LOWER rather than raise, and shows the 1m0s figure when the absolute
term is doing the work
`go test ./...` and `gofmt -l` clean. Findings 4 and 5 are documentation and packaging, verified by
reading the rendered command and the table.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
435b1a7 to
953f648
Compare
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
… path #141 sets goreleaser's `wrap_in_directory: true`, so the release archive unpacks into a directory and the binary is no longer at the root — which is exactly what `[ -f "$TMP/$BIN" ]` assumed. It is wrapped for a reason worth not undoing: a flat archive plus the documented `tar xzf` with no `-C` overwrites the README.md and LICENSE of whatever directory the user is standing in, which for somebody evaluating a proxy for their agent is their own project. So find the binary rather than assume it. That works for either layout, and the failure it avoids is the worst-placed one available: a stranger's very first install, reporting `binary_not_in_tarball` — which reads as a broken release rather than a file that moved. Found by checking #141's packaging change against this script rather than by waiting for it to fail. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
… path #141 sets goreleaser's `wrap_in_directory: true`, so the release archive unpacks into a directory and the binary is no longer at the root — which is exactly what `[ -f "$TMP/$BIN" ]` assumed. It is wrapped for a reason worth not undoing: a flat archive plus the documented `tar xzf` with no `-C` overwrites the README.md and LICENSE of whatever directory the user is standing in, which for somebody evaluating a proxy for their agent is their own project. So find the binary rather than assume it. That works for either layout, and the failure it avoids is the worst-placed one available: a stranger's very first install, reporting `binary_not_in_tarball` — which reads as a broken release rather than a file that moved. Found by checking #141's packaging change against this script rather than by waiting for it to fail. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…e whole config
`checksums:` is not a field in GoReleaser v2 — the key has always been `checksum:`. The plural made
it an unknown field, and GoReleaser refuses to start rather than ignoring it:
starting release
release failed after 0s
error=
| yaml: unmarshal errors:
| line 71: field checksums not found in type config.Project
So the release path produced NOTHING: no archives, no checksums.txt, no assets. Not a wrong
filename — a pipeline that never began.
Mine, from #141 (b08e543). What makes it worth stating plainly: #141 added `release.yaml` with a
`workflow_dispatch` snapshot entry, and its own commit body said that entry exists so "the release
path gets exercised before there is a tag to regret". That workflow had never been run. The first
run of it failed on this line, before a tag existed — which is the outcome it was added for, at the
cost of the release it was supposed to protect being broken the whole time in between.
The `name_template: checksums.txt` under it was correct and is now actually in effect. It matters:
GoReleaser's default checksum filename is `{{.ProjectName}}_{{.Version}}_checksums.txt`, and the
plugin's install.sh fetches `checksums.txt` — that is the only integrity check in the download path,
and it is fail-closed, so a mismatch there refuses the install rather than trusting the tarball.
Verified by re-running the same snapshot workflow on this branch rather than by reading: the run
that failed on `main` succeeds here, and builds all four archives.
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
Implements the proposal in #130, all three stages. Based on
docs/local-distribution, so this PR's diff is the implementation only — the spec and its two verification scripts come from #130, which should merge first.Four decisions were taken before writing anything, and all four are reviewable:
.claude/settings.local.json),--globalopt-in--idle-exitdefault and floormax(2 × store.ttl_seconds, 1h)— enforced at startupgo install; the tap repo and signing are still unownedStage 1 — the toolchain gate
Our own docs were the largest one.
quickstart-proxy.md,setup.mdandhosted.mdall told evaluators to install a C toolchain;setup.mdadditionally named bifrost's tokenizer as a cgo dependency, which it is not.Re-verified directly on go 1.26.4 rather than taken from the spec:
CGO_ENABLED=0with default tags builds all four release targets (27.1–33.5 MB stripped),filereports "statically linked",lddreports "not a dynamic executable", the binary starts and answers/healthz, and-tags cg_skeletonfails underCGO_ENABLED=0with the build-constraints signature — confirming tree-sitter is the only C dependency..goreleaser.yaml: plain GOOS/GOARCH matrix, no cross-toolchains. Nobrews:block, so nothing depends on a repo that does not exist..github/workflows/release.yaml: a tag publishes;workflow_dispatchbuilds the same matrix as a snapshot and publishes nothing. Its first step asserts the pure-Go claim withCC=/nonexistent-c-compiler, so a cgo dependency escaping the build tag fails in CI rather than at a stranger's install.make build-static. The Makefile keepsCGO_ENABLED=1becausego test -raceneeds it — reading that as a shipping requirement is how the wrong claim reached the docs, and the comment now says so.cachepreset ={cachesplit}. Notsafe: format/textclean/searchfold are lossless in meaning but still rewrite the JSON, so "we do not touch your context" stops being literally checkable. Confirmed end to end — a released-shape binary logspipeline=[cachesplit]underPRESET=cache.Stage 2 —
--idle-exitand the pluginOff by default; a gateway or eval-containers deployment must never self-terminate. A signal and the watchdog converge on the same teardown, so the self-killing path cannot drift from the one known to work.
Two properties are load-bearing:
end_turn, where 83.7% of the recoverable dollars sit. A request-only watchdog would kill the feature in its working window. So a pending ping both vetoes the exit and resets the clock: retiring the last ping buys a full fresh threshold rather than exiting moments later.store.ValidateIdleExitrefuses anything belowmax(2 × ttl, 1h)— ~5h34m at the default — at startup, not in a doc comment. 2× because the TTL is a sliding window.NewMemorynow calls the sameOptions.EffectiveTTLthe floor is computed from, so the two cannot drift.The plugin (
/plugin marketplace add rossoctl/context-guru→/context-guru:install) is three skills over three scripts plus aSessionStarthook. The hook self-gates on$ANTHROPIC_BASE_URLmatching its own port: the plugin installs at user scope, so its hooks run in every project, and the env value settings already write into the process environment is the per-project enablement signal — no second copy of the port to drift, and it degrades correctly (remove the key by hand and the hook stops firing). It matches the port, not "localhost", so someone routing to litellm on 4000 is not hijacked. Synchronous (closes the race with the first request), idempotent (SessionStartalso fires on clear/compact/resume/fork), and exit 0 on every path — a hook that fails here is a plugin that can brick every session on the machine.settings.pymerges exactly one key, backs the file up first, refuses to overwrite a base URL the user already set, removes only a URL it installed, and refuses to rewrite a settings file it cannot parse rather than replacing it.Stage 3 — gateway conformance
All five items, under the
cachepreset. Four were already correct and are now pinned; one was missing:cache_controlmakes Claude Code disable prompt caching for the rest of the conversation — so a budget mistake silently switches off what the funnel is selling.CLAUDE_CODE_ATTRIBUTION_HEADER=0is not needed in the installer.POST /anthropic/v1/messages/count_tokens— NEW. Absent it, Claude Code counts context by issuing inference requests: billed calls added by a proxy sold on removing them. Forwarded verbatim with no pipeline, because the client budgets its own transcript from the answer.Verification
Every test was revert-verified, with each mutation asserted to have landed in the source before its result was allowed to count. 14 mutations, each failing with the intended message and passing when restored — full matrix in the commit body.
Doing that properly caught three defects in my own tests:
mainnow stamps the clock at launch rather than relying on goroutine scheduling.json.Marshalof a map, which sorts keys — so re-encoding produced identical bytes. It now carries Claude Code's real key order ({"type":...,"text":...}), and both mutations fail.The plugin's shell and Python helpers are tested from Go (
context-guru-plugin/plugin_test.go) sogo test ./...and CI cover them: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for/healthz.go build ./...,go vet ./...,gofmt -land the fullgo test ./...are clean.Not verified, and one thing deliberately elsewhere
install.shresolves a GitHub release and no tag has ever been published. Until one is, it reportsno_release_found. Cutting a tag is the first thing to do after this merges.cacherow, and is fixed in a sibling PR (fix/preset-doc-drift) rather than here, because it is unrelated to distribution: three presets were documented as runningtoon(retired after acting 0 times on 5,752 production requests) and every row omittedtextclean/searchfold/linecap, whiledocs/reference/presets.mdwas correct at the same moment. This PR adds one row and leaves the rest of that table exactly as it is on main, so whichever PR merges second needs a trivial rebase there.Still open for you
userConfig.skeletonomitted from releases, source build documented, per the spec's proposal.-slimbuild, so the demo keeps the dashboard.🤖 Generated with Claude Code