[Deicyde]

Problem

Gaps in the operations runbook (#56) and its summary script. #52 and #63 made several worse by shifting the code they cite.

1. Step 1 prints secrets. sed -n '/environment = {/,/}/p' prints every launchd environment value unmasked, including any CLAUDE_CODE_OAUTH_TOKEN.
2. Step 5's token check leaves the token in the operator's shell. It sources nightly.local.env into the interactive Terminal, so the token stays in the environment of every later command there, including any agent session started from it.
3. The launchd probe doesn't run like the nightly. The nightly's plist (site/ops/launchd-plist.template) sets ProcessType Background and runs /bin/bash as the parent. The probe ran the CLI directly, or exec'd bash away, and never read runs or the last exit code before bootout.
4. Step 7 says a run registers only on success. It registers either way.
5. extract_failed and fetch_failed always map to a host cause, in the outcome table and in manage/nightly_health.py. Surface review failures without host access #52's ARTICLE_SPECIFIC_ERROR_RE treats them as article-specific, so one failing article reads as a broken host.
6. There is no Full Disk Access outcome, though site/ops/README.md documents the TCC failure (Operation not permitted, exit 126).
7. Line citations into site/moderate.py are stale after Surface review failures without host access #52 and wiki: serve wp-update the whole drifted set (cap 1000) instead of attempt stamps #63.
8. review_runs counts dry runs, which write nothing.
9. The P3 clear rate has a sample bias. While wp-update runs take at most 100 articles, the attempts come from the stuck queue head, and nothing says so.
10. The fresh-checkout note doesn't say what a clone must recreate, or that the site/cache history stays behind. site/ops/README.md doesn't link the runbook.

Change

• docs/OPERATIONS.md:
  • Step 1 masks the values of TOKEN/KEY/SECRET/PASSWORD-like names.
  • Step 5's Terminal token check runs in a subshell, and step 7's token form uses the same subshell. The checklist says to type bash first, because Terminal's default interactive zsh doesn't treat # as a comment.
  • The launchd probe runs as Background, with /bin/bash as its responsible process. Its -c script ends in exit $?, so bash runs the CLI as a child (as for the nightly) and launchd's last exit code is the CLI's. That holds for the token variant too. The probe prints runs and the last exit code before bootout, and the text says what runs = 0 and 126 mean.
  • Step 7: a run registers either way. The real confirmation is launchctl kickstart -k (or waiting for 03:20), then a review run with nonzero tokens on /stats.
  • Outcome table: the extract_failed/fetch_failed rows apply when every article that reached that stage shows them. A new article-specific row covers the rest, including the cached-HTML case, and it is scoped to PR Surface review failures without host access #52. A new Full Disk Access row lists exit code 126 and the older wrapper's limits.
  • site/moderate.py citations name functions (_import_ba, _write_article, run_jobs, _AbortingSem). A pre-Surface review failures without host access #52 batch_annotate.py line citation is dropped.
  • Drift gets the P3 sample-bias caveat, and names wiki: serve wp-update the whole drifted set (cap 1000) instead of attempt stamps #63 where it said "this change".
  • The fresh-checkout note lists wiki/.dev.vars (the wrapper's preflight stops without it), a Python 3.12+ venv at .venv or catalog/.venv, npm ci in wiki/ (the job's two KV pushes run npx wrangler), flushing or copying site/cache/.pending_posts/, and the site/cache history that stays behind.
• manage/nightly_health.py:
  • drops dry runs;
  • prints extract_failed/fetch_failed as article-specific unless every article that reached that stage shows them (a fetch failure counts only articles past the Worker read, and an extract failure only those also past the fetch);
  • adds a Full Disk Access cause;
  • adds a "lower bound" note on the clear rate when the largest wp-update run took exactly 100 articles, a full old page.
• site/ops/README.md links the runbook from its Operate section.

How tested

• manage/test_nightly_health.py: 21 passed, including 5 new tests: dry runs, host-wide vs article-specific, the per-stage denominators, Full Disk Access, and the sample note on and off. The dry-run, per-stage and sample-note tests each fail with their change reverted.
• With macOS's /bin/bash 3.2, the old -c script (ending in echo) exits 0 when the CLI fails with 126, and the new one (ending in exit $?) exits 126.
• On a sample launchctl print block, the step 1 masking sed masks CLAUDE_CODE_OAUTH_TOKEN, WIKILEAN_API_TOKEN, AWS_SECRET_ACCESS_KEY and DB_PASSWORD, and leaves PATH and HOME alone.
• Release-neutral per site/ops/brain_release_authority.py.

Risks

• Docs and a local diagnostic script only. Nothing deploys, and the nightly is unchanged.

What Jack must do

• Nothing.
• Not included:
  • site/moderate.py's run_jobs docstring still says exit 3 means "aborted (window/budget)", though since Surface review failures without host access #52 it also means an auth or credit abort. That file is release-affecting, so the fix waits for a PR that rides a release.
  • Copying the oldest site/cache/cron/moderate-*.log files off the host (the wrapper keeps 30) is a host action.

[Deicyde]

Recommendation: merge (at b9c88d5a, CI green; the review's fixes are in):

• The probe's -c script ends in exit $?, so launchd's last exit code is the CLI's, and 126 means Full Disk Access, as the text says.
• nightly_health.py counts each stage against the articles that reached it, and gives the lower-bound note only for a full 100-row page. Tests fail if either fix is reverted.
• The fresh-checkout note, step 7's subshell, the run-in-bash note, the FDA row and the Surface review failures without host access #52 scoping are corrected.

Left for a follow-up: the stale batch_annotate.py line citations. Release-neutral.

Posted by the WikiLean PR swarm (lead); updated 2026-10-18.