[frankbria]

The cold-start harness validates the CLI path on every release and has caught real defects — including a published 0.9.2 that could not run at all, found only because it installed from PyPI rather than from source. It never touched proof9: no Claude Code in the image, no reference to the plugin anywhere in it. With #1531 the README leads with the plugin install, so the lead path was the untested one.

What it does now

After the timed CLI walkthrough — and outside its total, which is the figure the README quotes — the harness:

Step	documented
plugin-install	yes	claude plugin install proof9 --marketplace frankbria/codeframe, exactly as the README documents it, from the published marketplace
plugin-enabled	check	the installed plugin shows as enabled
plugin-validate	check	claude plugin validate on the installed copy
plugin-test	check	the plugin's own test suite, run from the installed copy

documented=check is new: a verification on the published artifact, not a user step, so it doesn't read as docs drift (which is what documented=no means). Failures file P-PLUGIN-* findings like the rest of the harness.

CI already runs claude plugin validate ./proof9 and claude plugin test ./proof9 — against the plugin in the repository. This checks what a stranger receives.

Supporting changes

• Image gains Claude Code on Node 22, pinned to the version CI tests the plugin against (2.1.301).
• tests/ci/test_cleanroom_plugin_path.py keeps the two pins equal and the harness pointed at the published marketplace rather than a local path.
• Harness README documents documented=check and the published-artifact principle.

Verification

• Built the image and ran the plugin section of walkthrough.sh in it: plugin-install, plugin-enabled, plugin-validate, plugin-test all OK; 176 plugin tests pass from the installed copy; no findings.
• uv run pytest tests/ --ignore=tests/e2e -m "not lifecycle" → 6043 passed, 0 failed.
• Not run: the full walkthrough end to end, which needs a real API key and spends real model calls. The plugin section needs neither.

🤖 Generated with Claude Code

https://claude.ai/code/session_019cEymaSpmZ1arjngxx1VNM

[github-actions]

Claude finished @frankbria's task in 7m 24s —— View job

---

Claude PR Review — cleanroom plugin path

• Read full diff
• Inspect walkthrough.sh plugin section in context (step/note semantics, TOTAL computation, exit-code contract)
• Inspect Dockerfile changes (ARG pin vs test.yml, Node 22 install)
• Inspect tests/ci/test_cleanroom_plugin_path.py against the real files it guards
• Verify candidate findings against real context and post review

Result: no defects found. Full sweep notes are in the review-summary comment below.

[claude]

Review

This closes a real gap: the README lead path (plugin install) was never exercised by the cold-start harness. Keeping the plugin section outside the timed total preserves the quoted figure, and documented=check is a sensible distinction. Nothing blocking; robustness points only.

Suggestions

1. plugin-enabled is brittle. grep -A4 ... | grep -q enabled depends on output layout, and enabled also matches disabled / not enabled. Prefer JSON output if the CLI offers it, or a word-bounded match that excludes disabled.
2. The installed_plugins.json parse hard-codes the schema and the proof9@codeframe key, and 2>/dev/null hides parse errors. A harness-side format change would be reported as P-PLUGIN-NOT-INSTALLED ("the README install does not produce a usable plugin"), blaming the product. Surface the node error in the log, or use distinct findings for "file missing" and "key missing".
3. A failed plugin-install files no finding of its own. It only appears as a FAIL row in timings.tsv, plus the follow-on P-PLUGIN-NOT-INSTALLED. Check this matches how the CLI steps report failures.
4. The NodeSource curl | bash and the apt Node package are unpinned, while Claude Code is pinned. Low risk, but drift here shows up as an image build failure rather than a finding.
5. The tests are substring checks only. They would not notice the plugin block moving inside the timed total, or the ||/note wiring being dropped. "proof9/" not in script will also trip on any future comment mentioning a proof9/ path; consider a regex for a local-path install instead.

Overall a good, low-risk change. I would address (1) and (2) before relying on the findings it files.

[github-actions]

✅ Third Party Review: no defects found.

Scope swept: the four changed files plus their consumers (run.sh, build-demo.sh, tests/ci/ guards, .github/workflows/test.yml, .claude-plugin/marketplace.json).

Verified clean, among others:

• CI pin @anthropic-ai/claude-code@2.1.301 (test.yml:197) matches the Dockerfile ARG CLAUDE_CODE_VERSION; the new guard test's regexes resolve to exactly that one version on both sides.
• Marketplace manifest name is codeframe, so the installed-plugin key proof9@codeframe used by the installed_plugins.json parse and the plugin-enabled grep is consistent with --marketplace frankbria/codeframe.
• The plugin section runs after RUN_END/TOTAL are computed, so it is genuinely outside the quoted wall-clock figure.
• step/note semantics unchanged; every plugin failure path (install fail, missing install path, disabled, invalid, test failure) lands a P-PLUGIN-* finding, and the walkthrough's exit-code contract (findings, not exit status) is preserved.
• No consumer of timings.tsv assumes documented ∈ {yes,no} (build-demo.sh only column -ts it; the budget guard reads historical artifacts).
• The new test's negative assertions hold against the current walkthrough.sh (no proof9/ or @anthropic-ai/claude-code literals), and pytestmark = pytest.mark.v2 satisfies the tests/ci convention and the assertion-quality guard.

Model: k3[1m]