Review: Add Phys.org mirror + task verifiers (site by @Within-yao, verifiers by reviewer)

[jackjin1997]

Review: phys_org

This is the review-completion PR for #6 by @Within-yao. The contributor commit and author attribution are preserved as the first commit; verifier, task-contract, environment, and responsive review fixes are isolated in subsequent reviewer commits.

Related asset change: Hugging Face dataset PR #48, superseding inactive duplicate PRs #6 and #7.

What changed

• rebased the site onto current main as site 17 on port 40016
• added deterministic verifiers and rule-only judge rubrics for all 18 tasks
• audited the 18-task set against the project 15–20 guidance and retained it without padding to 20
• hardened no-op, shortcut, wrong-origin, wrong-account, overwritten-credential, state-mismatch, negation, and comparison-direction boundaries
• strengthened task 7 before/after state, task 13 password-hash evidence, task 15 click-driven multi-hop order, and task 9/16 search distractors
• corrected Task 10 so the task, rubric, and verifier all require opening the Popular result and reading the detail-only Provided by field
• made Technology journal synthesis subsection-specific and removed the empty Other Sciences category
• stripped AppleDouble entries at asset-pack time and expanded the site dependency manifest
• fixed the mobile header layout so a 390px viewport no longer overflows horizontally
• upgraded all 210 article images in HF Add Best Buy WebHarbor mirror #48 from 90x90 RSS thumbnails to exact title-matched official 800px Science X CDN images while preserving the seed DB byte-for-byte
• corrected the documented 17-site port sweep to cover 41000–41016

Verification performed

• frozen GitHub commit: 810d814f80c0c6a84b9d3b7aea17d65c4e64c4b5
• frozen HF Add Best Buy WebHarbor mirror #48 candidate commit: a0c081ebe8936bfb8fcabd71e2eaabaa2d49d055
• verifier and environment suites — 32/32 PASS
• python3 -m py_compile sites/phys_org/verify/*.py — PASS
• git diff --check — PASS
• clean remote HF asset — archive MD5 f1021749228f0e9466085c7e74ed1c3e, SHA-256 b34d7af7334dccb7a0f2f7c670deaadf640adf82ff02e45e16b7ab9c2893af7f, seed DB MD5 cc683dce62b228da98ebb984fe686bf8
• exact image mapping — 210/210 title matches against official Phys.org and Tech Xplore date archives, 0 missing, 0 ambiguous; all images are 800px wide
• clean single-site browser QA — visible search/filter and click-chain tasks, authenticated saved/account flows, state mutation, responsive 390x844 layout, real media, and byte-identical restart reset PASS
• final full 17-site environment smoke — standard asset fetch/check/build PASS; 17/17 HTTP 200 and health green before and after reset-all; Phys.org dirty DB restored byte-identically; reset-all 17/17 ready in 5s
• canonical execution — 18/18 real task trajectories with per-step screenshots; fresh deterministic grading 18/18 PASS
• independent Claude Code verifier-blind coverage — prior final review covers the 17 unchanged tasks; the affected Task 10 rerun also PASS at the current commit
• final reconciliation — combined Claude 18/18 PASS vs deterministic 18/18 PASS, 0 disagreements
• S8 readiness rerun — clean/mergeable GitHub head, reachable exact HF object, 32/32 tests, 18/18 canonical verifiers, and the complete 17-site environment gate all PASS

Review status

• All review gates are complete. PR Review: Add Phys.org mirror + task verifiers (site by @Within-yao, verifiers by reviewer) #74 is Ready for maintainer review at the frozen GitHub/HF revisions above.
• HF Add Best Buy WebHarbor mirror #48 may remain open during review because the tested asset is frozen by immutable commit. After review approval, merge HF Add Best Buy WebHarbor mirror #48 first, update .assets-revision to the resulting dataset main SHA, and recheck asset reachability/hash before the GitHub PR is merged. Trajectories and blind review only need repeating if application code or asset bytes change.

[jackjin1997]

Independent blind review — WHR-006 phys_org original-baseline: 18/18 PASS

Model. Claude (session configured as claude-fable-5), acting as an independent blind judge. Before the verdicts were frozen, no deterministic verifier source or verdicts, hidden expected answers, Codex conclusions, prior reviews, or later environment/asset changes were read. Frozen verdicts: cases/WHR-006-phys-org/independent-review/claude-review.json (webharbor-review-ops workspace).

Frozen code commit. aiming-lab/WebHarbor@00482e5170596ab4d2c80ad4dd4893b928495597 — asset revision 2c730fb9d63102a97b938ebe4d05e3e0107088db, asset archive md5 5c4294d956a7a8e0dbb7bade7bfd150e, seed db md5 b4a324122c3cb0a56b8d511e73ff13a7. Integrity check: task-contract.json and all 183 files under runs/ match the manifest sha256/byte counts exactly.

Coverage. All 18 tasks judged from the frozen inputs only: complete trajectories, before/after SQLite snapshots (ground truth re-derived via direct SQL), and per-step screenshots. The 4 state-changing runs (--6 comment, --7 save-with-note, --13 registration+settings, --14 unsave) were checked by exact full-table diffs: each contains precisely the required mutation (run --13's stored bcrypt hash verifies against the task's requested password; run --14 additionally logs one search_history row from search-based navigation, which does not touch the rubric-constrained saved list). The 14 read-only runs' after.db hashes are byte-identical to the seed.

Failures. None — PASS on all 18. Every final answer matches the value independently re-derived from the seed DB, and every rubric navigation checkpoint is evidenced by trajectory URLs/titles plus screenshots. Non-blocking observations: run --6 needed a direct-URL fallback after two click strategies failed to navigate; run --9's go_back errored in the harness and was recovered with a direct goto.

Checks not executed. Deterministic verifiers were not run and their verdicts remain unread (blind protocol). Trajectories were not re-executed against a live environment. Screenshots are viewport-top captures, so a few below-the-fold facts (journal / "Provided by" boxes, one comment thread) were corroborated against the frozen seed DB rather than pixels. Upstream phys.org was not consulted; pre-seeded users' passwords were not verified.

Posted per the review handoff; no code or Draft status changes.

[jackjin1997]

Follow-up: Task 11 reconciliation rerun — PASS

Model. Claude (session configured as claude-fable-5), again acting as an independent blind judge for this single rerun. Deterministic verifier source/verdicts, hidden expected answers, Codex reconciliation conclusions, discarded attempts, and prior reviewer rationale were not read before the verdict was frozen.

Frozen commit. aiming-lab/WebHarbor@6b8f161c79503232f9ba31488e58ba0b1ced15e8 — asset archive md5 675f29aa165b6adb85d0cea8f9f4a1b7, seed db md5 cc683dce62b228da98ebb984fe686bf8, recorded endpoint localhost:41017. task-contract.json and all 6 frozen files under runs/Phys.org--11/ verify against the manifest sha256/byte counts.

Task 11 verdict. PASS — the rerun opens both named article detail pages (full-page screenshots show their displayed publication dates, May 07, 2026 vs May 06, 2026), and the final answer names the article the rerun seed DB independently confirms as earlier (2026-05-06 15:40 vs 2026-05-07 11:20). after.db is byte-identical to before.db — no unintended state change for this read-only task. Verdict frozen at cases/WHR-006-phys-org/independent-review/claude-review-task11-reconcile.json (webharbor-review-ops workspace). Non-blocking notes: navigation used direct URL gotos (rubric only requires opening both detail pages), and the browser session was signed in as bob_c throughout.

Checks not executed. Deterministic verifier not run and its verdict unread (blind protocol); trajectory not re-executed against a live environment; upstream phys.org not consulted; only Task 11 was judged in this pass — the other 17 baseline tasks were not re-reviewed against commit 6b8f161c.

Follow-up per the Task 11 reconciliation handoff; no code or Draft status changes.

[jackjin1997]

Final independent blind review — canonical runs: 18/18 PASS

Model. Claude (session configured as claude-fable-5), independent blind judge. Deterministic verifier code and verifier.json files, expected answers, Codex conclusions, earlier Claude reviews, and prior GitHub review comments were not inspected before the verdicts were frozen. Ground truth was re-derived from the frozen seed DB by direct SQL. Frozen artifact (18 verdicts + summary): cases/WHR-006-phys-org/independent-review/claude-review-final-canonical.json (webharbor-review-ops workspace, outside the frozen input directory).

Frozen commits. GitHub: aiming-lab/WebHarbor@4c97fedbc57e550e5d8823e0314174446267f396. Hugging Face assets: HF PR #48 immutable revision a0c081ebe8936bfb8fcabd71e2eaabaa2d49d055 (archive md5 f1021749228f0e9466085c7e74ed1c3e, sha256 b34d7af7…, seed db md5 cc683dce62b228da98ebb984fe686bf8; taken as the environment baseline per the manifest — merge not required). Integrity: task-contract.json, prompt.md, and all 187 files under runs/ verify against the manifest sha256/byte counts (189/189).

Coverage. All 18 canonical runs judged from the frozen inputs only: complete trajectories, before/after SQLite snapshots, and per-step screenshots. All 18 before.db files are byte-identical to the canonical seed. The 4 state-changing runs (--6 comment, --7 save-with-note, --13 registration+settings, --14 unsave) were checked by exact full-table diffs — each contains precisely the required mutation and nothing else rubric-relevant (--13's runner-chosen password is stored as a well-formed, parseable bcrypt hash, satisfying the updated 'valid password hash' checkpoint; --14 logs one search_history row from search navigation, outside the rubric-constrained saved list). The updated task wordings ('graphene systems', 'CO2 systems', free-choice password) were followed exactly, and run --5's answer matches this seed's updated journal value — evidence the answers were read from this environment.

Failures. None — PASS on all 18. Every final answer matches the value independently re-derived from the seed, and every navigation checkpoint is evidenced by trajectory URLs/titles plus screenshots. Observations (non-blocking): --6 switched to another Biology article after two non-navigating clicks (task allows any Biology article); --11 recorded several no-op title clicks before both comparison pages were genuinely visited via direct anchor clicks (URLs + screenshots confirm real visits; one step 'thought' references a runner-side verifier check — my verdict rests only on the frozen navigation evidence and seed DB).

Checks not executed. Deterministic verifiers not run, their verdicts unread (blind protocol). No live re-execution; the HF PR #48 archive was not re-downloaded (revision/checksums taken from the manifest). Screenshots are viewport-top captures, so a few below-the-fold fields (journal/Provided-by boxes, one comment thread) were corroborated against the frozen seed DB rather than pixels. Pre-seeded users' passwords not verified; upstream phys.org not consulted.

Final review per the handoff; no code or Draft status changes.

[jackjin1997]

Follow-up: Phys.org--10 affected rerun — PASS

Model. Claude (session configured as claude-fable-5), independent blind judge for this single affected rerun. Verifier source/verdicts, hidden expected answers, Codex conclusions, discarded attempts, and prior reviewer rationale were not read before the verdict was frozen.

Frozen commit. aiming-lab/WebHarbor@810d814f80c0c6a84b9d3b7aea17d65c4e64c4b5 — HF asset PR #48 immutable revision a0c081ebe8936bfb8fcabd71e2eaabaa2d49d055, seed db md5 cc683dce62b228da98ebb984fe686bf8, recorded endpoint localhost:43017. All 9 frozen files (task-contract.json, prompt.md, trajectory, both DBs, 5 screenshots) verify against the manifest sha256/byte counts.

Verdict. PASS — the rerun opens Astronomy & Space, switches to the Popular sort (?sort=popular; full-page screenshot shows the target ranked first), and opens the first-ranked article. The final answer 'European Southern Observatory' matches both the Provided-by field visible in the article-page screenshot and the seed DB's value for the most-viewed astronomy article (8227 views, independently re-derived). after.db is byte-identical to before.db — no unintended state change. Verdict frozen at cases/WHR-006-phys-org/independent-review/claude-review-task10-810d814.json (webharbor-review-ops workspace). Non-blocking note: the browser session was signed in as alice_j throughout.

Checks not executed. Deterministic verifier not run and its verdict unread (blind protocol); no live re-execution; the HF PR #48 archive was not re-downloaded (revision/checksums from the manifest); only Phys.org--10 was judged in this pass — the other 17 canonical tasks were not re-reviewed against commit 810d814f.

Follow-up per the affected-run handoff; no code or public PR status changes.