Skip to content

review(cookpad): add deterministic task verifiers - #78

Open
reacher-z wants to merge 2 commits into
aiming-lab:mainfrom
reacher-z:review/cookpad-verifiers
Open

review(cookpad): add deterministic task verifiers#78
reacher-z wants to merge 2 commits into
aiming-lab:mainfrom
reacher-z:review/cookpad-verifiers

Conversation

@reacher-z

Copy link
Copy Markdown

Reviewer contribution

This builds on #52 while preserving XuanRui LI's original Cookpad commit and authorship. The reviewer changes are isolated in a second commit.

What changed

  • rebased the Cookpad mirror onto current main, retaining Merriam-Webster and assigning Cookpad the non-conflicting port 40016
  • added deterministic verifiers for all 19 tasks, with required-navigation checks to reject prior-knowledge shortcuts
  • added SQLite before/after checks for the two write tasks: Bob creating Cookpad Staples and replacing Sunday dinner with Waffles
  • replaced vague "benchmark password" wording with the exact seeded credentials for every standalone authenticated task
  • added explicit judge_rubric and verifier_path contracts without leaking an answer field
  • fixed unsafe inline image fallbacks that raised a JavaScript syntax error for titles such as World's Best Lasagna
  • added a tracked Cookpad SVG placeholder because the current asset archive contains the seed DB but no recipe images, preventing broken image boxes while real imagery is pending
  • corrected the documented site count and port range to 17 sites / 40000-40016

Verification

  • python3 -m unittest discover -s sites/cookpad/verify -p 'test_*.py' -v: 8 test groups pass
  • positive, no-op, wrong-answer, and prior-knowledge cases cover all 19 tasks
  • both stateful false-claim cases fail without the corresponding database transition
  • every verifier wrapper was exercised for both pass and fail exit codes
  • py_compile, JSONL parsing, shell syntax, git diff --check, and asset checks pass
  • Docker runtime health reports 17/17 mirrors alive, with Merriam-Webster on 40015 and Cookpad on 40016
  • real HTTP login/form flows pass both stateful verifiers against copied live SQLite state
  • browser QA confirmed login, recipe detail, and meal-plan replacement through the rendered UI; the placeholder loads successfully and no new JavaScript errors occur

Asset follow-up

The companion asset submission is ChilleD/WebHarbor discussion #35. Its current cookpad.tar.gz contains instance_seed/cookpad.db but empty image/cache directories. This PR supplies a safe fallback, but real recipe imagery should still be added before treating visual fidelity as complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant