Skip to content

Review: add IRS Refund Tracker mirror and deterministic verifiers - #80

Open
reacher-z wants to merge 2 commits into
aiming-lab:mainfrom
reacher-z:review/irs-refund-verifiers
Open

Review: add IRS Refund Tracker mirror and deterministic verifiers#80
reacher-z wants to merge 2 commits into
aiming-lab:mainfrom
reacher-z:review/irs-refund-verifiers

Conversation

@reacher-z

Copy link
Copy Markdown

Reviewer contribution

This completes a Track B review of #37 while preserving XuanRui LI's original commit and authorship. The original environment is retained in commit 63e61e9; the reviewer grading work is isolated in ee8972a.

What changed

  • rebased the IRS Refund Tracker mirror onto current main, retaining Merriam-Webster and assigning IRS the non-conflicting port 40016
  • synchronized the 17-site registry, Docker exposure, startup health counts, and port documentation
  • added deterministic verifiers and judge rubrics for all 18 tasks
  • required task-specific navigation so correct answers supplied from prior knowledge cannot pass
  • added SQLite seed/live-state verification for David Kim's profile-update task
  • supplied exact seeded credentials in every standalone authenticated task
  • kept ground truth inside verifier code; tasks.jsonl contains no answer key

Verification

  • python3 -m unittest discover -s sites/irs_refund/verify -p 'test_*.py' -v: 9 test groups pass
  • positive, no-op, wrong-answer, and prior-knowledge cases cover all 18 tasks
  • all 18 verifier wrappers pass their positive fixture and reject their no-op fixture
  • each of the three visible identity-checklist answers is accepted
  • task 8 rejects a correct self-report without the exact DB transition; the real browser-driven transition from (Seattle, Mail) to (Spokane, Email) passes
  • py_compile, JSONL parsing, shell syntax, git diff --check, and asset checks pass
  • final 5.92 GB Docker image builds; verifier files compile inside the image; 17/17 sites return HTTP 200
  • IRS reset is byte-identical at SHA-256 c422556eef1a7cc3a761a600bd2b9058f4489c0375d49efa5744d9cd6ff3b9f5; parallel reset-all completes in about 3.14 seconds and leaves 17/17 healthy
  • real Chromium QA passed homepage, seeded login, account edit, public two-step lookup, wrong-ZIP mismatch, amended-return search/article, Refund Sent notice filter, and 768 px responsive layout; no console warnings/errors or horizontal overflow

Asset dependency

The companion asset submission remains ChilleD/WebHarbor discussion #27. I verified its open commit c936d3ad013343cc69cafa63bf0d7dad102f85a7 and downloaded irs_refund.tar.gz with SHA-256 d23cf6ffe1db6b3baefebf299288ebbb09d783a016d2a5c7078647cafcd5a4a9; the archive contains the seed DB plus image/cache directories used for the full build. That HF change should be merged and pinned before this environment is treated as self-contained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant