Skip to content

Add Google Finance mirror + task verifiers (site by @QiushiSun, verifiers by reviewer) - #76

Open
reacher-z wants to merge 3 commits into
aiming-lab:mainfrom
reacher-z:review/google-finance-verifiers
Open

Add Google Finance mirror + task verifiers (site by @QiushiSun, verifiers by reviewer)#76
reacher-z wants to merge 3 commits into
aiming-lab:mainfrom
reacher-z:review/google-finance-verifiers

Conversation

@reacher-z

@reacher-z reacher-z commented Aug 17, 2026

Copy link
Copy Markdown

Reviewer PR for the Google Finance mirror (site by @QiushiSun, original PR #73). The paired assets are in Hugging Face PR #47.

This adds the reviewer-owned grading contract:

  • 20 deterministic verifier entry points under sites/google_finance/verify/
  • task-specific navigation checks to reject prior-knowledge shortcuts
  • frozen-answer checks for all 20 tasks
  • initial/live SQLite comparison for stateful task 19, requiring the exact Bank basket portfolio and JPM lot
  • verifier_path + judge_rubric on every task, with no answer key
  • positive/negative tests and usage documentation

Validation performed against commit a1879e9 and HF refs/pr/47:

  • full Docker build succeeded (5.96 GB)
  • control plane reported all 17 sites alive; every site root returned HTTP 200
  • reset-all returned all sites ready and Google Finance live DB was byte-identical to the seed
  • drove demo-account authentication and task 19 end to end: created Bank basket, cash 5000, one JPM lot of 25 shares at 300; page showed position market value $9,203.00 and gain +22.71%
  • the task-19 verifier passed on that live state and failed after reset with the same self-reported answer
  • compared the live Google Finance and mirror home/quote layouts; exercised the 1Y chart, Key moments and Financials navigation; no local console errors
  • test matrix passes for all tasks: correct PASS, no-op FAIL, correct-answer-without-navigation FAIL, wrong-answer FAIL, state-mismatch FAIL
  • all 20 wrapper scripts were exercised for exit-code behavior; eval_judge.py --verifier True integration passed
  • flake8, py_compile, and git diff --check pass

QiushiSun and others added 3 commits July 25, 2026 20:28
Mirrors https://www.google.com/finance/ as site 17 on port 40016.

Registers the site in the three required places (websyn_start.sh SITES,
control_server.py SITES, Dockerfile EXPOSE 40000-40016) and pins
MarkupSafe==3.0.3, which app.py imports directly for its server-rendered
SVG charts.

Covers the home page with seven region tabs, quote pages for stocks, ETFs,
indexes, sector indexes, crypto, FX and futures across eight chart ranges
and five tabs, eight market boards, scored search with autocomplete, news,
a currency converter, a compare view, and auth-gated watchlists and
portfolios. 10,275 seeded rows across 18 models; market data is synthetic
and frozen at 2026-07-24, company identities and logos are real.

20 benchmark tasks in sites/google_finance/tasks.jsonl, contributor keys
only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fetch_assets.sh otherwise resolves a revision that predates the paired
assets PR, and websyn_start.sh copies instance_seed to instance for every
site under set -e — so a missing directory aborts the whole entrypoint and
no site comes up, not just this one.

Verified from a clean clone of this branch: fetch_assets.sh, build.sh, and
a container on 41000-41016 give /health ok with 17/17 alive and every port
200, with the seed DB byte-identical after POST /reset/google_finance.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants