Add sample fixture drift checker

[neubig]

Refs #300.

Summary

• add scripts/check_sample_fixtures.py to regenerate sample_std.json from checked-in sample_atif.json
• support checking all datasets, checking selected datasets with --dataset, and refreshing stale fixtures with --update
• report unified diffs for drift without wiring the check into CI yet, because existing open PRs are still cleaning up known fixture drift

Validation

• python scripts/check_sample_fixtures.py --dataset openthoughts_agent_sft
• python -m py_compile scripts/check_sample_fixtures.py

[github-actions]

🟡 Acceptable — the script covers the common case correctly. Two issues worth fixing before wiring this into CI.

This review was generated by an AI agent (OpenHands) on behalf of the repository maintainers.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28748692073

[github-actions]

🟡 Acceptable — one new issue found beyond the two unresolved threads.

The two threads from the prior review (#23 invalid-dataset silent skip, #38 PYTHONPATH overwrite) are still unresolved and should be addressed before this is wired into CI.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28751015249

[github-actions]

🟡 Acceptable — previously unresolved threads (invalid-dataset validation and PYTHONPATH prepend) are both fixed in the current code. One new inconsistency worth correcting before wiring this into CI.

This review was generated by an AI agent (OpenHands) on behalf of the repository maintainers.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754027513

[github-actions]

🟡 Acceptable — all previously-flagged threads are resolved in the current code. One practical concern before this is wired into CI.

[RISK ASSESSMENT]

• [Overall PR] ⚠️ Risk Assessment: 🟢 LOW — developer utility script, not wired into CI, no production data paths affected.

VERDICT: ✅ Worth merging — address the --update stale-SFT gap before enabling in CI.

This review was generated by an AI agent (OpenHands) on behalf of the repository maintainers.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754172006

[github-actions]

🟢 Good taste — all previously flagged threads are resolved and the implementation is correct.

The checker faithfully emulates the reference pipeline: render_jsonl matches json_to_jsonl.py (both use ensure_ascii=True), and the output re-serialization matches jsonl_to_json.py (both produce json.dumps(…, indent=2, ensure_ascii=True) + "\n"). PYTHONPATH prepend, CalledProcessError stderr surfacing, invalid-dataset early exit, and the --update SFT-staleness warning are all in place.

[RISK ASSESSMENT]

• [Overall PR] ⚠️ Risk Assessment: 🟢 LOW — developer utility script, no production data paths, not yet wired into CI.

VERDICT: ✅ Worth merging.

This review was generated by an AI agent (OpenHands) on behalf of the repository maintainers.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/neulab/agent-data-protocol/actions/runs/28754348503