Fix TestSpec serialization silently dropping test ids that contain quotes

[eeshsaxena]

TestSpec.to_fail_to_pass_str/to_pass_to_pass_str build the list string by hand, wrapping each test id in single quotes:

return "[" + ", ".join(f"'{t}'" for t in self.fail_to_pass) + "]"

If a test id contains a quote, the result is invalid Python. Parametrized pytest ids do this in practice, e.g. tests/test_x.py::test_quote[can't] becomes ['tests/test_x.py::test_quote[can't]'].

load_task_from_directory writes these into instance_info.txt and later parses them back with eval(). When the string is malformed the except Exception branch runs, prints a warning, and replaces the whole list with [], so the task silently loses its fail_to_pass/pass_to_pass spec and is evaluated against no tests.

Using repr(list(...)) emits a valid literal for any test id, so it round-trips. Output is byte-for-byte identical for ids without quotes, so existing datasets are unaffected.

Added tests/test_wizard_models.py covering the common, quoted, and empty cases; the quoted case fails on main (the parser raises SyntaxError) and passes here. The repo had no test setup, so this also adds pytest as a dev dependency with a small [tool.pytest.ini_options] block.

uv run pytest tests/
# 3 passed

Summary by CodeRabbit

• Bug Fixes

  • Improved test specification serialization to preserve quoted test IDs and support reliable round-trip parsing.
  • Added handling for empty test specifications.

• Tests

  • Added coverage for common test IDs and IDs containing apostrophes or double quotes.
  • Added pytest configuration for the project’s source and test directories.

[coderabbitai]

📝 Walkthrough

Walkthrough

The project adds pytest development configuration. TestSpec now uses repr to serialize test ID lists. Tests verify round-trip parsing for ordinary, quoted, and empty test IDs.

Changes

TestSpec serialization

Layer / File(s)	Summary
Pytest development setup pyproject.toml	Adds pytest to the development dependency group and configures the Python path and test directory.
TestSpec serialization round-trips src/anvil/wizard/models.py, tests/test_wizard_models.py	Uses repr for both serialized test ID lists. Tests verify ordinary IDs, embedded quotes, and empty lists with ast.literal_eval.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: preserving test IDs that contain quotes during TestSpec serialization.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/test_wizard_models.py (1)

27-28: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Exercise the production instance_info.txt parser.

The assertions at Line 27 and Line 28 call ast.literal_eval directly. The reported failure occurs in the parser read path, where invalid literals can become empty lists. Add a test through that parser for both quoted fields to verify the complete round trip.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_wizard_models.py` around lines 27 - 28, Replace the direct
ast.literal_eval assertions in the test with coverage through the production
instance_info.txt parser, exercising both quoted fields and verifying their
complete round-trip values. Ensure the test captures the parser read path,
including preservation of expected values rather than silently converting
invalid literals to empty lists.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/test_wizard_models.py`:
- Around line 27-28: Replace the direct ast.literal_eval assertions in the test
with coverage through the production instance_info.txt parser, exercising both
quoted fields and verifying their complete round-trip values. Ensure the test
captures the parser read path, including preservation of expected values rather
than silently converting invalid literals to empty lists.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7ce76f4e-54dc-4bea-8ec4-9c410a7d585a

📥 Commits

Reviewing files that changed from the base of the PR and between 13b47ac and b9bd450.

📒 Files selected for processing (3)

• pyproject.toml
• src/anvil/wizard/models.py
• tests/test_wizard_models.py

[eeshsaxena]

Hi! Gentle nudge on this one whenever you have some bandwidth. It's a small, self-contained fix (Fix TestSpec serialization silently dropping test ids that contain quotes), and it's currently mergeable with no conflicts. No urgency at all, and I'm happy to make any changes you'd like. Thanks for maintaining anvil!