Skip to content

Exit cleanly after test failures - #3

Merged
liufengyun merged 4 commits into
mainfrom
fix/clean-failure-exit
Oct 18, 2026
Merged

liufengyun merged 4 commits into
mainfrom
fix/clean-failure-exit

Conversation

@liufengyun

@liufengyun liufengyun commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Summary

Failing Runner.runWithArgs runs append an exception traceback after the test summary. Exit with status 1 using Python's sys.exit and Ruby's Kernel.exit so the summary remains the final test report and normal runtime cleanup runs.

Add subprocess regression checks for both failure fixtures: status 1, the expected assertion details and summary, matching backend reports, and no runner traceback. Run these checks through the existing output-checking CI step, replacing the shell check that required the old tests failed exception message. Document the exit status in the README and record the fix under Unreleased in CHANGELOG.md.

Add a PR template with a test and changelog checklist and a compatibility section.

Closes #2.

Check list

  • Add tests
  • Update CHANGELOG.md

Validation passed: both runner conformance suites, output parity and clean-exit checks, and all three package builds. git diff --check passed.

Local validation used Jo 0.14.0, Python 3.10.12, and Ruby 3.0.2. CI targets Python 3.12 and Ruby 3.3.

Documentation changes were reviewed and checked with git diff --check; the runtime checks above passed for the implementation commit.

Compatibility

Public API signatures, package dependencies, and minimum runtime versions remain unchanged. Failed Runner.runWithArgs runs still report status 1, with the final abort exception and traceback removed from their output. Callers that need to handle failure themselves can use the Boolean returned by Runner.run.

@liufengyun
liufengyun requested a review from everdance October 18, 2026 18:39
@liufengyun
liufengyun merged commit e4be1f3 into main Oct 18, 2026
1 check passed
@liufengyun
liufengyun deleted the fix/clean-failure-exit branch October 18, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

shouldn't do abort on failed testing

2 participants