Repository navigation
Update call to match pytest-benchmark >= 5.3.0 - #20
Merged
Merged
Conversation
pytest-benchmark >= 5 declares --benchmark-json with type=Path and calls open(path, "wb") itself when saving results. The --lbench setup passed an already-open file object, so saving raised a TypeError and pytest-benchmark.json was left empty (0 bytes). Set benchmark_json to the path instead and require pytest-benchmark>=5, whose option handling this relies on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #20 +/- ##
=====================================
Coverage 3.48% 3.48%
=====================================
Files 26 26
Lines 1062 1062
=====================================
Hits 37 37
Misses 1025 1025 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
smcguire-cmu
approved these changes
Oct 13, 2026
smcguire-cmu
left a comment
smcguire-cmu
left a comment
Collaborator
There was a problem hiding this comment.
Awesome! I hated this at the time, it's nice to see it updated
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Avoids empty pytest-benchmark.json when running with --lbench, with newer versions of pytest-benchmark.
I went in with my human eyes and tracked down the change to an upstream commit made just before pytest-benchmark v5.3.0: ionelmc/pytest-benchmark@cb91999
What had broken:
Running
pytest --lbenchdidn't save benchmark results.At the end of the session, pytest-benchmark would raise
TypeError: expected str, bytes or os.PathLike object, not BufferedWriter, and leave behind the already-created 0 byte<run_dir>/pytest-benchmark.json.Why:
We were setting
config.option.benchmark_jsonto an open file handle, which was totally right at the time.But now, pytest-benchmark >= 5.3.0 declares it with
type=Pathand opens the file itself when trying to save results - so, the already-open file object fails, and the file created at startup stays empty.Fix:
benchmark_jsonto a path and let pytest-benchmark open itpytest-benchmark>=5.3.0, since this relies on its path-based option