Repository navigation
fix special-case NotImplementedType in binop signatures to match runtime semantics #1129 - #2677
asukaminato0721 wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes issue #1129, where binary operations involving dunder methods that return NotImplementedType would incorrectly leak the NotImplementedType into the inferred result type instead of following Python's runtime semantics (trying the reflected dunder when the forward one signals NotImplemented).
Changes:
- The
try_binop_callslogic inoperators.rsis updated to stripNotImplementedTypefrom successful dunder return types, accumulate non-NotImplementedTyperesults, and continue searching for reflected dunders when needed. NotImplementedTypeis added as a new stdlib entry inStdlib, gated to Python ≥ 3.10 (consistent with howEllipsisTypeis handled).- A regression test is added covering both the pure-
NotImplementedTypefallback and the mixedint | NotImplementedTypescenario.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
pyrefly/lib/alt/operators.rs |
Core fix: strips NotImplementedType from successful dunder results, accumulates partial results, and continues to reflected dunders when needed |
crates/pyrefly_types/src/stdlib.rs |
Adds NotImplementedType as an Option<StdlibResult<ClassType>> stdlib entry, guarded by version >= 3.10 |
pyrefly/lib/test/operators.rs |
Regression test covering the bug from issue #1129 for both pure and mixed NotImplementedType return types |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
89d59e9 to
a8f8e7f
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
stroxler
left a comment
There was a problem hiding this comment.
I don't think we can merge this as is - the mypy primer noise is a hint that something is wrong here.
I think the issue is probably tied to NotImplementedType vs Never / NoReturn.
Some AI review suggestions:
Blocking finding
Preserve genuinely non-returning dunders — [operators.rs:265–274](
pyrefly/pyrefly/lib/alt/operators.rs
Lines 265 to 274 in a987c5b
The is_never() check conflates two cases:
- The method returns only
NotImplementedType, so dispatch should continue. - The method returns
Never, so dispatch cannot continue.
With A.__add__(object) -> Never and B.__radd__(A) -> str, the patched checker infers str for A() + B(). At runtime, A.__add__ raises and the reflected method never executes. Without a reflected method, this instead produces a false unsupported-operation error. The same issue affects augmented assignment.
Fix: detect whether NotImplementedType was actually present; leave an original Never unchanged. This also matches the Never/NoReturn failures visible in the PR’s primer differential.
Other review concerns
- The mixed-return test does not match runtime semantics. The added test asserts
C() + D()isint, but executing that exact example returnsstr. For a generalint | NotImplementedTypereturn, reflected results remain possible. If strip-only behavior is intentional, describe it as an approximation rather than complete runtime modeling. - Keep the sentinel-absent fast path unchanged. Every successful dunder now allocates a vector, clones its result, and rebuilds unions—even without
NotImplementedType. Selective stripping would avoid this cost.
|
Diff from mypy_primer, showing the effect of this PR on open source code: ============================================================
SUMMARY
============================================================
Total: +7 new errors, -1 fixed errors
By preset: +2/-0 (default), +2/-0 (strict)
Projects with changes (2):
openlibrary: +4 -1
xarray: +3 -0
============================================================
FULL DIFF DETAILS
------------------------------------------------------------
openlibrary (https://github.com/internetarchive/openlibrary)
- ERROR openlibrary/book_providers.py:596:92-155: Implicit conversion of `Acquisition | NotImplementedType | Unknown | None` to `bool` is not allowed [implicit-bool]
+ ERROR openlibrary/book_providers.py:596:92-155: Implicit conversion of `Acquisition | Unknown | None` to `bool` is not allowed [implicit-bool]
+ ERROR openlibrary/solr/updater/edition.py:353:16-63: Returning implicit Any from function declared to return "bool" [no-any-return-implicit]
+ ERROR openlibrary/solr/updater/edition.py:526:16-75: Returning implicit Any from function declared to return "bool" [no-any-return-implicit]
+ ERROR openlibrary/solr/updater/edition.py:530:16-78: Returning implicit Any from function declared to return "bool" [no-any-return-implicit]
xarray (https://github.com/pydata/xarray)
+ ERROR xarray/core/resample_cftime.py:459:23-34: `+` is not supported between `datetime64` and `Tick` [unsupported-operation]
+ ERROR xarray/tests/test_cftime_offsets.py:558:14-29: `+` is not supported between `Day` and `Day` [unsupported-operation]
+ ERROR xarray/tests/test_cftime_offsets.py:565:14-29: The type of `result` is partially unknown; it contains an implicit `Any` [partially-unknown-type] |
stroxler
left a comment
There was a problem hiding this comment.
Looks like a good improvement! This is big change though - fresh PR review and analysis of mypy_primer results flagged a few more issues we should address before we merge
-
Inferred placeholder returns falsely reject inherited operators.
[operators.rs:268–283]()pyrefly/pyrefly/lib/alt/operators.rs
Lines 268 to 283 in ba97ed5
This explains the remaining xarray
Day + Dayprimer error. The inherited__add__callsself.__apply__; Pyrefly infers its return using the base implementation, which returnsNotImplemented.Dayoverrides__apply__, so the operation actually succeeds—but the patch treats the inferred placeholder as definitive rejection.I reproduced this with a minimal base/subclass example: no errors before the patch, unsupported-operation afterward, and CPython returns
Day. The fix should account for inferred placeholder returns before treating them like explicitly annotated sentinel-only returns. -
Equality needs identity fallback when both methods decline.
[operators.rs:294–310]()pyrefly/pyrefly/lib/alt/operators.rs
Lines 294 to 310 in ba97ed5
When both equality methods return
NotImplemented, Python falls back to identity comparison. The patch instead reports unsupported-operation. Mixed returns are also affected: methods annotatedLiteral[True] | NotImplementedTypeproduce inferredLiteral[True], although comparing distinct instances can returnFalse.Preserve a
boolidentity-fallback branch for==/!=, and cover both pure and mixed sentinel returns. -
The pure-sentinel arm drops deprecation diagnostics. [Line 269](
) continues before propagatingpyrefly/pyrefly/lib/alt/operators.rs
Line 269 in ba97ed5
callee_errors. I verified a deprecated forward method warns before the patch but not afterward. Preserve those diagnostics before continuing.
Summary
Fixes #1129
fixed the binop resolution path so NotImplementedType no longer leaks as a possible operator result.
successful dunder calls now strip only the exact NotImplementedType branch, keep searching reflected dunders when needed, and union any concrete results that can actually occur at runtime.
Test Plan
a regression test covering both pure fallback and mixed
int | NotImplementedTypebehavior.