Skip to content

Widen the synthesized __init__ parameter for fields with a custom pydantic schema - #5432

Open
soaring0616 wants to merge 2 commits into
meta:mainfrom
soaring0616:pydantic-custom-schema-init
Open

soaring0616 wants to merge 2 commits into
meta:mainfrom
soaring0616:pydantic-custom-schema-init

Conversation

@soaring0616

@soaring0616 soaring0616 commented Oct 28, 2026 •

Copy link
Copy Markdown

Summary

Pyrefly reports a false positive when a model is constructed with a value that the field's custom schema accepts, e.g. CustomModel(dtype=">i4"). The call is valid at runtime. The error appears in strict mode, and in lax mode for types that are in the lax conversion table (e.g. int, which becomes LaxInt).

This PR adds a pydantic_has_custom_schema flag to DataclassFieldKeywords. It is set when an Annotated metadata item has a __get_pydantic_core_schema__ hook that is defined outside Pydantic (or by Pydantic's BeforeValidator, PlainValidator or WrapValidator). Only get_dataclass_init reads it, to widen the synthesized __init__ parameter.

The widening applies in both strict and lax mode.

Fixes #2586

  • I am a human, and I fully understand the code I am submitting.

What changed

  • keywords.rs: adds the pydantic_has_custom_schema field.
  • pydantic.rs: sets the flag when a metadata item's __get_pydantic_core_schema__ hook is defined outside Pydantic. Hooks defined by Pydantic are skipped because they generally build on the declared type's own validation (for example, UuidVersion adds a constraint to it), so the accepted input type does not change. The exceptions are BeforeValidator, PlainValidator and WrapValidator, which receive the raw input. Metadata that is not a class (such as Any, which is what Field(gt=0) is inferred as) never counts.
  • dataclass.rs: get_dataclass_init reads the flag. When it is set, the local variable converter_param becomes Any, so only the synthesized __init__ parameter is widened.
  • class_field.rs: on the AssignedInBody path (fields with a default value), only strict was copied from the annotation flags, so the flag was dropped and the __init__ parameter kept the declared type. The flag is now carried at two places: when building annotation_flags (the filter no longer discards annotations that only set the flag, and the flag is copied), and when merging them into the flags of a right-hand-side Field(...) (f.pydantic_has_custom_schema |= annotation_flags.pydantic_has_custom_schema).

Why a separate flag

An earlier attempt (#3798) set the stored converter_param to Any. That field is read in three places: get_dataclass_init, get_dataclass_replace, and attribute assignment (resolve_class_attr_setter). Widening it would also widen attribute assignment, so m.dtype = ">i4" would be wrongly accepted when it should be rejected.

The new flag is read only by get_dataclass_init, so only the __init__ parameter is widened. test_annotated_custom_schema_widens_init_param covers this: it expects m.dtype = ">i4" to still be an error.

Test Plan

  • test_annotated_custom_schema_widens_init_param: in strict mode, __init__ accepts a value for a custom-schema field, while m.dtype = ">i4" is still rejected. It also checks that a field using Pydantic's UuidVersion is not widened.
  • test_annotated_custom_schema_with_default: the same for a field with a default value (the AssignedInBody path).
  • test_annotated_custom_schema_inherited: a subclass that inherits the field from its parent (strict=True) also gets the widened __init__ parameter.
  • test_annotated_custom_schema_widens_lax_conversion_table_type: in lax mode, a custom-schema field gets Any, while a normal int field still gets LaxInt.
  • test_annotated_metadata_without_custom_schema: metadata without __get_pydantic_core_schema__, including Field(gt=0), does not widen the parameter.

I ran cargo test -p pyrefly --lib pydantic::field; all 40 tests pass.
The full cargo test aborts on deep_phi_chain_term_expr with a stack overflow. This also happens on main, so I skipped that test with --skip.

Not included

  • __replace__ is not handled here.
  • BeforeValidator, PlainValidator and WrapValidator are not covered by a test yet: the Pydantic copy in the test environment does not include functional_validators.py.

AI disclosure

  • I used Claude (Sonnet 5.5) to study the codebase, assist with implementation and debugging, and polish my English.
  • I built and verified the change on my local machine, and ran a runtime experiment to confirm that assigning to the attribute does not convert the value.

Pyrefly incorrectly reported an error when constructing a model with a custom schema field like `CustomModel(dtype=">i4")`, although the call is valid at runtime.

The earlier attempt (meta#3798) widened the stored `converter_param`, which is also used for attribute assignment, and its check also matched metadata inferred as `Any`, so errors like `Model(y="1")` were missed.

Add an `init_param_any` flag to `DataclassFieldKeywords`. It is set when an `Annotated` metadata item defines `__get_pydantic_core_schema__`, and only `get_dataclass_init` reads it to widen the synthesized `__init__` parameter.

Metadata inferred as `Any` is not treated as a custom schema, so such fields are still checked.

The widening applies in both lax and strict mode.

Fixes meta#2586
@meta-cla

meta-cla Bot commented Oct 28, 2026

Copy link
Copy Markdown

Hi @soaring0616!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@meta-cla

meta-cla Bot commented Oct 28, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@meta-cla meta-cla Bot added the cla signed label Oct 28, 2026
@rchen152 rchen152 self-assigned this Oct 28, 2026
@github-actions

This comment has been minimized.

@rchen152 rchen152 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I left a couple of small suggestions to rename the init_param_any field and exclude classes defined in Pydantic itself from this behavior. Once that's done, I think this will be good to merge!

Comment thread crates/pyrefly_types/src/keywords.rs Resolved
Comment thread pyrefly/lib/alt/class/pydantic.rs Outdated
Comment on lines +724 to +726
let metadata_ty = self.expr_infer(metadata_item, &ignore_errors);
!metadata_ty.is_any()
&& self.has_static_attr(&metadata_ty, &GET_PYDANTIC_CORE_SCHEMA)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pydantic itself defines some classes that use __get_pydantic_core_schema__, but the only ones that should get __init__ parameter widening are BeforeValidator, PlainValidator, and WrapValidator. Other than those three, classes defined in Pydantic should be excluded. I think something like this should do the trick (caveat: I haven't compiled this, so consider it pseucode):

let cls = match self.expr_infer(metadata_item, &ignore_errors) {
    Type::ClassType(cls) => cls.class_object().clone(),
    Type::ClassDef(cls) => cls,
    _ => return false,
};
self.get_class_member_with_defining_class(&cls, &GET_PYDANTIC_CORE_SCHEMA)
    .is_some_and(|member| {
        let owner = &member.defining_class;
        owner.module_name().first_component() != "pydantic"
        || matches!(
            owner.name().as_str(),
            "BeforeValidator" | "PlainValidator" | "WrapValidator"
        )
    })

- Rename `init_param_any` to `pydantic_has_custom_schema`, so that the name
  describes a fact about the field rather than how the flag is used.
- Only a `__get_pydantic_core_schema__` hook defined outside Pydantic counts
  as a custom schema. Hooks defined by Pydantic are skipped because they
  generally build on the declared type's own validation. `BeforeValidator`,
  `PlainValidator` and `WrapValidator` are the exceptions, since they receive
  the raw input.
- Add a test showing that `UuidVersion` does not widen the `__init__`
  parameter.
@github-actions github-actions Bot added size/l and removed size/l labels Oct 30, 2026
@meta-codesync

meta-codesync Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D128513616. (Because this pull request was imported automatically, there will not be any future comments.)

@soaring0616

Copy link
Copy Markdown
Author

Thanks for the review! I pushed another commit (1f07f21) with some changes:

  • Renamed init_param_any to pydantic_has_custom_schema.
  • A hook now counts as a custom schema only if the class that defines it is outside Pydantic, or is one of BeforeValidator, PlainValidator and WrapValidator. (Thanks for your suggestion!)
  • I added a test with Pydantic’s UuidVersion to check that it is not widened.

One question about testing, related to the second comment (Claude suggests me to ask): the Pydantic copy under pyrefly/lib/test/pydantic/third-party/pydantic/ has no functional_validators.py, so I can’t test those three exceptions. Should I add a minimal stub for them, or is there a better way?

Also, I accidentally resolved one of the review threads, sorry about that, and I couldn’t find it to reopen it myself. If it doesn’t matter, please feel free to ignore this. But if I overlooked something, could you please reopen it? Thanks!

@github-actions

Copy link
Copy Markdown

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@rchen152 rchen152 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thank you!

It looks like you already addressed the review thread you resolved, so all good :)

To test classes in functional_validators, your suggestion to add minimal stubs is the right one, but I think the test coverage is already fine and it's okay to omit them.

@stroxler stroxler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bad-argument-type on Annotated pydantic fields with custom schema

3 participants