I have a big pile of improvements to upstream

[camillol]

I upgraded a large private codebase to Pydantic 2, and in the process I made a bunch of improvements to bump-pydantic. See this branch, and the list of commits.

Here is a list of issues this branch addresses (possibly slightly incomplete, it's lightly edited from a personal log):

• allow_mutation in Config should be converted to frozen Convert allow_mutation to frozen in model config #161
• json_encoders is allowed again, no need for comment json_encoders is back #162
• doe not handle values param in validators
• when replacing validators, classmethod can be duplicated Do not duplicate classmethod when replacing validators #160
• stops everything on the first parse error instead of continuing
• validator(always=True) should be converted to validate_default=True on Field
• Add the ability to migrate Ormar models (for ormar 0.20.0)
• Class definition scanner logic is wrong, e.g. if you have A(B) B(C) C(BaseModel) across three modules
• test_class_def_visitor.py is commented out
• should ignore .git
• tests don’t run ClassDefVisitor
• It thinks any nested Config class is a pydantic config. It should check that the parent is actually a model.
• root_validator gets turned into a model_validator without arguments, which is invalid (needs mode).
• we handled that for root_validator, now also need to handle root_validator()
• smart_union is now the default. It should be automatically removed.
• underscore_attrs_are_private is also now the default and should be removed.
• Scanning for classes takes forever on a large repo, should use Pyre
• In Pydantic 1, the type of fields could be inferred from the default value. In 2, they must have a type annotation.
  • Done for simple types of literal.
  • Added support for type inference.
  • Reduce the FQN for things from the current module
• BaseModel.json is deprecated, should use model_dump_json.
• parse_obj also deprecated. use model_validate
• construct(), copy(), dict(), json_schema(), json(), parse_obj(), update_forward_refs()
• __fields__, __private_attributes__, __validators__
• parse_raw is deprecated and should be replaced with model_validate_json
• Pydantic 2 wants TypedDict to be imported from typing_extensions if using Python < 3.12, because it needs __orig_bases__ which 3.12 introduced
• The validator migration does not recognize pydantic.validator etc.
• Same for the Field migration
• pydantic.Extra.forbid should be replaced with "forbid"
• skip_on_failure=True should be removed from root_validator
• Batch Pyre requests for speed
• Should replace json.loads(x.model_dump_json()) with x.model_dump(mode="json").
• parse_raw_as has been removed entirely. Use TypeAdapter.
• parse_obj_as is not fully removed but should be replaced with TypeAdapter too.
• In root_validators (→ model_validators) with mode="before", the second argument may be an instance of the actual model instead of a dict, in which case you should probably just return it. See example in docs. In fact it could be anything since you can pass anything to model_validate ! Add a TODO comment when migrating
• model_validator(mode="after") needs to be an instance method, not a class method
• For debugging, need a way to run bump-pydantic on a single file while still importing the class hierarchy from a full repo
• Add a TODO/warning for BaseModel subclasses that override deprecated method like dict or json
• If you have an Optional, in model.dict() if model else {} Pyre infers the first model as Optional even though None should be excluded there. work around.
• If you have a field starting with model_, e.g. model_name, Pydantic 2 will complain. You need to add protected_namespaces=() to the ConfigDict
  • model_config cannot be used as a field name at all
  • we need to add a model_config if the original class had no Config
• Sometimes it generates ConfigDict with two extra args
• It does not handle __root__ with a type annotation
• Some migrations break if you have nested BaseModel classes due to incorrect tracking logic.

Most of these changes should be generally useful: they migrate things that were not migrated, or they fix bugs in existing migrations. A few changes are needed to enable running bump_pydantic on a large repository. A couple of things depend on improvements to LibCST which I am also upstreaming.

I sent three of these as separate PRs about two months ago, but there has been no activity. Let's find a way to coordinate on how to get these changes upstreamed. I won't need this code again, but I'd like others to be able to benefit from it.

[povilasb]

That would be really useful for me too 🙇

[samuelcolvin]

Thanks so much for this. I'll get someone to look at it soon.