PEP 767: Annotating Read-Only Attributes

[Enegg]

Basic requirements (all PEP Types)

• Read and followed PEP 1 & PEP 12
• File created from the latest PEP template
• PEP has next available number, & set in filename (pep-NNNN.rst), PR title (PEP 123: <Title of PEP>) and PEP header
• Title clearly, accurately and concisely describes the content in 79 characters or less
• Core dev/PEP editor listed as Author or Sponsor, and formally confirmed their approval
• Author, Status (Draft), Type and Created headers filled out correctly
• PEP-Delegate, Topic, Requires and Replaces headers completed if appropriate
• Required sections included
  • Abstract (first section)
  • Copyright (last section; exact wording from template required)
• Code is well-formatted (PEP 7/PEP 8) and is in code blocks, with the right lexer names if non-Python
• PEP builds with no warnings, pre-commit checks pass and content displays as intended in the rendered HTML
• Authors/sponsor added to .github/CODEOWNERS for the PEP

Standards Track requirements

• PEP topic discussed in a suitable venue with general agreement that a PEP is appropriate
• Suggested sections included (unless not applicable)
  • Motivation
  • Rationale
  • Specification
  • Backwards Compatibility
  • Security Implications
  • How to Teach This
  • Reference Implementation
  • Rejected Ideas
  • Open Issues
• Python-Version set to valid (pre-beta) future Python version, if relevant
• Any project stated in the PEP as supporting/endorsing/benefiting from the PEP formally confirmed such
• Right before or after initial merging, PEP discussion thread created and linked to in Discussions-To and Post-History

---

📚 Documentation preview 📚: https://pep-previews--4127.org.readthedocs.build/

[cpython-cla-bot]

All commit authors signed the Contributor License Agreement.

[JelleZijlstra]

Thanks, this will be a great improvement to the type system!

[JelleZijlstra]

I guess we forgot to update the "real names" part here. It's fine to say something like "Enegg [email protected]" (#4058). I'll look into updating the template.

[JelleZijlstra]

Add a recommendation that users write ClassVar[ReadOnly[...]] to avoid this?

[Enegg]

Sure.
I'd assume the wording of

We recommend that in such circumstances ReadOnly is placed as the inner type qualifier, like ClassVar[ReadOnly[...]]

Is it ok to use "we" (the authors) within the PEP? Or perhaps "It is recommended ..." would be better?

[carljm]

"Authors should prefer ClassVar[ReadOnly[...]] over ReadOnly[ClassVar[...]]..." ?

[JelleZijlstra]

I'd write something like "To avoid issues with runtime introspection, use ClassVar[ReadOnly[...]] instead of ReadOnly[ClassVar[...]].

[Enegg]

I assumed the recommendation should apply in the general case 🤔
"To avoid issues with introspection, prefer nesting ReadOnly within other type qualifiers - use ClassVar[ReadOnly[...]] instead of ReadOnly[ClassVar[...]]."
Though it might give the wrong idea that you're always required to nest the type, or that one should put it within a generic that is not a type qualifier

[JelleZijlstra]

Expand this abbreviation

[Enegg]

I have assumed I could be less formal in the Open Issues section, my bad 😄

[JukkaL]

What is the motivation for this change? I'm opposed to changing the semantics of final, since this seems to have some problematic side effects. Currently class-level final attributes can be used to define constants, where the literal value is known statically:

class Foo:
    FOO: Final = 5
    BAR: Final = 6
    
    def __init__(...): ...

x: Literal[5] = Foo.FOO  # OK

(Mypyc even treats Foo.FOO as a compile-time constant at runtime, replacing references to Foo.FOO with 5.)

If we make the change as suggested in the PEP draft, it seems to imply that one of these is true:

1. To infer the type of a class-level final attribute, a type checker will need to combine types from assignments in two scopes: the class body and the __init__ method.
2. The code above would start generating a type error (since the inferred type of Foo.FOO would likely have to be int if we must allow an arbitrary __init__), breaking backward compatibility.

I think (2) above is not reasonable due to the backward compatibility break.

I also argue that (1) would also be quite a drastic change, since it adds "spooky action at a distance" where two assignments in two different parts of a file impact the type of an attribute. Here's what I'm thinking about:

class Foo
    foo: Final = 5  # No longer can use initializer to infer type

    # assume lots of other stuff
    # ...
    
    def __init__(self, foo: int | None) -> None:
        if foo is not None:
            self.foo = foo  # This assignment means that "foo" has type "int"

I don't think there is a precedent for the above kind of type inference behavior, and it would be quite complex to implement in mypy at least. Also, to preserve backward compatibility, it would require new kind of type inference logic at least in mypy, again because there is no precedent for this kind of type inference behavior. This seems unnecessarily complicated considering that ReadOnly seems to be a better fit for the above use case.

In summary, I think that this change seems out of place in the PEP, and increases implementation complexity significantly without a clear benefit that I can see.

[carljm]

I also don't think there is a strong need for Final and ReadOnly to behave the same way, and don't feel we need to make this change to Final in this PEP.

[Enegg]

I didn't initially plan on touching Final. I might have got the wrong idea, but @erictraut's initial feedback seemed to suggest changing the spec of Final to allow assignment at class level and within __init__ ¹:

I’m not convinced that we should allow double initialization — in both the class body and in init. Is there a good reason to allow both initialization techniques within the same class? Interestingly, mypy allows this for Final but pyright does not. The typing spec is currently silent on whether this should be allowed. Regardless of where we land on this decision, it would be good for this PEP to provide clarity for both ReadOnly and Final.

...hence the change. Likely should've discussed this more thoroughly.

Footnotes

1. Under the precondition that ReadOnly ships with the feature ↩

[carljm]

Eric's feedback definitely did suggest clarifying the spec for Final in this PEP, though it didn't advocate for any particular conclusion or change.

My feeling is that we don't need to change the behavior of Final to make it the same as ReadOnly; that causes churn, and I think it's OK if they behave differently. I see this difference as deriving naturally from their different meaning; Final is really supposed to mean "this is the only assignment to this name, it is final", whereas ReadOnly just means "once an instance of this class is constructed, this attribute can't be assigned to."

I (weakly) feel also that it's a bit out of scope for this PEP to clarify existing underspecified aspects of Final, when those could just as well be clarified in a separate PR to the typing spec, approved by the typing council. But I'm not strongly opposed to this. If we do it, I would suggest collecting everything in this PEP related to Final in a single section, which presents a comparison of the semantics of Final and ReadOnly, and any clarifications we want to make to the specification of Final, all together.

[beauxq]

Suggested change

	Type checkers should warn on read-only attributes which may be left uninitialized
	Type checkers may warn on read-only attributes which may be left uninitialized

Making sure that attributes are initialized is not an easy problem. While Python has not really been trying to do it much, Typescript has been trying to do it for a long time. A search in TypeScript issues for "strictPropertyInitialization" yields 98 open issues and 540 closed issues.

TypeScript has an annotation to assert that something is initialized when the type checker can't see it, and Python doesn't, so Python is less-equipped to handle this.

I would like it if type checkers started trying to move towards this. But I think making it a requirement in this PEP is not the right way to push towards it.

[rchen152]

This looks like it'll be a great addition!

[carljm]

Thanks for your work on the PEP! Some things to sort out, but overall it looks strong to me.

[carljm]

As discussed on Discourse, I think in practice this could be simplified to "was the object directly acquired from a call to super-type __new__", which I think is reasonable to track.

And it could also be an option to allow type checker to accept this unsoundness, in exchange for a simpler implementation.

[carljm]

This should probably also mention "calling __init__ on an existing instance" as another action that would wrongly bypass readonly, if not flagged by type checkers.

[carljm]

It would be good to clear this up, but it seems out of scope for this PEP, and like something that could be handled with a PR to the spec and approval by the typing council; it doesn't need a PEP.

[carljm]

What are we seeing ClassVar for here?

[Enegg]

There's this section in the linked spec:

Note that a ClassVar parameter cannot include any type variables, regardless of the level of nesting: ClassVar[T] and ClassVar[list[set[T]]] are both invalid if T is a type variable.

[carljm]

Ah, I see. To be honest, I think this whole footnote can be deleted. There is nothing in the section that it is footnoting that defines T as specifically a typevar, requiring this disclaimer; T is just a stand-in name for "some arbitrary type name". This is very typical in typing documentation/spec and doesn't require further comment, IMO.

[Enegg]

I've changed "object-oriented" to "statically and gradually typed", hopefully it's not a misnomer

[Enegg]

@carljm how do you like this?