More changes

Author: JelleZijlstra

peps/pep-0749.rst

@@ -198,10 +198,11 @@ The module will contain the following functionality:
   module, or class. This will replace :py:func:`inspect.get_annotations`. The latter
   will delegate to the new function. It may eventually be deprecated, but to
   minimize disruption, we do not propose an immediate deprecation.
-* ``get_annotate_function()``: A function that returns the ``__annotate__`` function
-  of an object, if it has one, or ``None`` if it does not. This is usually equivalent
-  to accessing the ``.__annotate__`` attribute, except in the presence of metaclasses
-  (see :ref:`below <pep749-metaclasses>`).
+* ``get_annotate_from_class_namespace(namespace: Mapping[str, Any])``: A function that
+  returns the ``__annotate__`` function from a class namespace dictionary, or ``None``
+  if there is none. This is useful in metaclasses during class construction. It is
+  a separate function to avoid exposing implementation details about the internal storage
+  for the ``__annotate__`` function (see :ref:`below <pep749-metaclasses>`).
 * ``Format``: an enum that contains the possible formats of annotations. This will
   replace the ``VALUE``, ``FORWARDREF``, and ``SOURCE`` formats in :pep:`649`.
   PEP 649 proposed to make these values global members of the :py:mod:`inspect`
@@ -238,7 +239,7 @@ The module will contain the following functionality:
   This is useful for
   implementing the ``SOURCE`` format in cases where the original source is not available,
   such as in the functional syntax for :py:class:`typing.TypedDict`.
-* ``value_to_string(value: object) -> str``: a function that converts a single value to a
+* ``type_repr(value: object) -> str``: a function that converts a single value to a
   string representation. This is used by ``annotations_to_string``.
   It uses ``repr()`` for most values, but for types it returns the fully qualified name.
   It is also useful as a helper for the ``repr()`` of a number of objects in the
@@ -501,45 +502,64 @@ attribute lookup is used, this approach breaks down in the presence of
 metaclasses, because entries in the metaclass's own class dictionary can render
 the descriptors invisible.
 
-While we considered several approaches that would allow ``cls.__annotations__``
-and ``cls.__annotate__`` to work reliably when ``cls`` is a type with a custom
-metaclass, any such approach would expose significant complexity to advanced users.
-Instead, we recommend a simpler approach that confines the complexity to the
-``annotationlib`` module: in ``annotationlib.get_annotations``, we bypass normal
-attribute lookup by using the ``type.__annotations__`` descriptor directly.
+We considered several solutions but landed on one where we store the ``__annotate__``
+and ``__annotations__`` objects in the class dictionary, but under a different,
+internal-only name. This means that the class dictionary entries will not interfere
+with the descriptors defined on :py:class:`type`.
+
+This approach means that the ``.__annotate__`` and ``.__annotations__`` objects in class
+objects will behave mostly intuitively, but there are a few downsides.
+
+One concerns the interaction with classes defined under ``from __future__ import annotations``.
+Those will continue to have the ``__annotations__`` entry in the class dictionary, meaning
+that they will continue to display some buggy behavior. For example, if a metaclass is defined
+with the ``__future__`` import enabled and has annotations, and a class using that metaclass is
+defined without the ``__future__`` import, accessing ``.__annotations__`` on that class will yield
+the wrong results. However, this bug already exists in previous versions of Python. It could be
+fixed by setting the annotations at a different key in the class dict in this case too, but that
+would break users who directly access the class dictionary (e.g., during class construction).
+We prefer to keep the behavior under the ``__future__`` import unchanged as much as possible.
+
+Second, in previous versions of Python it was possible to access the ``__annotations__`` attribute
+on instances of user-defined classes with annotations. However, this behavior was undocumented
+and not supported by :func:`inspect.get_annotations`, and it cannot be preserved under the
+:pep:`649` framework without bigger changes, such as a new ``object.__annotations__`` descriptor.
+This behavior change should be called out in porting guides.
 
 Specification
 -------------
 
-Users should always use ``annotationlib.get_annotations`` to access the
-annotations of a class object, and ``annotationlib.get_annotate_function``
-to access the ``__annotate__`` function. These functions will return only
-the class's own annotations, even when metaclasses are involved.
+The ``.__annotate__`` and ``.__annotations__`` attributes on class objects
+should reliably return the annotate function and the annotations dictionary,
+respectively, even in the presence of custom metaclasses.
 
-The behavior of accessing the ``__annotations__`` and ``__annotate__``
-attributes on classes with a metaclass other than ``builtins.type`` is
-unspecified. The documentation should warn against direct use of these
-attributes and recommend using the ``annotationlib`` module instead.
-
-Similarly, the presence of ``__annotations__`` and ``__annotate__`` keys
-in the class dictionary is an implementation detail and should not be relied
-upon.
+Users should not access the class dictionary directly for accessing annotations
+or the annotate function; the data stored in the class dictionary is an implementation
+detail and its format may change in the future. If only the class namespace
+dictionary is available (e.g., while the class is being constructed),
+``annotationlib.get_annotate_function`` may be used to retrieve the annotate function
+from the class dictionary.
 
 Rejected alternatives
 ---------------------
 
-We considered two broad approaches for dealing with the behavior
+We considered three broad approaches for dealing with the behavior
 of the ``__annotations__`` and ``__annotate__`` entries in classes:
 
 * Ensure that the entry is *always* present in the class dictionary, even if it
   is empty or has not yet been evaluated. This means we do not have to rely on
   the descriptors defined on :py:class:`type` to fill in the field, and
   therefore the metaclass's attributes will not interfere. (Prototype
   in `gh-120719 <https://github.com/python/cpython/pull/120719>`__.)
+* Warn users against using the ``__annotations__`` and ``__annotate__`` attributes
+  directly. Instead, users should call function in ``annotationlib`` that
+  invoke the :class:`type` descriptors directly. (Implemented in
+  `gh-122074 <https://github.com/python/cpython/pull/122074>`__.)
 * Ensure that the entry is *never* present in the class dictionary, or at least
   never added by logic in the language core. This means that the descriptors
   on :py:class:`type` will always be used, without interference from the metaclass.
-  (Prototype in `gh-120816 <https://github.com/python/cpython/pull/120816>`__.)
+  (Initial prototype in `gh-120816 <https://github.com/python/cpython/pull/120816>`__;
+  later implemented in `gh-132345 <https://github.com/python/cpython/pull/132345>`__.)
 
 Alex Waygood suggested an implementation using the first approach. When a
 heap type (such as a class created through the ``class`` statement) is created,
@@ -561,19 +581,8 @@ While this approach would fix the known edge cases with metaclasses, it
 introduces significant complexity to all classes, including a new built-in type
 (for the annotations descriptor) with unusual behavior.
 
-The alternative approach would be to never set ``__dict__["__annotations__"]``
-and use some other storage to store the cached annotations. This behavior
-change would have to apply even to classes defined under
-``from __future__ import annotations``, because otherwise there could be buggy
-behavior if a class is defined without ``from __future__ import annotations``
-but its metaclass does have the future enabled. As :pep:`649` previously noted,
-removing ``__annotations__`` from class dictionaries also has backwards compatibility
-implications: ``cls.__dict__.get("__annotations__")`` is a common idiom to
-retrieve annotations.
-
-This approach would also mean that accessing ``.__annotations__`` on an instance
-of an annotated class no longer works. While this behavior is not documented,
-it is a long-standing feature of Python and is relied upon by some users.
+The second approach is simple to implement, but has the downside that direct
+access to ``cls.__annotations__`` remains prone to erratic behavior.
 
 Adding the ``VALUE_WITH_FAKE_GLOBALS`` format
 =============================================
@@ -611,10 +620,17 @@ the ``VALUE_WITH_FAKE_GLOBALS`` format is requested, so the standard
 library will not call the manually written annotate function with
 "fake globals", which could have unpredictable results.
 
+The names of annotation formats indicate what kind of objects an
+``__annotate__`` function should return: with the ``STRING`` format, it
+should return strings; with the ``FORWARDREF`` format, it should return
+forward references; and with the ``VALUE`` format, it should return values.
+The name ``VALUE_WITH_FAKE_GLOBALS`` indicates that the function should
+still return values, but is being executed in an unusual "fake globals" environment.
+
 Specification
 -------------
 
-An additional format, ``FAKE_GLOBALS_VALUE``, is added to the ``Format`` enum in the
+An additional format, ``VALUE_WITH_FAKE_GLOBALS``, is added to the ``Format`` enum in the
 ``annotationlib`` module, with value equal to 2. (As a result, the values of the
 other formats will shift relative to PEP 649: ``FORWARDREF`` will be 3 and ``SOURCE``
 will be 4.)
@@ -625,10 +641,10 @@ they would return for the ``VALUE`` format. The standard library will pass
 this format to the ``__annotate__`` function when it is called in a "fake globals"
 environment, as used to implement the ``FORWARDREF`` and ``SOURCE`` formats.
 All public functions in the ``annotationlib`` module that accept a format
-argument will raise :py:exc:`NotImplementedError` if the format is ``FAKE_GLOBALS_VALUE``.
+argument will raise :py:exc:`NotImplementedError` if the format is ``VALUE_WITH_FAKE_GLOBALS``.
 
 Third-party code that implements ``__annotate__`` functions should raise
-:py:exc:`NotImplementedError` if the ``FAKE_GLOBALS_VALUE`` format is passed
+:py:exc:`NotImplementedError` if the ``VALUE_WITH_FAKE_GLOBALS`` format is passed
 and the function is not prepared to be run in a "fake globals" environment.
 This should be mentioned in the data model documentation for ``__annotate__``.