Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 20 additions & 18 deletions peps/pep-0849.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ as ``Literal[1, 2, 3]``. Many users intuitively want to instead spell this as
``1 | 2 | 3``, writing out the bare literals and using the union operator to join
them, which also is what other languages such as TypeScript use. This is
currently not possible in Python because the expression ``1 | 2 | 3`` evaluates
simply to ``3`` because ``|`` is interpreted as the binary-or operation rather than
simply to ``3`` because ``|`` is interpreted as the bitwise-or operation rather than
the union of types. Further, constant folding eliminates this
expression from appearing anywhere in the generated bytecode, and it thus is
impossible to retrieve the actual type annotation at runtime.
Expand Down Expand Up @@ -118,7 +118,7 @@ usual annotation dict ``{"a": int, "b": list[str]}``. But when called as

{
"a": AnnotationAST(ast.Name("int"), {"int": int}),
"b": AnnotationAST(ast.Subscript(ast.Name("list"), ast.Name("str")), {"list": list, "str": str),
"b": AnnotationAST(ast.Subscript(ast.Name("list"), ast.Name("str")), {"list": list, "str": str}),
}

However, most users will never see these objects directly. Rather, we propose
Expand All @@ -134,7 +134,7 @@ allow complete runtime introspection.

We also propose to add some additional utility functionality related to
annotate functions and these AST objects. In particular, a new
``create_annoate_function`` in the :py:mod:`annotationlib` module to easily
``create_annotate_function`` in the :py:mod:`annotationlib` module to easily
synthesize an annotate function. The core of this functionality is already
implemented in the :py:mod:`dataclasses` module, and we foresee that many
users will need this in order to create annotate functions that support
Expand All @@ -150,11 +150,11 @@ the class from above:

.. code-block:: python

MyClass:
class MyClass:
a: int
b: list[str]

Under this proposal its annotate function may behave similar to this Python code:
Under this proposal its annotate function may behave similarly to this Python code:

.. code-block:: python

Expand Down Expand Up @@ -185,13 +185,13 @@ using a function like this:

.. code-block:: python

def eval_type_annotation_AST(ast, namespace):
match ast:
def eval_type_annotation_AST(node, namespace):
match node:
case ast.Name:
return namespace[ast.id]
return namespace[node.id]
case ast.Attribute:
value = eval_type_annotation_AST(ast.value, namespace)
return getattr(value, ast.attr)
value = eval_type_annotation_AST(node.value, namespace)
return getattr(value, node.attr)
...


Expand Down Expand Up @@ -236,7 +236,7 @@ of users, who also benefit from the newly possible type annotations, we consider
this a worthwhile trade-off. The specification of the proposed format is also
open enough that many optimizations are possible should they be deemed necessary
in the future. For many users, this proposal will even be a slight performance
increase since their code never evaluates any annotate functions.
improvement since their code never evaluates any annotate functions.

.. _outside-annos:

Expand Down Expand Up @@ -289,13 +289,15 @@ The ``AST`` Format
A new value called ``AST`` is added to the ``annotationlib.Format`` enum with
value 5. Annotate functions do not have to support this format. If an
annotate function is called with this format, it must return a
``AnnotationAST`` object. These are instances of a proposed new class that
dict with str keys and ``AnnotationAST`` objects as values (or a single ``AnnotationAST`` object

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The prose earlier said it returns a dictionary of strings to AnnotationAST objects.

Though maybe a single AnnotationAST object would make sense? The namespace is the same for all annotations in a namespace, after all.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My original implementation used the layout that you're describing. But in the discourse thread David Ellis requested the nesting to be flipped in order to support synthesizing annotate functions more easily. Essentially, if each annotate object has its own reference to its namespace we can merge e.g. dataclasses from different modules without having to worry about namespace issues. The compiler generated annotate functions also don't actually create duplicate namespace objects, so memory usage isn't a big problem.

This layout also has the advantage that every annotation format returns a dict mapping variable names to some data. When I implemented the other version I had to add a lot of boilerplate case distinctions to things like the dataclasses or annotationlib modules.

for evaluate functions, like ``evaluate_value`` on ``TypeAliasType``).
These are instances of a proposed new class that
hold the annotation's AST as an ``ast.expr`` object and a namespace that is
used to evaluate them.

Compiler-generated annotate functions will always support this format.
They will store the necessary AST data stored as string constants and then
construct a new ``AnnotationAST`` objects each time they are called. The
They will store the necessary AST data as string constants and then

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to the grammatical fix on this line: shouldn't this be bytes constants?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the implementation is using genuine str objects. Bytes have the advantage that the encoding is slightly simpler since you don't need to worry about it being valid UTF-8. But strings let us intern them, which slightly reduces memory usage (about 1.5% of overall module/.pyc size). Execution time (both of the annotate call and module import) is the same for both types. I don't have a strong stance on which is better, if there's a reason to prefer one over the other I'm happy to just go with that.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could build bytes interning as part of this PEP, but I guess strings are fine too.

construct new ``AnnotationAST`` objects each time they are called. The
contained AST objects will be identical to the objects created by parsing the
annotation source code directly.

Expand Down Expand Up @@ -363,7 +365,7 @@ about existing user annotations being reinterpreted to different objects when
the library is updated. But this also is not an issue. The typing spec already
is covered by backwards compatibility concerns, which means that any currently
valid type annotations will not be changed to mean something different.
Potential future changes can only define new semantics to syntax constructs that
Potential future changes can only define new semantics for syntactic constructs that
currently are not valid type annotations and thus do not occur in user code.


Expand Down Expand Up @@ -424,15 +426,15 @@ relevant typing objects themselves. This can be achieved in several ways, for
example via a new optional argument to signal typing-specific semantics or a new
format. These approaches were rejected because they force the type-specific
semantics to be defined in the interpreter itself. This not only limits the
semantics to e.g. not require namespace lookups and also locks the usage of
semantics to e.g. not require namespace lookups but also locks the usage of
typing features to the Python version that is being used, rather than allowing
the currently possible backporting via ``typing_extensions``.

Storing Annotation Source Code
==============================

Instead of storing binary data that defines the annotations' ASTs an alternative
is to simply store the annotations source code directly. This also was presented
Instead of storing binary data that defines the annotations' ASTs, we could
simply store the annotations source code directly. This also was presented
as a possibility all the way back in :pep:`649`. The two approaches are largely
equivalent since one can create the AST from the source code and vice versa.
While the unparsed AST is not necessarily the exact string that occurred in
Expand Down
Loading