From 788eea835ec46384cbe56fa8c1cc8effd62e7fde Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:36:55 +0100 Subject: [PATCH 01/14] Strip thread name --- pep-0727.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index fd220f65f1b..80851c2cd8f 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -2,14 +2,14 @@ PEP: 727 Title: Documentation Metadata in Typing Author: Sebastián Ramírez Sponsor: Jelle Zijlstra -Discussions-To: https://discuss.python.org/t/pep-727-documentation-metadata-in-typing/32566 +Discussions-To: https://discuss.python.org/t/32566 Status: Draft Type: Standards Track Topic: Typing Content-Type: text/x-rst Created: 28-Aug-2023 Python-Version: 3.13 -Post-History: `30-Aug-2023 `__ +Post-History: `30-Aug-2023 `__ Abstract From bc12352855257f4f615acca1c64448287d3d0ac0 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:37:13 +0100 Subject: [PATCH 02/14] Link to Python docs --- pep-0727.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index 80851c2cd8f..2bfa3cde498 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -16,8 +16,8 @@ Abstract ======== This document proposes a way to complement docstrings to add additional documentation -to Python symbols using type annotations with ``Annotated`` (in class attributes, -function and method parameters, return values, and variables). +to Python symbols using type annotations with :py:class`~typing.Annotated` +(in class attributes, function and method parameters, return values, and variables). Motivation From 599733ac007bc1e91f082ff2548ff255a76a860c Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:37:22 +0100 Subject: [PATCH 03/14] Reference PEP 484 --- pep-0727.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0727.rst b/pep-0727.rst index 2bfa3cde498..bb5f4169bc2 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -60,7 +60,7 @@ documentation in some other way (e.g. an API, a CLI, etc). Some of these previous formats tried to account for the lack of type annotations in older Python versions by including typing information in the docstrings, but now that information doesn't need to be in docstrings as there is now an official -syntax for type annotations. +:pep:`syntax for type annotations <484>`. Rationale From 9efc6142d656a205145276b32c0260a7c99acf68 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:38:13 +0100 Subject: [PATCH 04/14] Fix heading levels --- pep-0727.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index bb5f4169bc2..31d04832d21 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -171,7 +171,7 @@ but implementers are not required to support them. Type Alias ----------- +'''''''''' When creating a type alias, like: @@ -206,7 +206,7 @@ could require more complex dereferencing logic. Annotating Type Parameters --------------------------- +'''''''''''''''''''''''''' When annotating type parameters, as in: @@ -225,7 +225,7 @@ conformant, but it's included for completeness. Annotating Unions ------------------ +''''''''''''''''' If used in one of the parameters of a union, as in: @@ -247,7 +247,7 @@ included for completeness. Nested ``Annotated`` --------------------- +'''''''''''''''''''' Continuing with the same idea above, if ``Annotated`` was used nested and used multiple times in the same parameter, ``doc()`` would refer to the type it @@ -281,7 +281,7 @@ of the parameter passed is of one type or another, but they are not required to Duplication ------------ +''''''''''' If ``doc()`` is used multiple times in a single ``Annotated``, it would be considered invalid usage from the developer, for example: From aac18fe8b1868a60120ee77aa6fc36813fbee44a Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:38:30 +0100 Subject: [PATCH 05/14] Add missing sections (as a comment) --- pep-0727.rst | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/pep-0727.rst b/pep-0727.rst index 31d04832d21..57ff3cf313a 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -326,6 +326,26 @@ example would be equivalent to: ) -> None: ... +.. you need to fill these in: + + Backwards Compatibility + ======================= + + [Describe potential impact and severity on pre-existing code.] + + + Security Implications + ===================== + + [How could a malicious user take advantage of this new feature?] + + + How to Teach This + ================= + + [How to teach users, new and experienced, how to apply the PEP to their work.] + + Early Adopters and Older Python Versions ======================================== From 57e8dcc6ff999c5ff82ea07a21a01f9060a80ec5 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:38:52 +0100 Subject: [PATCH 06/14] Rename EAaOPV to Reference Implementation --- pep-0727.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index 57ff3cf313a..36717381490 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -346,8 +346,8 @@ example would be equivalent to: [How to teach users, new and experienced, how to apply the PEP to their work.] -Early Adopters and Older Python Versions -======================================== +Reference Implementation +======================== For older versions of Python and early adopters of this proposal, ``doc()`` and ``DocInfo`` can be imported from the ``typing_extensions`` package. From 4d72db44214b45dd15867b9a9abe97e26bad73df Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:40:52 +0100 Subject: [PATCH 07/14] Code block formatting --- pep-0727.rst | 127 +++++++++++++++++++++++++-------------------------- 1 file changed, 61 insertions(+), 66 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index 36717381490..85844c395bd 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -118,42 +118,42 @@ statically accessible values. An example documenting the attributes of a class, or in this case, the keys of a ``TypedDict``, could look like this: -.. code-block:: +.. code:: python - from typing import Annotated, TypedDict, NotRequired, doc + from typing import Annotated, TypedDict, NotRequired, doc - class User(TypedDict): - firstname: Annotated[str, doc("The user's first name")] - lastname: Annotated[str, doc("The user's last name")] + class User(TypedDict): + firstname: Annotated[str, doc("The user's first name")] + lastname: Annotated[str, doc("The user's last name")] An example documenting the parameters of a function could look like this: -.. code-block:: +.. code:: python - from typing import Annotated, doc + from typing import Annotated, doc - def create_user( - lastname: Annotated[str, doc("The **last name** of the newly created user")], - firstname: Annotated[str | None, doc("The user's **first name**")] = None, - ) -> Annotated[User, doc("The created user after saving in the database")]: - """ - Create a new user in the system, it needs the database connection to be already - initialized. - """ - pass + def create_user( + lastname: Annotated[str, doc("The **last name** of the newly created user")], + firstname: Annotated[str | None, doc("The user's **first name**")] = None, + ) -> Annotated[User, doc("The created user after saving in the database")]: + """Create a new user in the system. + + It needs the database connection to be already initialized. + """ + pass The return of the ``doc()`` function is an instance of a class that can be checked and used at runtime, defined similar to: -.. code-block:: +.. code:: python - class DocInfo: - def __init__(self, documentation: str): - self.documentation = documentation + class DocInfo: + def __init__(self, documentation: str, /): + self.documentation = documentation ...where the attribute ``documentation`` contains the same value string passed to the function ``doc()``. @@ -175,9 +175,9 @@ Type Alias When creating a type alias, like: -.. code-block:: +.. code:: python - Username = Annotated[str, doc("The name of a user in the system")] + Username = Annotated[str, doc("The name of a user in the system")] ...the documentation would be considered to be carried by the parameter annotated @@ -185,20 +185,16 @@ with ``Username``. So, in a function like: -.. code-block:: +.. code:: python - def hi( - to: Username, - ) -> None: ... + def hi(to: Username) -> None: ... ...it would be equivalent to: -.. code-block:: +.. code:: python - def hi( - to: Annotated[str, doc("The name of a user in the system")], - ) -> None: ... + def hi(to: Annotated[str, doc("The name of a user in the system")]) -> None: ... Nevertheless, implementers would not be required to support type aliases outside of the final type annotation to be conformant with this specification, as it @@ -210,11 +206,11 @@ Annotating Type Parameters When annotating type parameters, as in: -.. code-block:: +.. code:: python - def hi( - to: list[Annotated[str, doc("The name of a user in a list")]], - ) -> None: ... + def hi( + to: list[Annotated[str, doc("The name of a user in a list")]], + ) -> None: ... ...the documentation in ``doc()`` would refer to what it is annotating, in this case, each item in the list, not the list itself. @@ -229,11 +225,11 @@ Annotating Unions If used in one of the parameters of a union, as in: -.. code-block:: +.. code:: python - def hi( - to: str | Annotated[list[str], doc("List of user names")], - ) -> None: ... + def hi( + to: str | Annotated[list[str], doc("List of user names")], + ) -> None: ... ...again, the documentation in ``doc()`` would refer to what it is annotating, in this case, this documents the list itself, not its items. @@ -255,14 +251,15 @@ is annotating. So, in an example like: -.. code-block:: +.. code:: python - def hi( - to: Annotated[ - Annotated[str, doc("A user name")] | Annotated[list, doc("A list of user names")], - doc("Who to say hi to"), - ], - ) -> None: ... + def hi( + to: Annotated[ + Annotated[str, doc("A user name")] + | Annotated[list, doc("A list of user names")], + doc("Who to say hi to"), + ], + ) -> None: ... The documentation for the whole parameter ``to`` would be considered to be @@ -286,11 +283,11 @@ Duplication If ``doc()`` is used multiple times in a single ``Annotated``, it would be considered invalid usage from the developer, for example: -.. code-block:: +.. code:: python - def hi( - to: Annotated[str, doc("A user name"), doc("The current user name")], - ) -> None: ... + def hi( + to: Annotated[str, doc("A user name"), doc("The current user name")], + ) -> None: ... Implementers can consider this invalid and are not required to support this to be @@ -302,28 +299,28 @@ can opt to support one of the ``doc()`` declarations. In that case, the suggestion would be to support the last one, just because this would support overriding, for example, in: -.. code-block:: +.. code:: python - User = Annotated[str, doc("A user name")] + User = Annotated[str, doc("A user name")] - CurrentUser = Annotated[User, doc("The current user name")] + CurrentUser = Annotated[User, doc("The current user name")] Internally, in Python, ``CurrentUser`` here is equivalent to: -.. code-block:: +.. code:: python - CurrentUser = Annotated[str, doc("A user name"), doc("The current user name")] + CurrentUser = Annotated[str, + doc("A user name"), + doc("The current user name")] For an implementation that supports the last ``doc()`` appearance, the above example would be equivalent to: -.. code-block:: +.. code:: python - def hi( - to: Annotated[str, doc("The current user name")], - ) -> None: ... + def hi(to: Annotated[str, doc("The current user name")]) -> None: ... .. you need to fill these in: @@ -352,16 +349,14 @@ Reference Implementation For older versions of Python and early adopters of this proposal, ``doc()`` and ``DocInfo`` can be imported from the ``typing_extensions`` package. -.. code-block:: +.. code:: python - from typing import Annotated + from typing import Annotated - from typing_extensions import doc + from typing_extensions import doc - def hi( - to: Annotated[str, doc("The current user name")], - ) -> None: ... + def hi(to: Annotated[str, doc("The current user name")]) -> None: ... Rejected Ideas @@ -427,8 +422,8 @@ to be used by those that are willing to take the extra verbosity in exchange for the benefits. -Doc is not Typing ------------------ +Documentation is not Typing +--------------------------- It could also be argued that documentation is not really part of typing, or that it should live in a different module. Or that this information should not be part From 75d66e0f896c613f533bd25f815904a82d9ff3ad Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:41:44 +0100 Subject: [PATCH 08/14] Remove leading elipses --- pep-0727.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index 85844c395bd..f615145ecdf 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -155,7 +155,7 @@ and used at runtime, defined similar to: def __init__(self, documentation: str, /): self.documentation = documentation -...where the attribute ``documentation`` contains the same value string passed to +Where the attribute ``documentation`` contains the same value string passed to the function ``doc()``. @@ -180,7 +180,7 @@ When creating a type alias, like: Username = Annotated[str, doc("The name of a user in the system")] -...the documentation would be considered to be carried by the parameter annotated +The documentation would be considered to be carried by the parameter annotated with ``Username``. So, in a function like: @@ -190,7 +190,7 @@ So, in a function like: def hi(to: Username) -> None: ... -...it would be equivalent to: +It would be equivalent to: .. code:: python @@ -212,7 +212,7 @@ When annotating type parameters, as in: to: list[Annotated[str, doc("The name of a user in a list")]], ) -> None: ... -...the documentation in ``doc()`` would refer to what it is annotating, in this +The documentation in ``doc()`` would refer to what it is annotating, in this case, each item in the list, not the list itself. There are currently no practical use cases for documenting type parameters, @@ -231,7 +231,7 @@ If used in one of the parameters of a union, as in: to: str | Annotated[list[str], doc("List of user names")], ) -> None: ... -...again, the documentation in ``doc()`` would refer to what it is annotating, +Again, the documentation in ``doc()`` would refer to what it is annotating, in this case, this documents the list itself, not its items. In particular, the documentation would not refer to a single string passed as a From 014a316380bcfeeabae37bb7f61a2bf18d265d7a Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:43:21 +0100 Subject: [PATCH 09/14] Tighten usage criteria --- pep-0727.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index f615145ecdf..14536a0071f 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -89,8 +89,7 @@ Specification -------------- The main proposal is to have a new function ``doc()`` in the ``typing`` module. -Even though this is not strictly related to the type annotations, it's expected -to go in ``Annotated`` type annotations, and to interact with type annotations. +This function MUST only be used within :py:class:`~typing.Annotated` annotations. There's also the particular benefit that it could be implemented in the ``typing_extensions`` package to have support for older versions of Python and From 492a73dce0b8f578e72c7e6b5a559e9e82789b59 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:43:35 +0100 Subject: [PATCH 10/14] Remove non-specification content --- pep-0727.rst | 4 ---- 1 file changed, 4 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index 14536a0071f..3a69fbddc21 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -91,10 +91,6 @@ Specification The main proposal is to have a new function ``doc()`` in the ``typing`` module. This function MUST only be used within :py:class:`~typing.Annotated` annotations. -There's also the particular benefit that it could be implemented in the -``typing_extensions`` package to have support for older versions of Python and -early adopters of this proposal. - This ``doc()`` function would receive one single parameter ``documentation`` with a documentation string. From ee92cae0faf590f36e336fb1493016dc70aa7103 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:50:53 +0100 Subject: [PATCH 11/14] Rewrite the Specification --- pep-0727.rst | 36 ++++++++++++++++-------------------- 1 file changed, 16 insertions(+), 20 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index 3a69fbddc21..040c6786732 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -84,31 +84,27 @@ like to adopt it. Specification ============= - -``typing.doc`` --------------- - -The main proposal is to have a new function ``doc()`` in the ``typing`` module. +The main proposal is to introduce a new function, ``typing.doc()``, +to be used when documenting Python objects. This function MUST only be used within :py:class:`~typing.Annotated` annotations. +The function takes a single string argument, ``documentation``, +and returns an instance of ``typing.DocInfo``, +which stores the input string unchanged. -This ``doc()`` function would receive one single parameter ``documentation`` with -a documentation string. +Any tool processing ``typing.DocInfo`` objects SHOULD interpret the string as +a docstring, and therefore SHOULD normalize whitespace +as if ``inspect.cleandoc()`` were used. -This string could be a multi-line string, in which case, when extracted by tools, -should be interpreted cleaning up indentation as if using ``inspect.cleandoc()``, -the same procedure used for docstrings. +The string passed to ``typing.doc()`` SHOULD be of the form that would be a valid docstring. +This means that `f-strings`__ and string operations SHOULD NOT be used. +As this cannot be enforced by the Python runtime, +tools SHOULD NOT rely on this behaviour, +and SHOULD exit with an error if such a prohibited string is encountered. -This string could probably contain markup, like Markdown or reST. As that could -be highly debated, that decision is left for a future proposal, to focus here -on the main functionality. +__ https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals -This specification targets static analysis tools and editors, and as such, the -value passed to ``doc()`` should allow static evaluation and analysis. If a -developer passes as the value something that requires runtime execution -(e.g. a function call) the behavior of static analysis tools is unspecified -and they could omit it from their process and results. For static analysis -tools to be conformant with this specification they need only to support -statically accessible values. +Examples +-------- An example documenting the attributes of a class, or in this case, the keys of a ``TypedDict``, could look like this: From b45c5ac03e4040d93df89e2316f808fc414882b2 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:55:19 +0100 Subject: [PATCH 12/14] Simplify Examples and move DocInfo to the reference implementation --- pep-0727.rst | 46 ++++++++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 24 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index 040c6786732..d4459a05537 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -106,29 +106,28 @@ __ https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-li Examples -------- -An example documenting the attributes of a class, or in this case, the keys -of a ``TypedDict``, could look like this: +Class attributes may be documented: .. code:: python - from typing import Annotated, TypedDict, NotRequired, doc - + from typing import Annotated, doc - class User(TypedDict): - firstname: Annotated[str, doc("The user's first name")] - lastname: Annotated[str, doc("The user's last name")] + class User: + first_name: Annotated[str, doc("The user's first name")] + last_name: Annotated[str, doc("The user's last name")] + ... -An example documenting the parameters of a function could look like this: +As can function or method parameters: .. code:: python from typing import Annotated, doc - def create_user( - lastname: Annotated[str, doc("The **last name** of the newly created user")], - firstname: Annotated[str | None, doc("The user's **first name**")] = None, + first_name: Annotated[str, doc("The user's first name")], + last_name: Annotated[str, doc("The user's last name")], + cursor: DatabaseConnection | None = None, ) -> Annotated[User, doc("The created user after saving in the database")]: """Create a new user in the system. @@ -137,19 +136,6 @@ An example documenting the parameters of a function could look like this: pass -The return of the ``doc()`` function is an instance of a class that can be checked -and used at runtime, defined similar to: - -.. code:: python - - class DocInfo: - def __init__(self, documentation: str, /): - self.documentation = documentation - -Where the attribute ``documentation`` contains the same value string passed to -the function ``doc()``. - - Additional Scenarios -------------------- @@ -349,6 +335,18 @@ For older versions of Python and early adopters of this proposal, ``doc()`` and def hi(to: Annotated[str, doc("The current user name")]) -> None: ... +The return of the ``doc()`` function is an instance of a class that can be checked +and used at runtime, defined similar to: + +.. code:: python + + class DocInfo: + def __init__(self, documentation: str, /): + self.documentation = documentation + +Where the attribute ``documentation`` contains the same value string passed to +the function ``doc()``. + Rejected Ideas ============== From 69e1baafea0b2a76ef074cc04c03fa08545f3cc0 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 01:58:47 +0100 Subject: [PATCH 13/14] Improve the Reference Implementation --- pep-0727.rst | 22 +++++++--------------- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/pep-0727.rst b/pep-0727.rst index d4459a05537..f8968d14459 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -323,29 +323,21 @@ example would be equivalent to: Reference Implementation ======================== -For older versions of Python and early adopters of this proposal, ``doc()`` and -``DocInfo`` can be imported from the ``typing_extensions`` package. +``typing.doc`` and ``typing.DocInfo`` are implemented as follows: .. code:: python - from typing import Annotated - - from typing_extensions import doc - - - def hi(to: Annotated[str, doc("The current user name")]) -> None: ... - -The return of the ``doc()`` function is an instance of a class that can be checked -and used at runtime, defined similar to: - -.. code:: python + def doc(documentation: str, /) -> DocInfo: + return DocInfo(documentation) class DocInfo: def __init__(self, documentation: str, /): self.documentation = documentation -Where the attribute ``documentation`` contains the same value string passed to -the function ``doc()``. + +These have been implemented in the `typing_extensions`__ package. + +__ https://pypi.org/project/typing-extensions/ Rejected Ideas From 5905f98686181a1c4493d24432b05c780f97d8fc Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Thu, 31 Aug 2023 02:03:53 +0100 Subject: [PATCH 14/14] Syntax --- pep-0727.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0727.rst b/pep-0727.rst index f8968d14459..c7ae517bc08 100644 --- a/pep-0727.rst +++ b/pep-0727.rst @@ -16,7 +16,7 @@ Abstract ======== This document proposes a way to complement docstrings to add additional documentation -to Python symbols using type annotations with :py:class`~typing.Annotated` +to Python symbols using type annotations with :py:class:`~typing.Annotated` (in class attributes, function and method parameters, return values, and variables).