Skip to content

Some comments on concurrent.interpreters documentation #136569

Closed
@bluetech

Description

@bluetech

Here are some other pedantic comments on the concurrent.interpreters I had while reading and trying it (it's exciting to have it!). I thought it might be useful to share, but they are all just suggestions for your consideration.


The call_in_thread is "Run the given function in the interpreter (in a new thread)" which doesn't mention it returns the Thread. (The return value is demonstrated in the Basic Usage example however).


The first sentence of the doc is "The concurrent.interpreters module constructs higher-level interfaces on top of the lower level _interpreters module.". The _interpreters module is undocumented, so this seems irrelevant to me. Maybe it makes sense as a side-note somewhere? But I think the first sentence should be a basic description of what the module does, rather than a somewhat obscure implementation detail.


Second sentence: "The module is primarily meant to provide a basic API for ..." I would just write "The module provides an API for ...". I understand why it's written this way, but IMO these reservations don't add much, better to just be direct.

A few more similar cases:

  • "An “interpreter” is effectively the execution context of the Python runtime" -- I would drop the "effectively".

  • "Each interpreter is effectively isolated from the others" -- I would drop the "effectively" -- the Note below is sufficient qualification.

  • "That isolation is primarily useful as a strong separation ..." -- would drop the "primarily" - that it's useful for one thing does not mean that it may not be useful for other things.

  • "This mostly involves starting a new thread, where you switch to another interpreter and run what you want there." - would drop the "mostly".


The concept and functionality of the interpreter have been a part of Python since version 2.2, but the feature was only available through the C-API and not well known, and the isolation was relatively incomplete until version 3.12.

Do these details matter for a reader reading the docs of concurrent.interpreters? I think "Added in version 3.14." is sufficient information, so I'd remove this paragraph, or move it to an "Historical Notes" section.

Similarly, "do not share the GIL [...] (This has been the case since Python 3.12.)" - would drop the parenthetical - the module is only added in Python 3.14, doesn't seem relevant.


Running in a different interpreter involves switching to it in the current thread and then calling some function. The runtime will execute the function using the current interpreter’s state.

The meaning of "current interpreter" is not clear here. I understand it means the switched-to interpreter but one can also interpret "current" as the interpreter doing the "running'.


All that said, interpreters do naturally support certain flavors of concurrency, as a powerful side effect of that isolation. There’s a powerful side effect of that isolation.

The second sentence is repetitious.


It’s a similar concurrency model to CSP or the actor model, a model which is relatively easy to reason about.

I would expand "CSP" to "Communicating Sequential Processes (CSP)".

You can take advantage of that concurrency model in a single thread, switching back and forth between interpreters, Stackless-style.

Make the "Stackless-style" a hyperlink? Readers might not know what it means.


There is a small number of Python types that actually share mutable data between interpreters: memoryview, Queue

While Queue is clear enough, I think memoryview could use more details, like is it thread safe to mutate or does it need synchronization?


(Not documentation related) Does it make sense for Interpreter to be a context manager which calls close() on exit? Apologies if this has been discussed before.


id - The underlying interpreter’s ID.

Maybe a bit more details on this, like type, is it unique across process lifetime, what's it useful for? (E.g. like threading.get_native_id doc).


close() Finalize and destroy the interpreter.

Maybe some details on what happens if closing an interpreter while it's still running?

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.14bugs and security fixes3.15new features, bugs and security fixesdocsDocumentation in the Doc dirstdlibPython modules in the Lib dirtopic-subinterpreters

    Projects

    Status

    Done

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions