Add a page to the documentation listing stdlib modules with a command-line interface (CLI)

[vstinner]

While reviewing PR #109152, I discover that the sqlite3 module has a command line interface. Maybe I knew and then I forgot.

Is there a a documentation page listing all modules with a command-line interface (CLI)? I don't know so.

By searching for main( pattern and for __main__.py files, so far, I found:

• ast: https://docs.python.org/dev/library/ast.html#command-line-usage
• asyncio: https://docs.python.org/dev/library/asyncio.html just one sentence, "You can experiment with an asyncio concurrent context in the REPL".
• base64: shhh, it's a secret, it's not documented!
• calendar: https://docs.python.org/dev/library/calendar.html#command-line-usage
• compileall: https://docs.python.org/dev/library/compileall.html#command-line-use
• ensurepip: https://docs.python.org/dev/library/ensurepip.html#command-line-interface
• gzip: https://docs.python.org/dev/library/gzip.html#command-line-interface
• idlelib: not sure if it's the same as: https://docs.python.org/dev/library/idle.html#command-line-usage
• json.tool: https://docs.python.org/dev/library/json.html#module-json.tool
• pdb: https://docs.python.org/3/library/pdb.html
• profile: https://docs.python.org/dev/library/profile.html#instant-user-s-manual
• py_compile: https://docs.python.org/dev/library/py_compile.html#command-line-interface
• quopri: (missing)
• site: https://docs.python.org/dev/library/site.html#command-line-interface
• sqlite3 https://docs.python.org/3/library/sqlite3.html#command-line-interface
• tabnanny: https://docs.python.org/dev/library/tabnanny.html
• tarfile: https://docs.python.org/dev/library/tarfile.html#command-line-interface
• timeit: https://docs.python.org/dev/library/timeit.html#command-line-interface
• tokenize: https://docs.python.org/dev/library/tokenize.html#command-line-usage
• trace: https://docs.python.org/dev/library/trace.html#command-line-usage
• turtledemo: https://docs.python.org/3/library/turtle.html#module-turtledemo
• uuid: https://docs.python.org/dev/library/uuid.html#command-line-example
• unittest: https://docs.python.org/dev/library/unittest.html#command-line-interface
• venv: https://docs.python.org/dev/library/venv.html#creating-virtual-environments
• webbrowser: https://docs.python.org/3.14/library/webbrowser.html
• zipapp: https://docs.python.org/dev/library/zipapp.html#command-line-interface
• zipfile: https://docs.python.org/dev/library/zipfile.html#command-line-interface

Some modules include self-tests in their main() function. I dislike that and I would suggest to remove it and at least move it to their test suite.

• ctypes.util: self-test
• ctypes.textpad: demo
• curses.has_key: self-test?
• dbm.__init__: get the DB type... but this CLI is not easy discoverable, it's in the init module!
• getopt: self-test
• heapq: self-test
• shlex: print tokens. Is it really useful?
• imaplib: self-test, "test the IMAP4_stream class"
• modulefinder: self-test
• netrc: just parse ~/.netrc if available, and then display nothing... not very useful
• random: benchmark
• smtplib: "Test the sendmail method", try to send an email to localhost SMTP server.
• symtable: self-test. parse itself (symtable.py)
• turtle: it's a demo. Looks more like an easter egg or advanced example than "an useful CLI".
• textwrap: self-test
• tkinter: self-test
• wsgiref.simple_server: self-test, run a server, open itself in a browser, stop the server.
• xml.sax.expatreader: Sheakspear easter egg.
• xml.sax.xmlreader: self-test.
• xmlrpc.client + xmlrpc.server: simple demo.

Linked PRs

• gh-109435: Add Doc/library/cmdline.rst #109436

[hugovk]

There's not a page listing them, and I was also thinking it would be good to have an index.

@simonw found 90 and wrote up some highlights on his blog recently: https://til.simonwillison.net/python/stdlib-cli-tools

@AA-Turner Is there some existing Sphinx automation we could use for this? Mark the CLI sections and have them all listed on a single page, with some custom intro text?

For example, we have a Superseded Modules page, but it's a manually maintained page.

[serhiy-storchaka]

You missed this. 😉 🥚

Not all modules with the CLI have main() function. See for example platform.

[vstinner]

You missed this. 😉 🥚
Not all modules with the CLI have main() function. See for example platform.

I completed the list in my PR.

I also completed the list of modules that I don't want to document in: #109435 (comment)

IMO documenting this module CLI is worth it, but I'm open to not document it :-)

[vstinner]

Since https://til.simonwillison.net/python/stdlib-cli-tools was written, PEP 594 removed many modules.

[AA-Turner]

See also the following for related discussion:

• Setup & Usage documentation for selected stdlib modules #59150
• Document all "python -m" utilities #73158
• Decide the fate of undocumented script behavior of some modules #93096
• gh-59150: Add table of runnable modules #93410

A

[AA-Turner]

Note on #109436, this was previously rejected -- we should instead aim to document command-line behaviour in appropriate places.

A

[vstinner]

Ah, I was not the first one looking for such list :-)

Decide the fate of undocumented script behavior of some modules

I propose to remove them.

[hugovk]

I think it's useful because https://docs.python.org/3/library/index.html is "here's a list of Python APIs you can use programmatically", and this page is "here's a list of Python APIs you can use via the command-line".

The discussion about a re CLI shows how useful these can be for cross-platform use, for example it opens up some tools that aren't always easily accessible on Windows.

[serhiy-storchaka]

For symtable see

• command line interface of symtable module is broken #85098

[vstinner]

#93410 proposed to display the list as a table, the idea is interesting. But I'm not interested to propose a change for that. If someone wants to enhance the list, please go ahead!

[merwok]

Could this be re-opened? This discussion was relatively short, not on python-dev and did not seem to review previous on the same topic.

Some specific comments of interest:

• Document all "python -m" utilities #73158 (comment)
• smtpd-as-a-script feature should be documented and should use argparse #55469 (comment)
• gh-59150: Add table of runnable modules #93410 (comment)

I still don’t see the use of a long list, or long table, of all the runnable modules. They don’t all have the same usefulness and audience, or the same maintained/documented status.

[vstinner]

Feel free to reopen the issue and/or start a new discussion if you want.

[vstinner]

@merwok: You reopened the issue but then did nothing. If you disagree with the added documentation, you can just remove it.

[merwok]

@hugovk @AA-Turner Would you object to a revert? (see comment from October 15 and the links there for justification)

[hugovk]

I actually found it useful this week, as a quick reference to check if a certain module could be used as a CLI (secrets, it can't; I'd been thinking of uuid).

I'd rather see it (incrementally) updated to make it more useful. IIRC there was a suggestion to group them somehow?

[vstinner]

What's the status of https://docs.python.org/dev/library/cmdline.html ?

[hugovk]

I still think it's useful: I told someone about it last month who said they wished they'd known about it before. There have been suggestions to show it in a table, or group them somehow. But if you want to close this issue, I think they could be new issues or PRs.

[pradyunsg]

I agree that it's useful, that we should improve it incrementally, and that this issue can be closed.

The current state where we have just a list of module references is better than nothing, and we can improve it further over time with new issues/PRs.

[vstinner]

I agree that it's useful, that we should improve it incrementally, and that this issue can be closed.

Ok, I close the issue again.