gh-133363: Fix Cmd completion for lines beginning with `!` #133364

godlygeek · 2025-05-04T01:56:40Z

When a line begins with ! and there's no do_shell method defined, parseline returns None as the cmd, which incorrectly leads to None being concatenated to complete_ and triggering a TypeError.

Instead, recognize None as a sentinel that means we should call completedefault, as an empty string already is.

Issue: cmd.Cmd fails tab completion for lines beginning with ! (bang space) #133363

When a line begins with `!` and there's no `do_shell` method defined, `parsecmd` returns `None` as the `cmd`, which incorrectly leads to `None` being concatenated to `complete_` and triggering a `TypeError`. Instead, recognize `None` as a sentinel that means we should call `completedefault`, as an empty string already is.

Lib/cmd.py

gaogaotiantian · 2025-05-04T02:00:03Z

Lib/test/test_cmd.py

+
+        output = run_pty(script, input)
+
+        self.assertIn(b'ello', output)


ello because that's what is completed? hello is not part of the output?

Oh, because that's what is used in the previous test.

ello because that's what's completed, yes. hello is part of the output, but I'm just matching the test above that I copy-pasted from, which only checks for ab_completion_test, heh

The tests do both pass if I adjust them to check for hello and tab_completion_test respectively.

I just realized I co-authored that test. Ha.

gaogaotiantian · 2025-05-04T02:03:56Z

Misc/NEWS.d/next/Library/2025-05-03-21-55-33.gh-issue-133363.PTLnRP.rst

+The :class:`cmd.Cmd` class has been fixed to call the ``completedefault``
+method whenever the ``do_shell`` method is not defined and tab completion is
+requested for a line beginning with ``!``. Previously ``completedefault``
+was called only if there were no spaces between the ``!`` and the cursor


Previously completedefault was called only if there were no spaces between the ! and the cursor position where tab completion was attempted.

Is that even true? I thought the completion never works:

elif line[0] == '!': if hasattr(self, 'do_shell'): line = 'shell ' + line[1:] else: return None, None, line

Anyway, normally in the news we only need to say what is fixed, we don't need to do a complete description of what's the previous behavior. News entry should be a concise sentence.

Is that even true? I thought the completion never works:

Yes, because parsecmd is only called if begidx > 0, so in the !foo<Tab> case (begidx=0 endidx=3) we don't hit this buggy codepath.

Anyway, normally in the news we only need to say what is fixed, we don't need to do a complete description of what's the previous behavior. News entry should be a concise sentence.

OK, I've shortened it and just say that it's fixed to reliably call the method, since previously it was inconsistent.

Okay but it's still not good, because that's not really a name. It still won't give us the correct completion.

Let's not worry about that, we can do that in pdb I believe. I don't want to change cmd so close to beta freeze. I'll loop back to this after beta freeze.

Okay but it's still not good, because that's not really a name. It still won't give us the correct completion.

True, though that seems to need a bigger change if you want to fix that... Do you want to special case ! at the start of the line, and do:

if begidx>0: ... elif line.startswith("!"): compfunc = self.completedefault else: compfunc = self.completenames

Even that's not enough to make PDB's completion work without the space, though, because that'll give a text="!h" and PDB will say that none of the object names it knows of start with "!". Which means that making that work requires changes in the PDB module, too.

I was wrong. ! was by default a delimiter and that's why your current code passes. cmd is handling this correctly because begidx will >0 when you complete !h<tab>. pdb somehow removed ! from delimiters and I did not figure out why. However, we can deal with in pdb only, this is not a cmd issue.

gaogaotiantian · 2025-05-04T02:06:13Z

Lib/test/test_cmd.py

+        """)
+
+        # '! h' and complete 'ello' to '! hello'
+        input = b"! h\t\n"


We should do add check here for !h\t\n (without space) as well. No need for another method, just another input.

That does need to be another method, because the assertions are:

self.assertIn(b'hello', output) self.assertIn(b'tab completion success', output)

and those would pass if either of the two inputs succeeded, not only if both of them did.

Or maybe we could use a subtest for this? I know how to avoid duplication with pytest, but I'm not really familiar with unittest. Does this work?

# '! h' or '!h' and complete 'ello' to 'hello' for input in [b"! h\t\n", b"!h\t\n"]: with self.subTest(input=input): output = run_pty(script, input) self.assertIn(b'hello', output) self.assertIn(b'tab completion success', output)

Seems to...

I mean just use the same script and do two run_pty - I don't think you even need a subtest. It's a very small case.

bedevere-bot · 2025-05-04T04:19:07Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL8 3.x (tier-2) has failed when building commit 1d9406e.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/559/builds/5937) and take a look at the build logs.
Check if the failure is related to this commit (1d9406e) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/559/builds/5937

Failed tests:

test.test_multiprocessing_spawn.test_processes

Failed subtests:

test_interrupt - test.test_multiprocessing_spawn.test_processes.WithProcessesTestProcess.test_interrupt

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=8)
                                                  ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/multiprocessing/spawn.py", line 122, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/multiprocessing/spawn.py", line 132, in _main
    self = reduction.pickle.load(from_parent)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_multiprocessing_spawn/test_processes.py", line 2, in <module>
    from test._test_multiprocessing import install_tests_in_module_dict
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/_test_multiprocessing.py", line 53, in <module>
    import multiprocessing.managers
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/multiprocessing/managers.py", line 1182, in <module>
    _BaseDictProxy = MakeProxyType('_BaseDictProxy', (
        '__contains__', '__delitem__', '__getitem__', '__ior__', '__iter__',
    ...<2 lines>...
        'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'
        ))
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/multiprocessing/managers.py", line 978, in MakeProxyType
    exec('''def %s(self, /, *args, **kwds):
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    return self._callmethod(%r, args, kwds)''' % (meth, meth), dic)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 0, in <module>
    
    
    from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=8)
    
KeyboardInterrupt
FAIL


Traceback (most recent call last):
  File "<string>", line 1, in <module>
    from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=5, pipe_handle=9)
                                                  ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/multiprocessing/spawn.py", line 122, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/multiprocessing/spawn.py", line 132, in _main
    self = reduction.pickle.load(from_parent)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_multiprocessing_spawn/test_processes.py", line 2, in <module>
    from test._test_multiprocessing import install_tests_in_module_dict
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/_test_multiprocessing.py", line 6, in <module>
    import unittest.mock
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/unittest/mock.py", line 27, in <module>
    import asyncio
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/__init__.py", line 8, in <module>
    from .base_events import *
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/base_events.py", line 45, in <module>
    from . import staggered
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/staggered.py", line 10, in <module>
    from . import tasks
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/tasks.py", line 29, in <module>
    from . import timeouts
  File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1342, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 938, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 758, in exec_module
  File "<frozen importlib._bootstrap_external>", line 891, in get_code
  File "<frozen importlib._bootstrap_external>", line 514, in _compile_bytecode
KeyboardInterrupt
FAIL


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/_test_multiprocessing.py", line 578, in test_interrupt
    self.assertEqual(exitcode, 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
AssertionError: -2 != 1

* origin/main: (111 commits) pythongh-91048: Add filename and line number to external inspection routines (pythonGH-133385) pythongh-131178: Add tests for `ast` command-line interface (python#133329) Regenerate pcbuild.sln in Visual Studio 2022 (python#133394) pythongh-133042: disable HACL* HMAC on Emscripten (python#133064) pythongh-133351: Fix remote PDB's multi-line block tab completion (python#133387) pythongh-109700: Improve stress tests for interpreter creation (pythonGH-109946) pythongh-81793: Skip tests for os.link() to symlink on Android (pythonGH-133388) pythongh-126835: Rename `ast_opt.c` to `ast_preprocess.c` and related stuff after moving const folding to the peephole optimizier (python#131830) pythongh-91048: Relax test_async_global_awaited_by to fix flakyness (python#133368) pythongh-132457: make staticmethod and classmethod generic (python#132460) pythongh-132805: annotationlib: Fix handling of non-constant values in FORWARDREF (python#132812) pythongh-132426: Add get_annotate_from_class_namespace replacing get_annotate_function (python#132490) pythongh-81793: Always call linkat() from os.link(), if available (pythonGH-132517) pythongh-122559: Synchronize C and Python implementation of the io module about pickling (pythonGH-122628) pythongh-69605: Add PyREPL import autocomplete feature to 'What's New' (python#133358) bpo-44172: Keep reference to original window in curses subwindow objects (pythonGH-26226) pythonGH-133231: Changes to executor management to support proposed `sys._jit` module (pythonGH-133287) pythongh-133363: Fix Cmd completion for lines beginning with `! ` (python#133364) pythongh-132983: Introduce `_zstd` bindings module (pythonGH-133027) pythonGH-91048: Add utils for printing the call stack for asyncio tasks (python#133284) ...

bedevere-app bot added the awaiting review label May 4, 2025

bedevere-app bot mentioned this pull request May 4, 2025

cmd.Cmd fails tab completion for lines beginning with ! (bang space) #133363

Closed

godlygeek mentioned this pull request May 4, 2025

gh-133351: Fix Cmd completion for lines beginning with ! #133362

Closed

gaogaotiantian reviewed May 4, 2025

View reviewed changes

Lib/cmd.py Outdated Show resolved Hide resolved

gaogaotiantian reviewed May 4, 2025

View reviewed changes

Check falsiness instead

973bef6

gaogaotiantian reviewed May 4, 2025

View reviewed changes

godlygeek added 2 commits May 3, 2025 22:07

Make news entry less descriptive

1c6248e

Test with and without a space after !

c259bef

gaogaotiantian approved these changes May 4, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting review labels May 4, 2025

gaogaotiantian self-requested a review May 4, 2025 02:27

godlygeek mentioned this pull request May 4, 2025

completion for remote pdb is not ideal #133351

Closed

gaogaotiantian approved these changes May 4, 2025

View reviewed changes

gaogaotiantian merged commit 1d9406e into python:main May 4, 2025
43 checks passed

bedevere-app bot removed the awaiting merge label May 4, 2025

godlygeek deleted the fix_cmd_bang_completion branch May 4, 2025 03:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-133363: Fix Cmd completion for lines beginning with `!` #133364

gh-133363: Fix Cmd completion for lines beginning with `!` #133364

godlygeek commented May 4, 2025 •

edited

Loading

gaogaotiantian May 4, 2025

gaogaotiantian May 4, 2025

godlygeek May 4, 2025 •

edited

Loading

gaogaotiantian May 4, 2025

gaogaotiantian May 4, 2025 •

edited

Loading

godlygeek May 4, 2025 •

edited

Loading

gaogaotiantian May 4, 2025

gaogaotiantian May 4, 2025

godlygeek May 4, 2025 •

edited

Loading

gaogaotiantian May 4, 2025

gaogaotiantian May 4, 2025

godlygeek May 4, 2025

godlygeek May 4, 2025

gaogaotiantian May 4, 2025

bedevere-bot commented May 4, 2025


		output = run_pty(script, input)

		self.assertIn(b'ello', output)

gh-133363: Fix Cmd completion for lines beginning with ! #133364

gh-133363: Fix Cmd completion for lines beginning with ! #133364

Conversation

godlygeek commented May 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

godlygeek May 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gaogaotiantian May 4, 2025 • edited Loading

Choose a reason for hiding this comment

godlygeek May 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

godlygeek May 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bedevere-bot commented May 4, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

gh-133363: Fix Cmd completion for lines beginning with `!` #133364

gh-133363: Fix Cmd completion for lines beginning with `!` #133364

godlygeek commented May 4, 2025 •

edited

Loading

godlygeek May 4, 2025 •

edited

Loading

gaogaotiantian May 4, 2025 •

edited

Loading

godlygeek May 4, 2025 •

edited

Loading

godlygeek May 4, 2025 •

edited

Loading