Skip to content

gh-126417: validate ABC methods on multiprocessing proxy types #126454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 11, 2024

Conversation

tungol
Copy link
Contributor

@tungol tungol commented Nov 5, 2024

In a similar situation, @JelleZijlstra asked about making sure all the relevant methods existed for contextvars.Context being registered to Mapping: #126451 (comment)

After that, I circled back to do the same for ListProxy and DictProxy. DictProxy already had everything it needed, but ListProxy is missing __iter__. In the original version of this MR I added it, but that caused a test failure in test_list_iter.

When __iter__ is proxied, iter(mylist) returns a list_iterator object. That behaves differently in some situations than the iterator object currently returned; Seems like it doesn't pick up the modification during iteration that test_list_iter checks for. Because of that, I changed this MR to just the test improvement. I think some custom methods would be needed to add __iter__ to ListProxy without changing the behavior.

also validate in the tests that ListProxy has all MutableSequence
methods and DictProxy has all MutableMapping methods.
@tungol tungol marked this pull request as draft November 5, 2024 21:10
@tungol

This comment was marked as outdated.

@tungol tungol changed the title gh-126417: add __iter__ to multiprocessing.managers.ListProxy gh-126417: validate ABC methods on multiprocessing proxy types Nov 5, 2024
@tungol tungol marked this pull request as ready for review November 5, 2024 21:39
@AlexWaygood
Copy link
Member

AlexWaygood commented Nov 5, 2024

I think it's fine that ListProxy doesn't have __iter__ when it comes to the ABC registration. The important thing is really that ListProxy is iterable, and it is; it's not really any less of a MutableSequence because of the fact that it uses the old-style iteration protocol rather than the new one.

Co-authored-by: Alex Waygood <[email protected]>
@tungol
Copy link
Contributor Author

tungol commented Nov 5, 2024

Yeah, I agree. I had thought it would be free to add it, at which point we might as well. But it's not bad to not have it.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems fine to me. It's not ideal to hardcode the method names in the test -- but it also is probably more trouble than it's worth to try to figure them out dynamically in the test; it's obviously good to keep tests as simple as possible. And it seems pretty unlikely that methods will be added or removed from these ABCs anytime soon, so it's not a massive problem to just hardcode the names.

@gpshead gpshead added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes and removed needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes labels Nov 11, 2024
@gpshead gpshead merged commit 6ee542d into python:main Nov 11, 2024
52 checks passed
@miss-islington-app
Copy link

Thanks @tungol for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 11, 2024
…ythonGH-126454)

Checks that appropriate dunder __ methods exist on the dict and list proxy types.

(cherry picked from commit 6ee542d)

Co-authored-by: Stephen Morton <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Nov 11, 2024

GH-126674 is a backport of this pull request to the 3.13 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 11, 2024
…ythonGH-126454)

Checks that appropriate dunder __ methods exist on the dict and list proxy types.

(cherry picked from commit 6ee542d)

Co-authored-by: Stephen Morton <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Nov 11, 2024
@bedevere-app
Copy link

bedevere-app bot commented Nov 11, 2024

GH-126675 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Nov 11, 2024
@tungol tungol deleted the multiprocessing-proxies branch November 11, 2024 08:17
picnixz pushed a commit to picnixz/cpython that referenced this pull request Dec 8, 2024
…ython#126454)

Checks that appropriate dunder __ methods exist on the dict and list proxy types.

Co-authored-by: Alex Waygood <[email protected]>
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
…ython#126454)

Checks that appropriate dunder __ methods exist on the dict and list proxy types.

Co-authored-by: Alex Waygood <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants