-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Combine sys_platform
and platform_system
in marker algebra
#9344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
cfc5550
to
1c03cb9
Compare
Yes. I was just about to comment on #9275 about this. For example, |
Yeah. I think it's a bit more challenging because there isn't a 1-to-1 mapping between |
(Not impossible but different than what I've encoded here.) |
Since |
51ecdd7
to
0340fb3
Compare
4b8f5c7
to
d51b0b5
Compare
On reflection, what I have here is not correct. It fails this test (i.e., it thinks these are disjoint): assert!(!is_disjoint(
"sys_platform == 'cygwin'",
"platform_system == 'Java'"
)); |
Somewhat losing confidence here because it turns out Android also returns |
## Summary This PR introduces a set of parallel structs to `MarkerValueString`, `MarkerValueExtra`, and `MarkerValueVersion` that remove various pieces of information and representations that shouldn't be available in the marker algebra. To start, I've _just_ removed the invalid extra component from `MarkerValueExtra` -- there are no other changes to the representation. So, throughout the marker algebra, we can't represent and thus don't have to worry about clauses with invalid extras. The subsequent changes I plan to make are: 1. Removing `python_version`, since we exclusively use `python_version_full` in the algebra. 2. Removing the deprecated aliases, such that we re-map to the correct marker value. 3. Consolidating `sys_platform` and `platform_release` (the original motivation).
0340fb3
to
ae3be00
Compare
d51b0b5
to
b018724
Compare
It seems like Poetry has gone though quite the rabbit hole for this themselves: python-poetry/poetry#5192. But even after quite some pull requests, they closed the isue with this comment:
It appears this may be an unresolvable issue, though perhaps it's feasible to just cull known impossible environments? I.e. we know that |
I think it's absolutely possible to solve in our marker system, I just don't have the right solution here. |
This is a tough one, my intuition is to encode know mismatch rather than known pairs. There's a tendency to subdivide platforms to accommodate new developments (the development of rust target triples is a good reference since it's more upstreamed than the python build vars), so i would encode tier 1 and tier 2 pairs that never match, such as darwin |
@konstin that's a good point and probably catches the majority of the standard cases with a smaller chance if breaking future cases |
Yes that's also what I want to do. Here's a new idea: we encode all mutually exclusive pairs, and for each pair (A, B), we add |
ae3be00
to
2a90f06
Compare
Closing in favor of #9444. |
## Summary This is an alternative to #9344. If accepted, I need to audit the codebase and call sites to apply it everywhere, but the basic idea is: rather than encoding mutually-incompatible pairs of markers in the representation itself, we have an additional method on `MarkerTree` that expands the false-y definition to take into account assumptions about which markers can be true alongside others. We then check if the the current marker implies that at least one of them is true. So, for example, we know that `sys_platform == 'win32'` and `platform_system == 'Darwin'` are mutually exclusive. When given a marker expression like `python_version >= '3.7'`, we test if `python_version >= '3.7'` and `sys_platform != 'win32' or platform_system != 'Darwin'` are disjoint, i.e., if the following can't be satisfied: ``` python_version >= '3.7' and (sys_platform != 'win32' or platform_system != 'Darwin') ``` Since, if this can't be satisfied, it implies that the left-hand expression requires `sys_platform == 'win32'` and `platform_system == 'Darwin'` to be true at the same time. I think the main downsides here are: 1. We can't _simplify_ markers based on these implications. So we'd still write markers like `sys_platform == 'win32' and platform_system != 'Darwin'`, even though we know the latter expression is redundant. 2. It might be expensive? I'm not sure. I don't think we test for falseness _that_ often though. Closes #7760. Closes #9275.
## Summary A revival of an old idea (#9344) that I have slightly more confidence in now. I abandoned this idea because (1) it couldn't capture that, e.g., `platform_system == 'Windows' and sys_platform == 'foo'` (or some other unknown value) are disjoint, and (2) I thought that Android returned `"android"` for one of `sys_platform` or `platform_system`, which would've made this logic incorrect. However, it looks like Android... doesn't do that? And the values here are almost always in a small, known set. So in the end, the tradeoffs here actually seem pretty good. Vis-a-vis our current solution, this can (e.g.) _simplify out_ expressions like `sys_platform == 'win32' or platform_system == 'Windows'`.
Summary
This PR adds a representation to the marker algebra to capture "known pairs" of
sys_platform
andplatform_system
values.For example, we want to be able to detect that
sys_platform == 'win32'
andplatform_system == 'Darwin'
are disjoint.There is some risk here, in that if there are platforms for which, e.g.,
sys_platform == 'win32
butplatform_system
is notWindows
, we'll get things wrong. I'm trying to weigh whether that's a real risk.In theory, we could maybe also include
os.name
here?Closes #7760.
Closes #9275.