[ty] Infer nonlocal types as unions of all reachable bindings #18750
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sharkdp
changed the title
|
sharkdp
force-pushed
the
david/public-types
branch
from
a0e9746 to
70742b3
Compare
sharkdp
added a commit
that referenced
this pull request
Jun 23, 2025
## Summary As far as I can tell, the two existing tests did the exact same thing. Remove the redundant test, and add tests for all combinations of declared/not-declared and local/"public" use of the name. Proposing this as a separate PR before the behavior might change via #18750
sharkdp
force-pushed
the
david/public-types
branch
from
70742b3 to
1af17e0
Compare
CodSpeed WallTime Performance ReportMerging #18750 will not alter performanceComparing Summary
|
This comment was marked as off-topic.
This comment was marked as off-topic.
sharkdp
force-pushed
the
david/public-types
branch
from
c173689 to
403108d
Compare
sharkdp
commented
Jun 24, 2025
sharkdp
commented
Jun 24, 2025
sharkdp
force-pushed
the
david/public-types
branch
from
00fda72 to
78d8b69
Compare
sharkdp
commented
Jun 25, 2025
| @@ -1651,7 +1651,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { | |||
| if let Some(builder) = self.context.report_lint(&CONFLICTING_DECLARATIONS, node) { | |||
| builder.into_diagnostic(format_args!( | |||
| "Conflicting declared types for `{place}`: {}", | |||
| conflicting.display(db) | |||
| conflicting.iter().map(|ty| ty.display(db)).join(", ") | |||
There was a problem hiding this comment.
Can't easily get this to work with TypeArrayDisplay, so just join it manually.
There was a problem hiding this comment.
I added this function yesterday, which could possibly be generalised:
ruff/crates/ty_python_semantic/src/types/diagnostic.rs
Lines 2039 to 2064 in c1fed55
its behaviour is that if you have a list of two types A and B, it creates the string "`A` and `B`", if you have a list of three types A, B and C, it creates the string "`A`, `B` and `C`", for four types it creates the string "`A`, `B`, `C` and `D`", etc.
There was a problem hiding this comment.
I guess a generalised version would be something like this (untested):
fn describe_types_list<'db, I, IT, D>(db: &'db dyn Db, types: I) -> String
where
I: IntoIterator<IntoIter = IT>,
IT: ExactSizeIterator<Item = D> + DoubleEndedIterator,
D: std::fmt::Display,
{
let types = types.into_iter();
debug_assert!(types.len() >= 2);
let final_type = types.next_back().unwrap();
let penultimate_type = types.next_back().unwrap();
let mut buffer = String::new();
for type in types {
buffer.push('`');
buffer.push_str(type);
buffer.push_str("`, ");
}
buffer.push('`');
buffer.push_str(penultimate_type);
buffer.push_str("` and `");
buffer.push_str(final_type);
buffer.push('`');
buffer
}
There was a problem hiding this comment.
if you have a list of three types
A,BandC, it creates the string"`A`, `B` and `C`", for four types it creates the string"`A`, `B`, `C` and `D`", etc.
I see we are Oxford comma nemeses... 🐂
There was a problem hiding this comment.
Thank you, Alex. I'll keep this for a follow-up.
This comment was marked as resolved.
This comment was marked as resolved.
sharkdp
commented
Jun 25, 2025
sharkdp
force-pushed
the
david/public-types
branch
from
0c77760 to
d055a0d
Compare
sharkdp
force-pushed
the
david/public-types
branch
from
d1c15ca to
4226670
Compare
sharkdp
changed the title
dcreager
added a commit
that referenced
this pull request
Jun 26, 2025
* main: [ty] Add regression-benchmark for attribute-assignment hang (#18957) [ty] Format conflicting types as an enumeration (#18956) [ty] Prevent union builder construction for just one declaration (#18954) [ty] Infer nonlocal types as unions of all reachable bindings (#18750) [`pyflakes`] Mark `F504`/`F522`/`F523` autofix as unsafe if there's a call with side effect (#18839) [`playground`] Add ruff logo docs link to Header.tsx (#18947) [ty] Reduce the overwhelming complexity of `TypeInferenceBuilder::infer_call_expression` (#18943) [ty] Add subdiagnostic about empty bodies in more cases (#18942) [ty] Move search path resolution to `Options::to_program_settings` (#18937) [`flake8-errmsg`] Extend `EM101` to support byte strings (#18867) Move big rule implementations (#18931) [`pylint`] Allow fix with comments and document performance implications (`PLW3301`) (#18936)
sharkdp
added a commit
that referenced
this pull request
Jul 1, 2025
…lookups (#18955) ## Summary Remove a hack in control flow modeling that was treating `return` statements at the end of function bodies in a special way (basically considering the state *just before* the `return` statement as the end-of-scope state). This is not needed anymore now that #18750 has been merged. In order to make this work, we now use *all reachable bindings* for purposes of finding implicit instance attribute assignments as well as for deferred lookups of symbols. Both would otherwise be affected by this change: ```py def C: def f(self): self.x = 1 # a reachable binding that is not visible at the end of the scope return ``` ```py def f(): class X: ... # a reachable binding that is not visible at the end of the scope x: "X" = X() # deferred use of `X` return ``` Implicit instance attributes also required another change. We previously kept track of possibly-unbound instance attributes in some cases, but we now give up on that completely and always consider *implicit* instance attributes to be bound if we see a reachable binding in a reachable method. The previous behavior was somewhat inconsistent anyway because we also do not consider attributes possibly-unbound in other scenarios: we do not (and can not) keep track of whether or not methods are called that define these attributes. closes astral-sh/ty#711 ## Ecosystem analysis I think this looks very positive! * We see an unsurprising drop in `possibly-unbound-attribute` diagnostics (599), mostly for classes that define attributes in `try … except` blocks, `for` loops, or `if … else: raise …` constructs. There might obviously also be true positives that got removed, but the vast majority should be false positives. * There is also a drop in `possibly-unresolved-reference` / `unresolved-reference` diagnostics (279+13) from the change to deferred lookups. * Some `invalid-type-form` false positives got resolved (13), because we can now properly look up the names in the annotations. * There are some new *true* positives in `attrs`, since we understand the `Attribute` annotation that was previously inferred as `Unknown` because of a re-assignment after the class definition. ## Test Plan The existing attributes.md test suite has sufficient coverage here.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR includes a behavioral change to how we infer types for nonlocal uses of symbols within a module. Where we would previously use the type that a use at the end of the scope would see, we now consider all reachable bindings and union the results:
This helps especially in cases where the the end of the scope is not reachable:
This PR also proposes to skip the boundness analysis of nonlocal uses. This is consistent with the "all reachable bindings" strategy, because the implicit
x = <unbound>binding is also always reachable, and we would have to emit "possibly-unresolved" diagnostics for every nonlocal use otherwise. Changing this behavior allows common use-cases like the following to type check without any errors:closes astral-sh/ty#210
closes astral-sh/ty#607
closes astral-sh/ty#699
Follow up
It is now possible to resolve the following TODO, but I would like to do that as a follow-up, because it requires some changes to how we treat implicit attribute assignments, which could result in ecosystem changes that I'd like to see separately.
ruff/crates/ty_python_semantic/src/semantic_index/builder.rs
Lines 1095 to 1117 in 315fb0f
Ecosystem analysis
Full report
possibly-unresolved-referencediagnostics (7818) because we do not analyze boundness for nonlocal uses of symbols inside modules anymore.unresolved-referencediagnostics (231) in scenarios like this:Test Plan
New Markdown tests