Skip to content

Commit 083df0c

Browse files
authored
[red-knot] Support custom __getattr__ methods (#16668)
## Summary Add support for calling custom `__getattr__` methods in case an attribute is not otherwise found. This allows us to get rid of many ecosystem false positives where we previously emitted errors when accessing attributes on `argparse.Namespace`. closes #16614 ## Test Plan * New Markdown tests * Observed expected ecosystem changes (the changes for `arrow` also look fine, since the `Arrow` class has a custom [`__getattr__` here](https://github.com/arrow-py/arrow/blob/1d70d0091980ea489a64fa95a48e99b45f29f0e7/arrow/arrow.py#L802-L815))
1 parent a176c1a commit 083df0c

File tree

2 files changed

+132
-2
lines changed

2 files changed

+132
-2
lines changed

crates/red_knot_python_semantic/resources/mdtest/attributes.md

+92
Original file line numberDiff line numberDiff line change
@@ -1199,6 +1199,98 @@ reveal_type(C.__mro__) # revealed: tuple[Literal[C], Literal[B], Any, Literal[A
11991199
reveal_type(C.x) # revealed: Literal[1] & Any
12001200
```
12011201

1202+
## Classes with custom `__getattr__` methods
1203+
1204+
### Basic
1205+
1206+
If a type provides a custom `__getattr__` method, we use the return type of that method as the type
1207+
for unknown attributes. Consider the following `CustomGetAttr` class:
1208+
1209+
```py
1210+
from typing import Literal
1211+
1212+
def flag() -> bool:
1213+
return True
1214+
1215+
class GetAttrReturnType: ...
1216+
1217+
class CustomGetAttr:
1218+
class_attr: int = 1
1219+
1220+
if flag():
1221+
possibly_unbound: bytes = b"a"
1222+
1223+
def __init__(self) -> None:
1224+
self.instance_attr: str = "a"
1225+
1226+
def __getattr__(self, name: str) -> GetAttrReturnType:
1227+
return GetAttrReturnType()
1228+
```
1229+
1230+
We can access arbitrary attributes on instances of this class, and the type of the attribute will be
1231+
`GetAttrReturnType`:
1232+
1233+
```py
1234+
c = CustomGetAttr()
1235+
1236+
reveal_type(c.whatever) # revealed: GetAttrReturnType
1237+
```
1238+
1239+
If an attribute is defined on the class, it takes precedence over the `__getattr__` method:
1240+
1241+
```py
1242+
reveal_type(c.class_attr) # revealed: int
1243+
```
1244+
1245+
If the class attribute is possibly unbound, we union the type of the attribute with the fallback
1246+
type of the `__getattr__` method:
1247+
1248+
```py
1249+
reveal_type(c.possibly_unbound) # revealed: bytes | GetAttrReturnType
1250+
```
1251+
1252+
Instance attributes also take precedence over the `__getattr__` method:
1253+
1254+
```py
1255+
# Note: we could attempt to union with the fallback type of `__getattr__` here, as we currently do not
1256+
# attempt to determine if instance attributes are always bound or not. Neither mypy nor pyright do this,
1257+
# so it's not a priority.
1258+
reveal_type(c.instance_attr) # revealed: str
1259+
```
1260+
1261+
### Type of the `name` parameter
1262+
1263+
If the `name` parameter of the `__getattr__` method is annotated with a (union of) literal type(s),
1264+
we only consider the attribute access to be valid if the accessed attribute is one of them:
1265+
1266+
```py
1267+
from typing import Literal
1268+
1269+
class Date:
1270+
def __getattr__(self, name: Literal["day", "month", "year"]) -> int:
1271+
return 0
1272+
1273+
date = Date()
1274+
1275+
reveal_type(date.day) # revealed: int
1276+
reveal_type(date.month) # revealed: int
1277+
reveal_type(date.year) # revealed: int
1278+
1279+
# error: [unresolved-attribute] "Type `Date` has no attribute `century`"
1280+
reveal_type(date.century) # revealed: Unknown
1281+
```
1282+
1283+
### `argparse.Namespace`
1284+
1285+
A standard library example of a class with a custom `__getattr__` method is `argparse.Namespace`:
1286+
1287+
```py
1288+
import argparse
1289+
1290+
def _(ns: argparse.Namespace):
1291+
reveal_type(ns.whatever) # revealed: Any
1292+
```
1293+
12021294
## Objects of all types have a `__class__` method
12031295

12041296
The type of `x.__class__` is the same as `x`'s meta-type. `x.__class__` is always the same value as

crates/red_knot_python_semantic/src/types.rs

+40-2
Original file line numberDiff line numberDiff line change
@@ -2008,12 +2008,50 @@ impl<'db> Type<'db> {
20082008
| Type::FunctionLiteral(..) => {
20092009
let fallback = self.instance_member(db, name_str);
20102010

2011-
self.invoke_descriptor_protocol(
2011+
let result = self.invoke_descriptor_protocol(
20122012
db,
20132013
name_str,
20142014
fallback,
20152015
InstanceFallbackShadowsNonDataDescriptor::No,
2016-
)
2016+
);
2017+
2018+
let custom_getattr_result =
2019+
|| {
2020+
// Typeshed has a fake `__getattr__` on `types.ModuleType` to help out with dynamic imports.
2021+
// We explicitly hide it here to prevent arbitrary attributes from being available on modules.
2022+
if self.into_instance().is_some_and(|instance| {
2023+
instance.class.is_known(db, KnownClass::ModuleType)
2024+
}) {
2025+
return Symbol::Unbound.into();
2026+
}
2027+
2028+
self.try_call_dunder(
2029+
db,
2030+
"__getattr__",
2031+
&CallArguments::positional([Type::StringLiteral(
2032+
StringLiteralType::new(db, Box::from(name.as_str())),
2033+
)]),
2034+
)
2035+
.map(|outcome| Symbol::bound(outcome.return_type(db)))
2036+
// TODO: Handle call errors here.
2037+
.unwrap_or(Symbol::Unbound)
2038+
.into()
2039+
};
2040+
2041+
match result {
2042+
member @ SymbolAndQualifiers {
2043+
symbol: Symbol::Type(_, Boundness::Bound),
2044+
qualifiers: _,
2045+
} => member,
2046+
member @ SymbolAndQualifiers {
2047+
symbol: Symbol::Type(_, Boundness::PossiblyUnbound),
2048+
qualifiers: _,
2049+
} => member.or_fall_back_to(db, custom_getattr_result),
2050+
SymbolAndQualifiers {
2051+
symbol: Symbol::Unbound,
2052+
qualifiers: _,
2053+
} => custom_getattr_result(),
2054+
}
20172055
}
20182056

20192057
Type::ClassLiteral(..) | Type::SubclassOf(..) => {

0 commit comments

Comments
 (0)