Move object dunders from FunctionModel to ObjectModel
#2847
Conversation
Signed-off-by: Emmanuel Ferdman <[email protected]>
Codecov Reportβ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2847 +/- ##
==========================================
- Coverage 93.37% 93.33% -0.04%
==========================================
Files 92 92
Lines 11148 11168 +20
==========================================
+ Hits 10409 10424 +15
- Misses 739 744 +5
Flags with carried forward coverage won't be shown. Click here to find out more.
π New features to boost your workflow:
|
There was a problem hiding this comment.
As can be seen from the docs you link, most of these should live on ObjectModel instead as they come from object.
We tried to do so before in #1519 but failed. Perhaps you want to have another look at that PR and see if you can fix the issue we faced there?
|
@DanielNoord Thanks! Iβll take a look at this. From that thread, it seems you were close to a solution, except for one test case that didnβt pass. Do you remember which test it was? |
|
@emmanuel-ferdman I believe it was an issue in the |
Signed-off-by: Emmanuel Ferdman <[email protected]>
for more information, see https://pre-commit.ci
Signed-off-by: Emmanuel Ferdman <[email protected]>
|
@DanielNoord I've put together an initial solution for moving object dunders from The solution uses a placeholder pattern:
How it works:
Thanks for any feedback π |
| # Builtin dunder methods have empty bodies, return Uninferable. | ||
| if ( | ||
| self.root().qname() == "builtins" | ||
| and self.name.startswith("__") | ||
| and self.name.endswith("__") | ||
| and self.parent | ||
| and self.parent.__class__.__name__ == "ClassDef" | ||
| ): | ||
| yield util.Uninferable | ||
| return |
There was a problem hiding this comment.
I think I sort of understand this change, but don't understand why we need to special case it like this.
Can we replace the checks with a look up in special_attributes? Or doesn't that work?
There was a problem hiding this comment.
We need this case because infer_call_result() has to decide what a call returns, and for built-in dunder methods there is no Python body to inspect (since they're implemented in C). If we tried to parse return nodes we would either fail or produce incorrect results.
special_attributes only tells us that an attribute exists (a placeholder), not whether there is a Python body to infer a return value from. Therefore it cannot replace the empty-body/builtins check used here.
But on second thought, I simplified the case by explicitly checking len(self.body) == 0.
| special_attr = self.special_attributes.lookup(name) | ||
| if not isinstance( | ||
| special_attr, (util.UninferableBase, node_classes.Unknown) | ||
| ): | ||
| result = [special_attr] | ||
| return result |
There was a problem hiding this comment.
What is the effect of this? What will we eventually return if the if is not True?
There was a problem hiding this comment.
This block is a short-circuit used when there are no concrete locals/ancestor definitions: originally it always returned special_attributes.lookup(name) (even if that was Unknown or Uninferable). The new behavior only returns the special_attr when it is a concrete value (not node_classes.Unknown or util.UninferableBase), preventing a placeholder from being returned prematurely and masking an override in a metaclass/base class. If the if is not true we continue the normal lookup (metaclass lookup, collect/filter locals/ancestors) and ultimately return any real definitions found - otherwise an AttributeInferenceError is raised. Placeholders (Unknown/Uninferable) therefore mean βkeep looking,β not βreturn this as the final result.β
| except AttributeInferenceError as exc: | ||
| if self.special_attributes and name in self.special_attributes: | ||
| return [self.special_attributes.lookup(name)] | ||
| special_attr = self.special_attributes.lookup(name) |
There was a problem hiding this comment.
Similar question as above
There was a problem hiding this comment.
Same pattern as the ClassDef.getattr() check - when special_attr is Unknown or Uninferable, we skip the early return and continue to the if lookupclass: block which searches the class for the attribute. This ensures we find actual implementations in classes rather than stopping at the Unknown placeholder.
| if not isinstance(mcs, nodes.ClassDef): | ||
| # Not a valid first argument. | ||
| return None | ||
| raise InferenceError( |
There was a problem hiding this comment.
This feels like it could be merged on its own, is that True? In that case I'd probably split this out to ensure we don't do too much in a single PR.
There was a problem hiding this comment.
This change is independent of the object dunder work. It's about improving type.new() validation error messages - changing silent None returns to descriptive InferenceError exceptions. The existing tests already expect InferenceError, so it's fully compatible. I'm happy to split it into a separate PR to keep this one focused on the object dunder changes. Should I remove it from this PR and create a new one, or would you prefer I keep it here as a small related improvement?
| # Special case: __hash__ = None overrides ObjectModel for unhashable types. | ||
| if name == "__hash__" and value is None: | ||
| _attach_local_node(node, nodes.const_factory(value), name) |
There was a problem hiding this comment.
Should this be the case for all attributes where value is None? Or only hash?
There was a problem hiding this comment.
I believe only for __hash__. In Python __hash__ = None has a special semantic meaning - it explicitly marks a type as unhashable and must override the inherited object.__hash__. Other attributes set to None do not carry this override semantics and should not implicitly replace special_attributes entries, because that could hide real implementations or metadata. So I belive we should have this special-case restricted to __hash__ only.
| child = object_build_datadescriptor(node, member) | ||
| elif isinstance(member, tuple(node_classes.CONST_CLS)): | ||
| if alias in node.special_attributes: | ||
| # Special case: __hash__ = None overrides ObjectModel for unhashable types. |
There was a problem hiding this comment.
Same question as above
There was a problem hiding this comment.
Same reasoning as the previous hash question - hash = None is the only attribute where None has special semantic meaning in Python (marks unhashable types). Other None values don't need this override behavior.
| # Builtin dunder methods now return Uninferable instead of raising InferenceError | ||
| result = next(lenmeth.infer_call_result(None, None)) | ||
| assert result is util.Uninferable |
There was a problem hiding this comment.
Ah, I think the changes above now make more sense to me.
pylint works correctly with this change? Thanks for testing that.
There was a problem hiding this comment.
Yes - pylint works correctly (verified by running pylint test suite with astroid changes). pylint handles Uninferable in two ways: (1) explicit isinstance(value, util.UninferableBase) checks (e.g., pylint/checkers/typecheck.py:1365-1369, and (2) Uninferable is falsy like None, so if inferred: checks skip both. The behavior is identical - when builtin dunders can't be inferred, pylint skips type checking whether it gets None from an exception or Uninferable from the return value.
| ) | ||
| inferred = next(eq_result.infer()) | ||
| assert isinstance(inferred, nodes.Const) | ||
| assert inferred.value == "custom equality" |
Signed-off-by: Emmanuel Ferdman <[email protected]>
emmanuel-ferdman
changed the title
FunctionModel to ObjectModel
Type of Changes
Description
Based on the Python docs, this PR does:
__setattr___and__delattr___typos (Current code uses 3 underscores but should be 2) - I belive its by mistake. Try to reporduce:__le__,__ge__,__builtins__,__getstate__,__init_subclass__,__type_params__Fixes #2742 #2741.