Skip to content

Fix used-before-assignment for PEP 695 type aliases + parameters #10488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions doc/whatsnew/fragments/9815.false_positive
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Fix used-before-assignment for PEP 695 type aliases and parameters.

Closes #9815
7 changes: 7 additions & 0 deletions pylint/checkers/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -1646,6 +1646,13 @@ def is_node_in_type_annotation_context(node: nodes.NodeNG) -> bool:
return False


def is_node_in_pep695_type_context(node: nodes.NodeNG) -> nodes.NodeNG | None:
"""Check if node is used in a TypeAlias or as part of a type param."""
return get_node_first_ancestor_of_type(
node, (nodes.TypeAlias, nodes.TypeVar, nodes.ParamSpec, nodes.TypeVarTuple)
)


def is_subclass_of(child: nodes.ClassDef, parent: nodes.ClassDef) -> bool:
"""Check if first node is a subclass of second node.

Expand Down
23 changes: 11 additions & 12 deletions pylint/checkers/variables.py
Original file line number Diff line number Diff line change
Expand Up @@ -1927,22 +1927,21 @@ def _check_consumer(

# Skip postponed evaluation of annotations
# and unevaluated annotations inside a function body
# as well as TypeAlias nodes.
if not (
self._postponed_evaluation_enabled
self._postponed_evaluation_enabled # noqa: RUF021
and (
isinstance(stmt, nodes.AnnAssign)
or (
isinstance(stmt, nodes.FunctionDef)
and node
not in {
*(stmt.args.defaults or ()),
*(stmt.args.kw_defaults or ()),
}
)
or isinstance(stmt, nodes.FunctionDef) # noqa: RUF021
and node
not in {
*(stmt.args.defaults or ()),
*(stmt.args.kw_defaults or ()),
}
)
) and not (
isinstance(stmt, nodes.AnnAssign)
or isinstance(stmt, nodes.AnnAssign) # noqa: RUF021
and utils.get_node_first_ancestor_of_type(stmt, nodes.FunctionDef)
or isinstance(stmt, nodes.TypeAlias)
):
self.add_message(
"used-before-assignment",
Expand Down Expand Up @@ -2018,7 +2017,7 @@ def _report_unfound_name_definition(
if (
self._postponed_evaluation_enabled
and utils.is_node_in_type_annotation_context(node)
):
) or utils.is_node_in_pep695_type_context(node):
return False
if self._is_builtin(node.name):
return False
Expand Down
1 change: 1 addition & 0 deletions pylint/testutils/functional/find_functional_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
"ext",
"regression",
"regression_02",
"used_02",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometimes I wonder if it still makes sense to keep the artificial file limit around. IMO it's just annoying most of the time. Should we remove it?

REASONABLY_DISPLAYABLE_VERTICALLY = 49
"""'Wet finger' number of files that are reasonable to display by an IDE.
'Wet finger' as in 'in my settings there are precisely this many'.
"""

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the file limit it was really annoying to find a functional tests though (scrolling for eternity). It's not going to be annoying for a while if we remove the file limit contraint though. Might be a release time check maybe ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the file limit it was really annoying to find a functional tests though (scrolling for eternity).

I mostly use Go to File ... in VS Code so I can't speak to that.

My issue here was a more practical one. I wanted to add one more test which should be placed alongside the other used_before_assignment test cases but when I did it immediately triggered limit and I wasn't even able to continue working on the test case. Sure I could move a lot of tests around here but that would just inflate the diff. Furthermore, moving tests makes git blame a lot more difficult. Sure there is .git-blame-ignore-revs but it only contains one commit.

It's not going to be annoying for a while if we remove the file limit contraint though. Might be a release time check maybe ?

That would probably just delay the issue and make releases more time consuming.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you never just browse the functional tests to see if something match what you want to test ? I agree with the git blame not working well if we move a lot of file. (It's already the case almost everywhere, I did refactor a lot in the past, and I'm not alone in 20 years, some git blame in pylint already make me feel like a cyber-crime forensic. I'm not saying we should make it worse voluntarily though).

I'm a little reluctant to remove all rules. One possible solution would be to find another clear organization for the functional tests (like in the extension functional tests directory, probably). One directory per message, avoid testing multiple message in the same file. Or remove constraint on regression directory only : the issue always arise in this directory, right ? Or remove constraint on regression directory but force regression tests to be inside tests/functional/regression/ with a file name being an int as in tests/functional/regression/8736.py. What do you think ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you never just browse the functional tests to see if something match what you want to test ?

Tbh not really. I usually just search for the error message and see which existing tests pop up or which tests fail if I change something 😅

I agree with the git blame not working well if we move a lot of file.

Yeah, it's broken already basically. Thankfully it still mostly works for the checkers themselves which is usually enough to find the right PR.

I'm a little reluctant to remove all rules. One possible solution would be to find another clear organization for the functional tests (like in the extension functional tests directory, probably). One directory per message, avoid testing multiple message in the same file.

We mostly do that already but sometimes it's just more practical to test multiple different things in one go. Otherwise we end up duplicating a lot of test code which will make maintaining it just more difficult.

Or remove constraint on regression directory only : the issue always arise in this directory, right ? Or remove constraint on regression directory but force regression tests to be inside tests/functional/regression/ with a file name being an int as in tests/functional/regression/8736.py. What do you think ?

The unused_variable and used_before_assignment folders are also frequent issues, so it's not just the regression dirs. If it were just one test file, that would be fine. However, with #10382 there are now tests with 3 or even 4 supporting files. Maybe a first small step could be to ignore these and only count .py files?

--
We don't need to find a particular solution for the issue here. It was just something that bothered me, so I wanted to highlight it and start the discussion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but isn't the discussion's goal to find a solution to the annoyance ;) ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
"""Direct parent directories that should be ignored."""

Expand Down
23 changes: 16 additions & 7 deletions tests/functional/u/used/used_before_assignment_py312.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,23 @@
"""used-before-assignment re: python 3.12 generic typing syntax (PEP 695)"""
"""Tests for used-before-assignment with Python 3.12 generic typing syntax (PEP 695)"""
# pylint: disable = invalid-name,missing-docstring,too-few-public-methods,unused-argument

from typing import TYPE_CHECKING, Callable

from typing import Callable
type Point[T] = tuple[T, ...]
type Alias[*Ts] = tuple[*Ts]
type Alias[**P] = Callable[P]

# pylint: disable = invalid-name, missing-class-docstring, too-few-public-methods
type Alias2[**P] = Callable[P, None]

# https://github.com/pylint-dev/pylint/issues/9815
type IntOrX = int | X # [used-before-assignment] FALSE POSITIVE
type AliasType = int | X | Y

class X:
pass

if TYPE_CHECKING:
class Y: ...

class Good[T: Y]: ...
type OtherAlias[T: Y] = T | None

# https://github.com/pylint-dev/pylint/issues/9884
def func[T: Y](x: T) -> None: # [redefined-outer-name] FALSE POSITIVE
...
2 changes: 1 addition & 1 deletion tests/functional/u/used/used_before_assignment_py312.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
used-before-assignment:11:20:11:21::Using variable 'X' before assignment:HIGH
redefined-outer-name:22:9:22:13:func:Redefining name 'T' from outer scope (line 6):UNDEFINED
16 changes: 16 additions & 0 deletions tests/functional/u/used_02/used_before_assignment_py313.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""Tests for used-before-assignment with Python 3.13 type var defaults (PEP 696)"""
# pylint: disable=missing-docstring,unused-argument,too-few-public-methods

from typing import TYPE_CHECKING

if TYPE_CHECKING:
class Y: ...

class Good1[T = Y]: ...
class Good2[*Ts = tuple[int, Y]]: ...
class Good3[**P = [int, Y]]: ...
type Alias[T = Y] = T | None

# https://github.com/pylint-dev/pylint/issues/9884
def func[T = Y](x: T) -> None: # [redefined-outer-name] FALSE POSITIVE
...
2 changes: 2 additions & 0 deletions tests/functional/u/used_02/used_before_assignment_py313.rc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[testoptions]
min_pyver=3.13
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
redefined-outer-name:15:9:15:14:func:Redefining name 'T' from outer scope (line 12):UNDEFINED
Loading