Ruff rules for comprehensions and performance

[cclauss]

https://docs.astral.sh/ruff

% ruff check --select=C4,PERF --statistics

48	C408   	[ ] unnecessary-collection-call
 1	C400   	[ ] unnecessary-generator-list
 1	C402   	[ ] unnecessary-generator-dict
 1	C414   	[ ] unnecessary-double-cast-or-process
 1	C416   	[ ] unnecessary-comprehension
 1	C417   	[ ] unnecessary-map
 1	C419   	[ ] unnecessary-comprehension-in-call
 1	C420   	[*] unnecessary-dict-comprehension-for-iterable
 1	PERF203	[ ] try-except-in-loop
 1	PERF401	[ ] manual-list-comprehension

% ruff rule PERF401

manual-list-comprehension (PERF401)

Derived from the Perflint linter.

Fix is sometimes available.

What it does

Checks for for loops that can be replaced by a list comprehension.

Why is this bad?

When creating a transformed list from an existing list using a for-loop,
prefer a list comprehension. List comprehensions are more readable and
more performant.

Using the below as an example, the list comprehension is ~10% faster on
Python 3.11, and ~25% faster on Python 3.10.

Note that, as with all perflint rules, this is only intended as a
micro-optimization, and will have a negligible impact on performance in
most cases.

Example

original = list(range(10000))
filtered = []
for i in original:
    if i % 2:
        filtered.append(i)

Use instead:

original = list(range(10000))
filtered = [x for x in original if x % 2]

If you're appending to an existing list, use the extend method instead:

original = list(range(10000))
filtered.extend(x for x in original if x % 2)

[DhavalGojiya]

@cclauss

When we don't specify any rules for the Ruff linter, the default rules are:

["E4", "E7", "E9", "F"]

Since you ran:

ruff check --select=C4,PERF --statistics

At this point, Ruff will check the codebase against only two rule categories:

1. C4
2. PERF

and you fixed the rule violation accordingly.

However, in the future, when continuing work on this project, if a contributor makes a similar mistake - for example, using an explicit for loop instead of a list comprehension - the current CI pipeline (which runs only ruff check .) will validate code only against the default rules:

["E4", "E7", "E9", "F"]

CI pipeline will not be able to catch issues for this new rule since it checks against defaults only.
So, should we add C4 and PERF explicitly in pyproject.toml like this?

[tool.ruff.lint]
# On top of the defaults (`E4`, `E7`, `E9`, and `F`), enable C4 and PERF
select = ["E4", "E7", "E9", "F", "C4", "PERF"]

Ref: Ruff doc

[cclauss]

@DhavalGojiya Yes, I know how ruff works... More to come.

• https://github.com/cclauss/itinerant-tester/actions/runs/18445793810/job/52552312339#step:17:18