ruff rules for comprehensions and performance

[cclauss]

% ruff check --select=ASYNC,C4,E,F,PERF,PIE,W \
             --ignore=E722,E731,E741,F401,PERF203,PIE790 \
             --line-length=130 \
             --statistics

11	C402   	[*] unnecessary-generator-dict
11	W293   	[*] blank-line-with-whitespace
 8	PERF401	[ ] manual-list-comprehension
 5	C401   	[*] unnecessary-generator-set
 2	C405   	[*] unnecessary-literal-set
 2	PIE808 	[*] unnecessary-range-start
 1	C413   	[*] unnecessary-call-around-sorted
 1	PERF102	[*] incorrect-dict-iterator
[*] fixable with `ruff check --fix`

% ruff rule PERF401

manual-list-comprehension (PERF401)

Derived from the Perflint linter.

Fix is sometimes available.

What it does

Checks for for loops that can be replaced by a list comprehension.

Why is this bad?

When creating a transformed list from an existing list using a for-loop,
prefer a list comprehension. List comprehensions are more readable and
more performant.

Using the below as an example, the list comprehension is ~10% faster on
Python 3.11, and ~25% faster on Python 3.10.

Note that, as with all perflint rules, this is only intended as a
micro-optimization, and will have a negligible impact on performance in
most cases.

Example

original = list(range(10000))
filtered = []
for i in original:
    if i % 2:
        filtered.append(i)

Use instead:

original = list(range(10000))
filtered = [x for x in original if x % 2]

If you're appending to an existing list, use the extend method instead:

original = list(range(10000))
filtered.extend(x for x in original if x % 2)

Take care that if the original for-loop uses an assignment expression
as a conditional, such as if match:=re.match("\d+","123"), then
the corresponding comprehension must wrap the assignment
expression in parentheses to avoid a syntax error.

[mtrofin]

Thanks, @cclauss!

Thinking out loud, we should probably add ruff to the CI. I'll do it after this lands.

[cclauss]

% uv tool run yapf --in-place --parallel --recursive .

[mtrofin]

This is ready to merge

[mtrofin]

@cclauss could you "squash & merge" (i.e. checking if that works for you or if you're waiting for us :) I would prefer you merging in case you wanted to update the commit message, etc)

[boomanaiden154]

could you "squash & merge" (i.e. checking if that works for you or if you're waiting for us

People without permissions need someone to click merge on their behalf.

I can do it here once CI passes.

[mtrofin]

could you "squash & merge" (i.e. checking if that works for you or if you're waiting for us

People without permissions need someone to click merge on their behalf.

OK, fixed.

I can do it here once CI passes.