Skip to content

Add typehint overloads for __getitem__ #109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Nov 12, 2025
Merged

Add typehint overloads for __getitem__ #109

merged 5 commits into from
Nov 12, 2025

Conversation

@Thijss
Copy link
Member

@Thijss Thijss commented Oct 31, 2025 •
edited
Loading

Adds proper overloads for getitem for typecheckers.

piggyback:

  • I've also dropped support for tuple masks, since numpy handles them unexpectedly (see comment in test). This is technically breaking but I don't expect many people to use it. And it will break loudly (NotImplementedError)

Thijss added 3 commits October 31, 2025 11:52
@Thijss
Make __getitem__ typesafe by restricting input
a20d678 
Signed-off-by: Thijs Baaijen <[email protected]>
@Thijss
revert some changes
ab3dd19 
Signed-off-by: Thijs Baaijen <[email protected]>
@Thijss
revert some changes
f2aabd6 
Signed-off-by: Thijs Baaijen <[email protected]>
@Thijss Thijss changed the title Overload getitem Add tyehint overloads for __getitem__ Oct 31, 2025
@Thijss Thijss changed the title Add tyehint overloads for __getitem__ Add typehint overloads for __getitem__ Oct 31, 2025
vincentkoppen
vincentkoppen reviewed Nov 4, 2025
View reviewed changes
jaapschoutenalliander
jaapschoutenalliander reviewed Nov 12, 2025
View reviewed changes
Copy link
Member

@jaapschoutenalliander jaapschoutenalliander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thing to test: impact on performance (https://github.com/PowerGridModel/power-grid-model-ds/blob/main/tests/performance/array_performance_tests.py). Since these changes can impact a lot of calls

@Thijss
Copy link
Member Author

Thijss commented Nov 12, 2025

overload-getitem

-------------------- perftest_initialize --------------------

size: 10000

	structured: intialise                                                                                | 0.36s
	rec: intialise                                                                                       | 0.35s
	fancy: intialise                                                                                     | 0.67s

size: 50000

	structured: intialise                                                                                | 0.39s
	rec: intialise                                                                                       | 0.40s
	fancy: intialise                                                                                     | 1.25s

-------------------- perftest_slice --------------------

size: 10000

	structured: input_array[0:10]                                                                        | 0.00s
	rec: input_array[0:10]                                                                               | 0.03s
	fancy: input_array[0:10]                                                                             | 0.00s

size: 50000

	structured: input_array[0:10]                                                                        | 0.00s
	rec: input_array[0:10]                                                                               | 0.03s
	fancy: input_array[0:10]                                                                             | 0.00s

-------------------- perftest_set_field --------------------

size: 10000

	structured: input_array['id'] = 1                                                                    | 0.09s
	rec: input_array['id'] = 1                                                                           | 0.09s
	fancy: input_array['id'] = 1                                                                         | 0.15s

size: 50000

	structured: input_array['id'] = 1                                                                    | 0.45s
	rec: input_array['id'] = 1                                                                           | 0.45s
	fancy: input_array['id'] = 1                                                                         | 0.76s

-------------------- perftest_set_attr --------------------

size: 10000

	structured: input_array['id'] = 1                                                                    | 0.09s
	rec: input_array.id = 1                                                                              | 0.10s
	fancy: input_array.id = 1                                                                            | 0.16s

size: 50000

	structured: input_array['id'] = 1                                                                    | 0.45s
	rec: input_array.id = 1                                                                              | 0.46s
	fancy: input_array.id = 1                                                                            | 0.76s

-------------------- perftest_loop_slice_1 --------------------

size: 1000

	structured: for i in range(1000): input_array[i]                                                     | 0.02s
	rec: for i in range(1000): input_array[i]                                                            | 0.26s
	fancy: for i in range(1000): input_array[i]                                                          | 0.62s

size: 5000

	structured: for i in range(5000): input_array[i]                                                     | 0.09s
	rec: for i in range(5000): input_array[i]                                                            | 1.29s
	fancy: for i in range(5000): input_array[i]                                                          | 3.08s

-------------------- perftest_loop_data_slice_1 --------------------

size: 1000

	structured: for i in range(1000): input_array[i]                                                     | 0.02s
	rec: for i in range(1000): input_array[i]                                                            | 0.26s
	fancy: for i in range(1000): input_array.data[i]                                                     | 0.07s

size: 5000

	structured: for i in range(5000): input_array[i]                                                     | 0.09s
	rec: for i in range(5000): input_array[i]                                                            | 1.27s
	fancy: for i in range(5000): input_array.data[i]                                                     | 0.34s

-------------------- perftest_loop_slice --------------------

size: 1000

	structured: for i in range(1000): input_array[i:i+1]                                                 | 0.03s
	rec: for i in range(1000): input_array[i:i+1]                                                        | 2.72s
	fancy: for i in range(1000): input_array[i:i+1]                                                      | 0.39s

size: 5000

	structured: for i in range(5000): input_array[i:i+1]                                                 | 0.16s
	rec: for i in range(5000): input_array[i:i+1]                                                        | 13.84s
	fancy: for i in range(5000): input_array[i:i+1]                                                      | 1.97s

-------------------- perftest_loop_set_field --------------------

size: 1000

	structured: for i in range(1000): input_array['id'][i] = 1                                           | 0.03s
	rec: for i in range(1000): input_array['id'][i] = 1                                                  | 0.69s
	fancy: for i in range(1000): input_array['id'][i] = 1                                                | 0.20s

size: 5000

	structured: for i in range(5000): input_array['id'][i] = 1                                           | 0.19s
	rec: for i in range(5000): input_array['id'][i] = 1                                                  | 3.49s
	fancy: for i in range(5000): input_array['id'][i] = 1                                                | 0.98s

-------------------- perftest_loop_get_field --------------------

size: 1000

	structured: for row in input_array: row['id']                                                        | 0.10s
	rec: for row in input_array: row['id']                                                               | 0.49s
	fancy: for row in input_array: row['id']                                                             | 0.72s

size: 5000

	structured: for row in input_array: row['id']                                                        | 0.47s
	rec: for row in input_array: row['id']                                                               | 2.44s
	fancy: for row in input_array: row['id']                                                             | 3.58s

-------------------- perftest_loop_data_get_field --------------------

size: 1000

	structured: for row in input_array.data: row['id']                                                   | Not supported
	rec: for row in input_array.data: row['id']                                                          | Not supported
	fancy: for row in input_array.data: row['id']                                                        | 0.10s

size: 5000

	structured: for row in input_array.data: row['id']                                                   | Not supported
	rec: for row in input_array.data: row['id']                                                          | Not supported
	fancy: for row in input_array.data: row['id']                                                        | 0.47s

-------------------- perftest_loop_get_attr --------------------

size: 1000

	structured: for row in input_array: row.id                                                           | Not supported
	rec: for row in input_array: row.id                                                                  | 0.96s
	fancy: for row in input_array: row.id                                                                | 0.68s

size: 5000

	structured: for row in input_array: row.id                                                           | Not supported
	rec: for row in input_array: row.id                                                                  | 4.81s
	fancy: for row in input_array: row.id                                                                | 3.42s

-------------------- perftest_fancypy_concat --------------------

size: 10000

	structured: np.concatenate([input_array, input_array])                                               | 0.18s
	rec: np.concatenate([input_array, input_array])                                                      | 0.19s
	fancy: fp.concatenate(input_array, input_array)                                                      | 0.19s

size: 50000

	structured: np.concatenate([input_array, input_array])                                               | 0.73s
	rec: np.concatenate([input_array, input_array])                                                      | 0.74s
	fancy: fp.concatenate(input_array, input_array)                                                      | 0.73s

-------------------- perftest_fancypy_unique --------------------

size: 1000

	structured: np.unique(input_array)                                                                   | 0.69s
	rec: np.unique(input_array)                                                                          | 0.69s
	fancy: fp.unique(input_array)                                                                        | 0.19s

size: 5000

	structured: np.unique(input_array)                                                                   | 4.64s
	rec: np.unique(input_array)                                                                          | 4.67s
	fancy: fp.unique(input_array)                                                                        | 1.25s

-------------------- perftest_fancypy_sort --------------------

size: 1000

	structured: np.sort(input_array)                                                                     | 0.65s
	rec: np.sort(input_array)                                                                            | 0.65s
	fancy: fp.sort(input_array)                                                                          | 0.14s

size: 5000

	structured: np.sort(input_array)                                                                     | 4.47s
	rec: np.sort(input_array)                                                                            | 4.49s
	fancy: fp.sort(input_array)                                                                          | 0.98s

     229158919 function calls (226156434 primitive calls) in 74.130 seconds

main

-------------------- perftest_initialize --------------------

size: 10000

	structured: intialise                                                                                | 0.39s
	rec: intialise                                                                                       | 0.38s
	fancy: intialise                                                                                     | 0.71s

size: 50000

	structured: intialise                                                                                | 0.42s
	rec: intialise                                                                                       | 0.41s
	fancy: intialise                                                                                     | 1.25s

-------------------- perftest_slice --------------------

size: 10000

	structured: input_array[0:10]                                                                        | 0.00s
	rec: input_array[0:10]                                                                               | 0.03s
	fancy: input_array[0:10]                                                                             | 0.01s

size: 50000

	structured: input_array[0:10]                                                                        | 0.00s
	rec: input_array[0:10]                                                                               | 0.03s
	fancy: input_array[0:10]                                                                             | 0.01s

-------------------- perftest_set_field --------------------

size: 10000

	structured: input_array['id'] = 1                                                                    | 0.09s
	rec: input_array['id'] = 1                                                                           | 0.10s
	fancy: input_array['id'] = 1                                                                         | 0.16s

size: 50000

	structured: input_array['id'] = 1                                                                    | 0.45s
	rec: input_array['id'] = 1                                                                           | 0.45s
	fancy: input_array['id'] = 1                                                                         | 0.77s

-------------------- perftest_set_attr --------------------

size: 10000

	structured: input_array['id'] = 1                                                                    | 0.09s
	rec: input_array.id = 1                                                                              | 0.10s
	fancy: input_array.id = 1                                                                            | 0.16s

size: 50000

	structured: input_array['id'] = 1                                                                    | 0.46s
	rec: input_array.id = 1                                                                              | 0.47s
	fancy: input_array.id = 1                                                                            | 0.76s

-------------------- perftest_loop_slice_1 --------------------

size: 1000

	structured: for i in range(1000): input_array[i]                                                     | 0.02s
	rec: for i in range(1000): input_array[i]                                                            | 0.25s
	fancy: for i in range(1000): input_array[i]                                                          | 0.76s

size: 5000

	structured: for i in range(5000): input_array[i]                                                     | 0.09s
	rec: for i in range(5000): input_array[i]                                                            | 1.25s
	fancy: for i in range(5000): input_array[i]                                                          | 3.85s

-------------------- perftest_loop_data_slice_1 --------------------

size: 1000

	structured: for i in range(1000): input_array[i]                                                     | 0.02s
	rec: for i in range(1000): input_array[i]                                                            | 0.25s
	fancy: for i in range(1000): input_array.data[i]                                                     | 0.07s

size: 5000

	structured: for i in range(5000): input_array[i]                                                     | 0.09s
	rec: for i in range(5000): input_array[i]                                                            | 1.26s
	fancy: for i in range(5000): input_array.data[i]                                                     | 0.34s

-------------------- perftest_loop_slice --------------------

size: 1000

	structured: for i in range(1000): input_array[i:i+1]                                                 | 0.03s
	rec: for i in range(1000): input_array[i:i+1]                                                        | 2.76s
	fancy: for i in range(1000): input_array[i:i+1]                                                      | 0.53s

size: 5000

	structured: for i in range(5000): input_array[i:i+1]                                                 | 0.16s
	rec: for i in range(5000): input_array[i:i+1]                                                        | 13.95s
	fancy: for i in range(5000): input_array[i:i+1]                                                      | 2.68s

-------------------- perftest_loop_set_field --------------------

size: 1000

	structured: for i in range(1000): input_array['id'][i] = 1                                           | 0.03s
	rec: for i in range(1000): input_array['id'][i] = 1                                                  | 0.69s
	fancy: for i in range(1000): input_array['id'][i] = 1                                                | 0.22s

size: 5000

	structured: for i in range(5000): input_array['id'][i] = 1                                           | 0.18s
	rec: for i in range(5000): input_array['id'][i] = 1                                                  | 3.42s
	fancy: for i in range(5000): input_array['id'][i] = 1                                                | 1.06s

-------------------- perftest_loop_get_field --------------------

size: 1000

	structured: for row in input_array: row['id']                                                        | 0.10s
	rec: for row in input_array: row['id']                                                               | 0.50s
	fancy: for row in input_array: row['id']                                                             | 0.73s

size: 5000

	structured: for row in input_array: row['id']                                                        | 0.49s
	rec: for row in input_array: row['id']                                                               | 2.42s
	fancy: for row in input_array: row['id']                                                             | 3.64s

-------------------- perftest_loop_data_get_field --------------------

size: 1000

	structured: for row in input_array.data: row['id']                                                   | Not supported
	rec: for row in input_array.data: row['id']                                                          | Not supported
	fancy: for row in input_array.data: row['id']                                                        | 0.10s

size: 5000

	structured: for row in input_array.data: row['id']                                                   | Not supported
	rec: for row in input_array.data: row['id']                                                          | Not supported
	fancy: for row in input_array.data: row['id']                                                        | 0.49s

-------------------- perftest_loop_get_attr --------------------

size: 1000

	structured: for row in input_array: row.id                                                           | Not supported
	rec: for row in input_array: row.id                                                                  | 0.97s
	fancy: for row in input_array: row.id                                                                | 0.70s

size: 5000

	structured: for row in input_array: row.id                                                           | Not supported
	rec: for row in input_array: row.id                                                                  | 4.84s
	fancy: for row in input_array: row.id                                                                | 3.47s

-------------------- perftest_fancypy_concat --------------------

size: 10000

	structured: np.concatenate([input_array, input_array])                                               | 0.19s
	rec: np.concatenate([input_array, input_array])                                                      | 0.19s
	fancy: fp.concatenate(input_array, input_array)                                                      | 0.19s

size: 50000

	structured: np.concatenate([input_array, input_array])                                               | 0.77s
	rec: np.concatenate([input_array, input_array])                                                      | 0.84s
	fancy: fp.concatenate(input_array, input_array)                                                      | 0.75s

-------------------- perftest_fancypy_unique --------------------

size: 1000

	structured: np.unique(input_array)                                                                   | 0.69s
	rec: np.unique(input_array)                                                                          | 0.72s
	fancy: fp.unique(input_array)                                                                        | 0.19s

size: 5000

	structured: np.unique(input_array)                                                                   | 4.79s
	rec: np.unique(input_array)                                                                          | 4.82s
	fancy: fp.unique(input_array)                                                                        | 1.22s

-------------------- perftest_fancypy_sort --------------------

size: 1000

	structured: np.sort(input_array)                                                                     | 0.65s
	rec: np.sort(input_array)                                                                            | 0.66s
	fancy: fp.sort(input_array)                                                                          | 0.15s

size: 5000

	structured: np.sort(input_array)                                                                     | 4.65s
	rec: np.sort(input_array)                                                                            | 4.65s
	fancy: fp.sort(input_array)                                                                          | 0.99s

     241178884 function calls (238176399 primitive calls) in 77.060 seconds

@Thijss
merge main
65434f0 
Signed-off-by: Thijs Baaijen <[email protected]>
@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 12, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@Thijss Thijss merged commit a03329a into main Nov 12, 2025
15 checks passed
@Thijss Thijss deleted the overload-getitem branch November 12, 2025 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jaapschoutenalliander jaapschoutenalliander jaapschoutenalliander left review comments

@vincentkoppen vincentkoppen vincentkoppen approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Thijss @vincentkoppen @jaapschoutenalliander