test_expanduser_pwd2 in test_posixpath can be very slow

[itamaro]

Bug report

Bug description:

This test can take many minutes to complete, on systems where pwd.getpwall() might return a very large number of entries.

./python -m test -v test_posixpath -m test_expanduser_pwd2

I don't see any obvious benefit to this test running on literally every single entry.

My proposal is to simply limit the iteration in the test to the first 10 (or any other arbitrary reasonable number) entries (see attached PR).
Another option is to put the test behind some resource to allow skipping it easily without needing to patch.

CPython versions tested on:

CPython main branch, 3.13, 3.12, 3.14

Operating systems tested on:

Linux

Linked PRs

• gh-132185: Avoid slow test_expanduser_pwd2 in test_posixpath #132186
• gh-132185: Speed up expanduser() test with large password database #132231
• [3.13] gh-132185: Speed up expanduser() test with large password database (GH-132231) #132443

[serhiy-storchaka]

This test was initially added in #117503. If there are users with non-ASCII names, they are usually not in first 10 entries, so your proposed change breaks the purpose of the test.

#121207 added subTest(), and it made the test slower, because it now needs to save information for each entry. #121228 added the use of getpwnam() and added more info for subTest(), that made it even slower.

cc @vstinner

[itamaro]

This test was initially added in #117503. If there are users with non-ASCII names, they are usually not in first 10 entries, so your proposed change breaks the purpose of the test.

thanks for adding the historical context and original intent!

it seems somewhat fragile to rely on the system having users with non-ASCII names for covering this functionality, isn't it?
I'm not too familiar with this module and the underlying system APIs, but would it be possible to explicitly test the non-ASCII functionality using mocks?

[serhiy-storchaka]

I just used the method that seemed easier to me at the time. Using a mock has its advantages, but using a real password database also has its advantages.

#132231 only limits the number of entries if the CPU resource is not enabled. 100k users is too much, so I'll perhaps limit the number of names even if the CPU resource is enabled.

[itamaro]

@serhiy-storchaka would you like to go ahead with your fix? (reviewed & approved!)