ctypes.util.find_library() does not find DLLs in directories added by os.add_dll_directory()

[davidva-ecco]

Bug report

Bug description:

Python version: 3.12.6
OS: Windows 11 64-bit

Behavior

When I use ctypes.util.find_library(<DLL name>) to search for a DLL in a Windows directory added to the search path with os.add_dll_directory(<DLL folder>), it is not found. However, if I provide find_library() with the path and name of the DLL, it is found.

Unfortunately this breaks a third-party module I am importing that uses find_library(<DLL name>). I can open a ticket with the module developers, but it did work previously when I first wrote the script, although I don't remember which Python version I had installed at the time (I will try older versions to see if I can find where it broke and update this ticket).

Expected Behavior

ctypes.util.find_library(<DLL name>) locates <DLL name>.dll in a directory added with os.add_dll_directory().

Example Script

Copy this script to a file in the same directory as vxlapi64.dll (Vector XL Driver DLL) and run it with python <script name>.py. I believe the issue is reproduceable with a different DLL in the same directory, but the script will need the DLL names updated. The DLL should also not be present in the system location, e.g. C:/Windows/System32.

import os
from ctypes.util import find_library

# Add the directory this script is in (also contains vxlapi64.dll)
add_obj = os.add_dll_directory(os.path.dirname(__file__))

# Check for library using DLL name
if find_library("vxlapi64"):
    print("vxlapi64.dll from add_dll_directory found")
else:
    print("vxlapi64.dll from add_dll_directory NOT FOUND")

# Check for library using DLL path + name
if find_library(os.path.dirname(__file__) + "/vxlapi64"):
    print("vxlapi64.dll from absolute path found")
else:
    print("vxlapi64.dll from absolute path NOT FOUND")

Output:

vxlapi64.dll from add_dll_directory NOT FOUND
vxlapi64.dll from absolute path found

Is there another way to add a DLL so an imported module can locate it with find_library(<DLL name>)?

CPython versions tested on:

3.12

Operating systems tested on:

Windows

[davidva-ecco]

From #111104 , find_library() checks each directory in os.environ['PATH'], so I was able to get find_library() to find vxlapi64.dll by placing it in a directory already in my OS environment PATH.

However, I don't understand why find_library() is using the environment PATH, since the porting notes about add_dll_directory() (https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew) state:

"DLL dependencies for extension modules and DLLs loaded with ctypes on Windows are now resolved more securely. Only the system paths, the directory containing the DLL or PYD file, and directories added with add_dll_directory() are searched for load-time dependencies. Specifically, PATH and the current working directory are no longer used, and modifications to these will no longer have any effect on normal DLL resolution."

Should find_library() be using something other than/in addition to os.environ['PATH'] when searching for DLLs?

How does Python know what "the directory containing the DLL or PYD file" is, if it hasn't been added with os.add_dll_directory()?

[zooba]

find_library was just never updated, though as it happens, I don't think Windows offers a way to query the DLL search path other than just loading the library. So it can't really be implemented properly anyway. I imagine the same is true for other platforms, and the best way to find a library is to either do a normal search (without relying on OS search algorithms) or to just load it (assuming it's loadable).

The behaviour isn't inconsistent with the what's new note you've quoted - using find_library to find a DLL file isn't the same as loading one, and certainly isn't the same as "load-time dependencies" or "normal DLL resolution".

Should find_library() be using something other than/in addition to os.environ['PATH'] when searching for DLLs?

I'm not sure it exists, so probably we should just add a warning that it can't use the correct search path on Windows and you may want to search for DLLs in another way.

How does Python know what "the directory containing the DLL or PYD file" is, if it hasn't been added with os.add_dll_directory()?

Because it's either searched using sys.path (for an extension module) or using the path provided by the user (when explicitly loading a DLL). Once Python has started loading the first one, it is entirely uninvolved in dependencies - the OS figures those out, which is where add_dll_directory comes into play.

[davidva-ecco]

Thanks for your reply, that makes sense. It sounds more like this needs to be fixed in the third-party module, so I'll close this and follow up with the module developers.

[zooba]

FWIW, adding the DLL path to environ['PATH'] as well as add_dll_directory is probably okay for a targeted case like this. There are some situations where DLL load won't look at PATH at all, but find_library still should.

And hopefully there's nothing else in the directory that would break a subprocess launch. Windows using the same variable for both global commands and DLL dependencies was clearly a mistake, but migrating to a fix (the separate, non-environment DLL path) takes a lot of time.