`sysconfig.get_path("purelib")` returns `site-packages` folder instead of `app_packages`

[timrid]

Describe the bug

sysconfig.get_path("purelib") returns site-packages folder instead of app_packages

Some libraries access sysconfig.get_path("purelib") to get the path of the folder, where the 3rd party libraries are installed. E.g. pydevd (see here) or debugpy (see here)

Currently sysconfig.get_paths() returns on Windows

{
    'stdlib': '...\build\helloworld\windows\app\src\Lib',
    'platstdlib': '...\build\helloworld\windows\app\src\Lib',
    'purelib': '...\build\helloworld\windows\app\src\Lib\site-packages',
    'platlib': '...\build\helloworld\windows\app\src\Lib\site-packages',
    'include': '...\build\helloworld\windows\app\src\Include',
    'platinclude': '...\build\helloworld\windows\app\src\Include',
    'scripts': '...\build\helloworld\windows\app\src\Scripts',
    'data': '...\build\helloworld\windows\app\src'
}

But ...\build\helloworld\windows\app\src\Lib\site-packages does not exists, because all libraries are inside ...\build\helloworld\windows\app\src\app_packages.

Steps to reproduce

1. Create an example briefcase app
2. Add

import sysconfig
print(sysconfig.get_path("purelib"))

3. Run briefcase run

Expected behavior

sysconfig.get_path("purelib") should return the path to the app_packages folder instead of the non existing site-packages folder.

Screenshots

No response

Environment

• Operating System: Windows 11
• Python version: 3.13
• Software versions:
  • Briefcase: 0.3.24

Logs

Additional context

There has already been discussion about whether it makes sense to use the standard site-packages folder instead of a custom app_packages folder, and why it was rejected: #2195

I created a working sysconfig patch function that can may be used:

Details

def _patch_sysconfig():
    import sysconfig

    BRIEFCASE_SCHEME = "briefcase"

    sysconfig._INSTALL_SCHEMES[BRIEFCASE_SCHEME] = {
        "stdlib": "{installed_base}/Lib",
        "platstdlib": "{base}/Lib",
        "purelib": "{base}/app_packages",
        "platlib": "{base}/app_packages",
        "include": "{installed_base}/Include",
        "platinclude": "{installed_base}/Include",
        "scripts": "{base}/Scripts",
        "data": "{base}",
    }

    __orig_get_paths = sysconfig.get_paths
    __orig_get_path = sysconfig.get_path

    def get_default_scheme():
        return BRIEFCASE_SCHEME

    def get_paths(*args, **kwargs):
        if len(args) >= 1:
            new_args = list(args)
            if new_args[0] is None:
                new_args[0] = BRIEFCASE_SCHEME
            args = tuple(new_args)
        elif "scheme" not in kwargs or kwargs.get("scheme") is None:
            kwargs["scheme"] = BRIEFCASE_SCHEME
        return __orig_get_paths(*args, **kwargs)

    def get_path(*args, **kwargs):
        if len(args) >= 2:
            new_args = list(args)
            if new_args[1] is None:
                new_args[1] = BRIEFCASE_SCHEME
            args = tuple(new_args)
        elif "scheme" not in kwargs or kwargs.get("scheme") is None:
            kwargs["scheme"] = BRIEFCASE_SCHEME
        return __orig_get_path(*args, **kwargs)

    sysconfig.get_default_scheme = get_default_scheme
    sysconfig.get_paths = get_paths
    sysconfig.get_path = get_path

[freakboy3742]

Well this is definitely an interesting can of worms... :-)

It's interesting to me that an embedded Windows CPython interpreter generates a purelib and platlib definition that isn't correct.

Regardless - this looks like something that we need to have an answer for. Your proposed patch would be one way to address the issue; however, I wonder if it wouldn't be a lot simpler to just inject sysconfig._INSTALL_SCHEMES[sysconfig.get_default_scheme()] = BRIEFCASE_SCHEME, rather than try to install an entirely new scheme, and override all the mechanisms that need to use that scheme by default.

That said - the Windows case specifically has also come up in the context of #2430. In that context, it looks like it might actually be preferable for us to switch to using Lib/site-packages simply because of baked-in expectations of the broader Python ecosystem. That would remove the need for an override altogether for Windows.

A similar argument could be made for Linux. On macOS, the bigger question becomes what sys.prefix should be, and whether that introduces the need for a different scheme for "embedded framework" apps. A similar argument can probably be made for iOS as well.

[kengoon]

@freakboy3742 is it official moving to Lib/**, I want to know if should just focus on Kivy bootstrap and discard the new changes I'm working on to support kivy share/libs

[freakboy3742]

@freakboy3742 is it official moving to Lib/**, I want to know if should just focus on Kivy bootstrap and discard the new changes I'm working on to support kivy share/libs

We haven't made an official decision on this yet.