Building a .whl with package discovery find_packages()

[jmachcse]

Hello all -
I've got a question regarding setuptools's ability to find packages automatically using find_packages().

My file structure is thus:

mypkg/
    main.py
    src/
        pkg1/__init__.py
        pkg2/__init__.py
        ...
        pkg9/__init__.py
    scripts/
        __init__.py
    bin/
        __init__.py

The command I run to generate my .whl is python setup.py bdist_wheel --universal
My console output follows:

(testenv) C:\wheeltest>python setup.py bdist_wheel --universal
running bdist_wheel
running build
running build_py
error: package directory 'mypkg\pkg1' does not exist

My setup.py follows:

from setuptools import setup, find_packages

setup(
    name="testwheel",
    version="1.0",
    description="1.0 first wheel",
    package_dir={"": "mypkg"},
    packages=find_packages(
        where="mypkg",
        include=["src.*"],
        exclude=[],
    ),
)

It might be a poor understanding of what a wheel actually is and what files I need/want to include but - I would be led to believe that I want all of my packages included so I can have a fully constructed .whl. How can I change my setup.py to include all of the packages within my src folder? And furthermore, how do I include main.py which is crucial to running my program?

Thanks in advance for any help!

[webknjaz]

What sort of layout do you need? You could use py_modules=['main'] to get that main.py installed right in the root of site-packages/. But This seems wrong. I'd recommend putting it into the package (a folder with __init__.py)

Also, where="mypkg", seems weird. It's usually src/ but then, this dir usually lives in the project root. What are you trying to achieve with this layout?

FWIW try going through https://packaging.python.org/tutorials/packaging-projects/.

[jmachcse]

Thanks for the reply!
Ultimately, I'm looking to turn the entire project into a .whl either for distribution or for internal dev use. It looks like I've figured it out.
Correct me if I'm wrong on any of this please:
I made my .whl, installed it into a test environment, and I am able to import my pkgs i.e. import pkg1 as A and then use A as an individual module like print(A.helloWorld)

I was hoping I would be able to import things like from mypkg import pkg1 as A but it appears I can't do that. In fact, I can't seem to import my entire package i.e. import mypkg or I suppose in the case I posed in my original question, import testwheel. Is there a way I should be able to do this, or is my wheel / package functioning as intended?

[webknjaz]

Okay. If you want mypkg, you should restructure your project.
First, have setup.py / setup.cfg at the top-level. Also, have src/ on the top-level too. But inside, you'd have mypkg with pkg1 / pkg2 etc.

It'd look like this:

my-pkg-project/  # your project root
    src/
        main.py
        mypkg/
            __init__.py  # this makes `mypkg` a package in terms of Python's import system; technically, it's not necessary under Python 3 but may be useful. If you don't want it, you'll need to change `find_packages` to `find_namespace_packages` because the former doesn't know how to detect implicit packages w/o init
            pkg1/__init__.py
            pkg2/__init__.py
            ...
            pkg9/__init__.py
    scripts/  # these are not going to be accessible under `mypkg`
        __init__.py
    bin/  # these are not going to be accessible under `mypkg`
        __init__.py

Then, make sure to have where="src" and then, it'll locate any folders that are recognized as packages inside of "src" but not "src" itself.
For "main.py" you'd probably need to use py_modules, it won't be a part of mypkg with this layout. I don't know what it does for you but if it's just an entry point CLI script, you should probably rename it to __main__.py and put under mypkg: this way, you'll make it runnable via python -m mypkg.
I didn't include scripts and bin in the example layout because I don't know what is their role in your specific project and what you want from them. I suppose they may be helper scripts to maintain the project/repo and maybe build artifacts that you don't intend to include in the installable Python package distribution. But if not, you'll need to move them according to your needs.

P.S. By the way, it's much cleaner to use setup.cfg instead of setup.py — https://setuptools.readthedocs.io/en/latest/userguide/package_discovery.html

[jmachcse]

Thank you for the followup!
Yes, it appears that your solution has worked for my purposes. You're right, scripts/bin both just contain helper scripts. Not needed for what I want to distribute.
Thank you for your help, I really appreciate it!