Pip installation and auto installed "glue" addons

[dreispt]

Addon modules feature "autoinstall" flag, that makes them be automatically installed once all their dependencies are installed.

For example, a "project_sale" glue module would be automatically installed after both "project" and "sale" have been installed.

But if we are using pip install to get modules (rather than git cloning a repo with many modules), "project_sale" won't be available in our env, and won't be auto-installed.

Is there a way for pip installation to support glue modules?
A "recommended" list?
Detecting them when packaging and having them as additional dependencies?

[sbidoul]

One way to handle this is to add them to the setup.py of the "main" module. For instance for account_invoice_merge, we could have such a setup.py:

import setuptools

setuptools.setup(
    setup_requires=['setuptools-odoo'],
    odoo_addon=True,
    install_requires=[
        'odoo8-addon-account_invoice_merge_payment',
        'odoo8-addon-account_invoice_merge_purchase',
    ],
)

[dreispt]

That's a new feature to develop, right?

[sbidoul]

No no, it's implemented. These setup.py are entirely normal so anything that is supported by setuptools can be added. Existing keys are preserved except keys of list type (such as install_requires) which are extended by setuptools-odoo.

setuptools-odoo-make-default that runs nightly generates the canonical setup.py, but it does not touch existing ones. So we can enrich them manually.

[dreispt]

My expectation idea was for the possible glue modules to be added as dependencies to the "main" modules automatically, rather than being done manually later.
That would mean additional logic in setuptools-odoo-make-default, if I understand correctly.

[sbidoul]

That would mean additional logic in setuptools-odoo-make-default, if I understand correctly.

Indeed. I've no idea how to do that though.

[dreispt]

I have an idea I can try.
I'm not familiar with this code base. Can you point me the packaging code that computes the requirements/dependency list?

[sbidoul]

You can start here.

How would you determine what the main addon is?

[sbidoul]

Also glue modules may be created after the main module' setup.py already exists... so definitely not easy to automate.

[dreispt]

Thanks for the pointer.
Where exactly do you generate the setup.py for each addon module, including the install_requires key for each one?

[dreispt]

PS: found it!

[sbidoul]

@dreispt the default setup.py is generated by setuptools-odoo-make-default, using this template.

The install_requires list is populated from the addon manifest and external python dependencies when setup.py is executed with the code you refer to.

What you want to do is adapt setuptools-odoo-make-default. But I still really doubt its worth the effort as it can't be very reliable, and will always require manual tweaking. For instance auto_install modules could conceivably be in different repos so there is hardly a way to discover them automatically in all cases.

[dreispt]

For instance auto_install modules could conceivably be in different repos so there is hardly a way to discover them automatically in all cases.

That is a corner case we can live without.
The recommended way to handle glue modules is to have them bundled in the same repo as the "main" module. This is the case for the OCA repos.

In a nutshell, the idea is: when packaging an addon, scan the repo dir for auto_install modules that have this as a dependency, and have them added to the install_requires. (In a naive implementation the scan step would be run N^2 times, but IMO that won't be a real problem and could be optimized later.)

I'm not yet sure about the wokflow of the packaging script. I get the is that it first prepares the setup/ dir, and then iterates through each subdir to package that module.
The naive unoptimized implementation is to have this subdir packaging action to scan the top dir (setup/?) for other auto_install modules that depend on it.

[sbidoul]

In OCA, the nightly packaging script does this:

• oca-clone-everything
• for each branch (8.0, 9.0, 10.0) and each repo:
• run setuptools-odoo-make-default to generate the missing setup.py if any, and git commit new ones
• for each addon in setup/*, run python setup.py bdist_wheel and store the result in https://wheelhouse.odoo-community.org/oca if it does not exist yet
• update symlinks in https://wheelhouse.odoo-community.org/oca-simple
• upload https://wheelhouse.odoo-community.org/oca/odoo*-addon-*.whl to pypi