Skip to content

Implicit namespace __path__ not merged for multiple VCS/editable dependencies sharing the same namespace (develop=true broken, develop=flase works, pinned works) #10919

@Kowshik0932

Description

@Kowshik0932

Description

Summary

When multiple VCS/editable packages share the same implicit namespace package
(PEP 420), Poetry installs successfully without any errors. However, at
application startup, when Python begins resolving imports, only one
directory is registered in the namespace __path__. All other packages
contributing to the same namespace become invisible, resulting in
ModuleNotFoundError at runtime.

  • poetry installsucceeds with no errors
  • Application startup / import resolution → ModuleNotFoundError

This is broken with develop = true (editable/VCS installs).
This works correctly with pinned installs.

Affected versions: Poetry 1.8.2 with Python 3.10.9


Background — What Are Implicit Namespace Packages?

PEP 420 allows multiple separate repositories/distributions to contribute to the
same Python namespace without a central __init__.py.
Python resolves them by merging all contributing directories into the namespace's __path__.

Package repo Contributes to
myorg.service.one/myorg/service/one/ myorg namespace
myorg.service.two/myorg/service/two/ myorg namespace
myorg.base.storage/myorg/base/ myorg namespace
myorg.base.common/myorg/base/common/ myorg namespace

For this to work, myorg.__path__ must contain all four directories.
Poetry only registers one when packages are installed as editable.


Minimal Reproducible Example

Package Structure

myorg.service.one/
└── myorg/
    └── service/
        └── one/
            └── __init__.py

myorg.service.two/
└── myorg/
    └── service/
        └── two/
            └── __init__.py

myorg.base.storage/
└── myorg/
    └── base/
        └── storage/
            └── __init__.py    ← lives in myorg.base sub-namespace

myorg.base.common/
└── myorg/
    └── base/
        └── common/
            └── __init__.py    ← same sub-namespace, different repo

pyproject.toml

[tool.poetry]
name = "myorg-app"
version = "0.1.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "^3.10"
myorg-service-one  = { git = "https://github.com/org/myorg.service.one.git",  develop = true }
myorg-service-two  = { git = "https://github.com/org/myorg.service.two.git",  develop = true }
myorg-base-storage = { git = "https://github.com/org/myorg.base.storage.git", develop = true }
myorg-base-common  = { git = "https://github.com/org/myorg.base.common.git",  develop = true }

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Steps to Reproduce

# Step 1: Install — succeeds with NO errors
poetry install
#  Installing myorg-service-one
#  Installing myorg-service-two
#  Installing myorg-base-storage
#  Installing myorg-base-common

# Step 2: Start the application — fails at import time
python app.py
# OR
gunicorn myorg.service.one.app:create_app

# Step 3: Error appears during import resolution at startup:
# ModuleNotFoundError: No module named 'myorg.base.storage'

# Step 4: Diagnose — check what __path__ actually contains
python -c "import myorg; print(myorg.__path__)"
# Only ONE path instead of four 

# Step 5: Switch to pinned — works fine
# myorg-service-one  = "1.0.0"
# myorg-service-two  = "1.0.0"
# myorg-base-storage = "1.0.0"
# myorg-base-common  = "1.0.0"
poetry install
python app.py  #  starts correctly

Error at Application Startup

Traceback (most recent call last):
  File "app.py", line 3, in <module>
    from myorg.base.storage import StorageClient
  File ".venv/src/myorg.base.common/myorg/base/__init__.py"
ModuleNotFoundError: No module named 'myorg.base.storage'

The error does not appear during poetry install.
It only appears when Python resolves imports at application startup.


Expected Behaviour

All namespace paths should be merged — same as what pinned installs produce:

# myorg.__path__ should contain ALL contributing directories:
myorg.__path__ = [
    '.venv/src/myorg.service.one/myorg',
    '.venv/src/myorg.service.two/myorg',
    '.venv/src/myorg.base.storage/myorg',
    '.venv/src/myorg.base.common/myorg',
]

# Sub-namespace should also be fully merged:
myorg.base.__path__ = [
    '.venv/src/myorg.base.common/myorg/base',
    '.venv/src/myorg.base.storage/myorg/base',
]

# All imports should succeed:
import myorg.service.one
import myorg.service.two
import myorg.base.storage
import myorg.base.common

Actual Behaviour

poetry install succeeds, but at runtime only the last package
installed by Poetry wins the namespace:

# myorg.__path__ only has ONE directory:
myorg.__path__ = ['.venv/src/myorg.service.two/myorg']   # only ONE

# Sub-namespace only has ONE directory:
myorg.base.__path__ = ['.venv/src/myorg.base.common/myorg/base']  # missing storage

# Imports fail at application startup:
import myorg.service.one    # ModuleNotFoundError
import myorg.base.storage   # ModuleNotFoundError

Confirmed Behaviour Across Install Methods

Install method poetry install App startup imports __path__ merged
Pinned (= "1.0.0") Success Works All paths
Path editable (develop = true) Success ModuleNotFoundError Only one path
VCS editable (git + develop = true) Success ModuleNotFoundError Only one path

Key observation: poetry install always succeeds.
The failure only occurs at Python import resolution time during app startup.


Root Cause Analysis

Why install succeeds but runtime fails:

poetry install (develop = true)
    ↓
Poetry clones each repo into:
    .venv/src/myorg.service.one/
    .venv/src/myorg.service.two/
    .venv/src/myorg.base.storage/
    .venv/src/myorg.base.common/
    ↓
Each package gets its OWN isolated .pth file:
    myorg-service-one.pth  → .venv/src/myorg.service.one
    myorg-service-two.pth  → .venv/src/myorg.service.two
    myorg-base-storage.pth → .venv/src/myorg.base.storage
    myorg-base-common.pth  → .venv/src/myorg.base.common
    ↓
Install reports success — all packages are on disk
    ↓
Application starts → Python processes .pth files
    ↓
Python finds "myorg" namespace in FIRST matching .pth and STOPS
    ↓
Only ONE contributor registered in myorg.__path__
All others invisible at import time
    ↓
ModuleNotFoundError at app startup

How pinned installs avoid this (correct):

poetry install (pinned)
    ↓
pip installs into .venv/lib/python3.x/site-packages/
each package has proper .dist-info metadata
    ↓
pkg_resources / importlib.metadata reads ALL .dist-info at startup
    ↓
ALL namespace contributors merged into myorg.__path__ 
    ↓
All imports succeed at app startup 

Why this is a Poetry responsibility:

Poetry knows at install time that multiple packages share the same namespace.
It should generate a single combined .pth file that registers all
contributors together — similar to how zc.buildout (Plone/Zope) hardcodes
all namespace paths into the generated interpreter script:

# What zc.buildout generates (correct approach):
sys.path[0:0] = [
    '/apps/buildout/src/myorg.service.one',
    '/apps/buildout/src/myorg.service.two',
    '/apps/buildout/src/myorg.base.storage',
    '/apps/buildout/src/myorg.base.common',
]

Instead, Poetry generates isolated .pth files per package:

# What Poetry currently generates (broken at runtime):
myorg-service-one.pth  → .venv/src/myorg.service.one
myorg-service-two.pth  → .venv/src/myorg.service.two
myorg-base-storage.pth → .venv/src/myorg.base.storage
myorg-base-common.pth  → .venv/src/myorg.base.common

# Python picks first match → only one contributor registered 

Impact

This issue affects any project that:

  • Uses implicit namespace packages (PEP 420) — no __init__.py in namespace dir
  • Has multiple packages contributing to the same namespace
  • Installs those packages as editable (develop = true) or VCS dependencies
  • Common in monorepo setups, plugin architectures, and large enterprise
    projects split across many repositories under a shared namespace

Related Issues


Proposed Fix

Poetry should detect at install time when multiple editable/VCS packages share
the same namespace and generate a single combined .pth file that registers
all contributors:

# Proposed: one combined .pth instead of isolated ones
.venv/lib/python3.10/site-packages/myorg-namespace.pth:
    .venv/src/myorg.service.one
    .venv/src/myorg.service.two
    .venv/src/myorg.base.storage
    .venv/src/myorg.base.common

This would ensure Python sees all namespace contributors at interpreter startup,
before any import occurs — matching the behaviour of pinned installs and
eliminating the silent install-success / runtime-failure gap.

Workarounds

Manually patch all namespace __path__s at application startup
(in entry point):

import os
import importlib

VENV_SRC = os.path.join(os.path.dirname(__file__), ".venv", "src")

def patch_all_namespace_paths(src_root):
    namespace_paths = {}
    for pkg_dir in sorted(os.listdir(src_root)):
        pkg_root = os.path.join(src_root, pkg_dir)
        if not os.path.isdir(pkg_root):
            continue
        for dirpath, dirnames, _ in os.walk(pkg_root):
            dirnames[:] = [d for d in dirnames
                           if not d.startswith(('.', '__pycache__'))]
            rel = os.path.relpath(dirpath, pkg_root)
            if rel == '.':
                continue
            namespace_paths.setdefault(
                rel.replace(os.sep, '.'), []
            ).append(dirpath)

    def _patch(dotted, dirs):
        try:
            mod = importlib.import_module(dotted)
            if hasattr(mod, '__path__'):
                mod.__path__.extend(
                    d for d in dirs
                    if os.path.isdir(d) and d not in mod.__path__
                )
        except Exception:
            pass

    list(map(lambda item: _patch(*item), namespace_paths.items()))

patch_all_namespace_paths(VENV_SRC)

This should not be necessary — Poetry should handle namespace merging for
editable installs the same way it does for pinned installs.

Alternatively, switching all dependencies from develop = true to pinned
versions resolves the issue but removes the ability to develop locally.

Poetry Installation Method

install.python-poetry.org

Operating System

ubuntu 22

Poetry Version

1.8.2

Poetry Configuration

poetry config --list

 cache-dir = "/home/.cache/pypoetry"

 experimental.system-git-client = false

 installer.max-workers = null

 installer.modern-installation = true

 installer.no-binary = null

 installer.parallel = true

 keyring.enabled = true

 solver.lazy-wheel = true

 virtualenvs.create = true

 virtualenvs.in-project = true

 virtualenvs.options.always-copy = false

 virtualenvs.options.no-pip = false

 virtualenvs.options.no-setuptools = false

 virtualenvs.options.system-site-packages = false

 virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/.cache/pypoetry/virtualenvs

 virtualenvs.prefer-active-python = false

 virtualenvs.prompt = "{project_name}-py{python_version}"

 warnings.export = true

Python Sysconfig

sysconfig.log
Paste the output of 'python -m sysconfig', over this line.

Example pyproject.toml

[tool.poetry]
name = "myorg-app"
version = "0.1.0"
description = "Reproduces namespace package merging issue"
authors = ["your name <your@email.com>"]

[tool.poetry.dependencies]
python = "^3.10"

# Multiple packages sharing "myorg" namespace — all editable
myorg-service-one  = { git = "https://github.com/org/myorg.service.one.git",  develop = true }
myorg-service-two  = { git = "https://github.com/org/myorg.service.two.git",  develop = true }
myorg-base-storage = { git = "https://github.com/org/myorg.base.storage.git", develop = true }
myorg-base-common  = { git = "https://github.com/org/myorg.base.common.git",  develop = true }

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Poetry Runtime Logs

Note: I am unable to share actual company logs due to confidentiality.
Below is a sanitized reproduction of the error using generic package names.

Command

poetry run gunicorn myorg.service.one.wsgi:app -vvv

Error at Application Startup

Traceback (most recent call last):
  File ".venv/lib/python3.10/site-packages/gunicorn/util.py", line 371, in load_wsgi
    mod = import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, anchor)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File ".venv/src/myorg.service.one/myorg/service/one/app/__init__.py", line 3, in <module>
    from myorg.base.storage import StorageClient
  File ".venv/src/myorg.base.common/myorg/base/__init__.py"
ModuleNotFoundError: No module named 'myorg.base.storage'

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working as expectedstatus/triageThis issue needs to be triaged

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions