Skip to content

Commit d4a4cee

Browse files
authored
Add PyPI alternatives for local dependencies in requirements.yaml (#252)
1 parent cddf104 commit d4a4cee

File tree

12 files changed

+1897
-51
lines changed

12 files changed

+1897
-51
lines changed

README.md

Lines changed: 73 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -54,8 +54,9 @@ Try it now and streamline your development process!
5454
- [Implementation](#implementation)
5555
- [`[project.dependencies]` in `pyproject.toml` handling](#projectdependencies-in-pyprojecttoml-handling)
5656
- [:jigsaw: Build System Integration](#jigsaw-build-system-integration)
57-
- [Environment Variables](#environment-variables)
58-
- [`UNIDEP_SKIP_LOCAL_DEPS`](#unidep_skip_local_deps)
57+
- [Local Dependencies in Monorepos](#local-dependencies-in-monorepos)
58+
- [PyPI Alternatives for Local Dependencies](#pypi-alternatives-for-local-dependencies)
59+
- [Build System Behavior](#build-system-behavior)
5960
- [Example packages](#example-packages)
6061
- [Setuptools Integration](#setuptools-integration)
6162
- [Hatchling Integration](#hatchling-integration)
@@ -239,7 +240,7 @@ See [Build System Integration](#jigsaw-build-system-integration) for more inform
239240
- Use `pip:` to specify packages that are only available through Pip.
240241
- Use `conda:` to specify packages that are only available through Conda.
241242
- Use `# [selector]` (YAML only) or `package:selector` to specify platform-specific dependencies.
242-
- Use `local_dependencies:` to include other `requirements.yaml` or `pyproject.toml` files and merge them into one. Also allows projects that are not managed by `unidep` to be included, but be aware that this skips their dependencies!
243+
- Use `local_dependencies:` to include other `requirements.yaml` or `pyproject.toml` files and merge them into one. Also allows projects that are not managed by `unidep` to be included, but be aware that this skips their dependencies! Can specify PyPI alternatives for monorepo setups (see [PyPI Alternatives for Local Dependencies](#pypi-alternatives-for-local-dependencies)).
243244
- Use `optional_dependencies:` to specify optional dependencies. Can be installed like `unidep install ".[test]"` or `pip install ".[test]"`.
244245
- Use `platforms:` to specify the platforms that are supported. If omitted, all platforms are assumed to be supported.
245246

@@ -392,36 +393,85 @@ dependencies = [
392393
393394
`unidep` seamlessly integrates with popular Python build systems to simplify dependency management in your projects.
394395

395-
### Environment Variables
396+
### Local Dependencies in Monorepos
396397

397-
#### `UNIDEP_SKIP_LOCAL_DEPS`
398+
Local dependencies are essential for monorepos and multi-package projects, allowing you to:
399+
- Share code between packages during development
400+
- Maintain separate releases for each package
401+
- Test changes across multiple packages simultaneously
398402

399-
Local dependencies are useful for monorepos, shared configuration files, and local development workflows where you're developing multiple related packages.
400-
However, when building wheels for distribution (e.g., for PyPI), including local dependencies can create hardcoded `file://` paths in the wheel metadata, making wheels non-portable.
403+
However, when building wheels for distribution, local paths create non-portable packages that only work on the original system.
401404

402-
**Build backend behavior differs:**
403-
- **Setuptools**: Automatically filters out invalid `file://` URLs during wheel building (this environment variable not needed)
404-
- **Hatchling**: Includes all dependencies as specified, requiring explicit filtering for distribution
405+
### PyPI Alternatives for Local Dependencies
405406

406-
Set this environment variable to skip including local dependencies as `file://` URLs when building distributable artifacts:
407+
UniDep solves this problem by letting you specify both local paths (for development) and PyPI packages (for distribution):
408+
409+
```yaml
410+
# requirements.yaml
411+
dependencies:
412+
- numpy
413+
- pandas
414+
415+
local_dependencies:
416+
# Standard string format for local dependencies
417+
- ../shared-lib
418+
419+
# Dictionary format with optional PyPI alternative for build-time
420+
- local: ../auth-lib
421+
pypi: company-auth-lib>=1.0
422+
423+
- local: ../utils
424+
pypi: company-utils~=2.0
425+
```
426+
427+
Or in `pyproject.toml`:
428+
429+
```toml
430+
[tool.unidep]
431+
dependencies = ["numpy", "pandas"]
432+
433+
local_dependencies = [
434+
# Standard string format for local dependencies
435+
"../shared-lib",
436+
437+
# Dictionary format with optional PyPI alternative for build-time
438+
{local = "../auth-lib", pypi = "company-auth-lib>=1.0"},
439+
{local = "../utils", pypi = "company-utils~=2.0"},
440+
]
441+
```
442+
443+
**How it works:**
444+
- **During development** (e.g., `unidep install` or `pip install -e .`): Uses local paths when they exist
445+
- **When building wheels**: PyPI alternatives (if specified) are used to create portable packages
446+
- The standard string format continues to work as always for local dependencies
447+
448+
> [!TIP]
449+
> PyPI alternatives ensure your wheels are portable and can be installed anywhere, not just on the build system.
450+
451+
### Build System Behavior
452+
453+
**Important differences between build backends:**
454+
- **Setuptools**: Builds wheels containing `file://` URLs with absolute paths. These wheels only work on the original system.
455+
- **Hatchling**: Rejects `file://` URLs by default, preventing non-portable wheels.
456+
457+
To ensure portable wheels, you can use the `UNIDEP_SKIP_LOCAL_DEPS` environment variable:
407458

408459
```bash
460+
# Force use of PyPI alternatives even when local paths exist
461+
UNIDEP_SKIP_LOCAL_DEPS=1 python -m build
462+
409463
# For hatch projects
410464
UNIDEP_SKIP_LOCAL_DEPS=1 hatch build
411465
412466
# For uv build
413467
UNIDEP_SKIP_LOCAL_DEPS=1 uv build
414-
415-
# For python -m build
416-
UNIDEP_SKIP_LOCAL_DEPS=1 python -m build
417468
```
418469

419470
> [!NOTE]
420-
> When this variable is set, local dependencies are skipped but their actual dependencies (extracted from `requirements.yaml` or `pyproject.toml`) are still included in the built wheel. This ensures the wheel remains functional while avoiding non-portable absolute paths.
421-
>
422-
> **Backend-specific workaround**: This environment variable is primarily needed for Hatchling-based projects. Setuptools automatically handles this filtering, making wheels portable by default.
423-
>
424-
> **Two contexts, same codebase**: Local dependencies are included by default (great for `unidep install` during development), but this environment variable provides the context switch needed when building for distribution.
471+
> **When `UNIDEP_SKIP_LOCAL_DEPS=1` is set:**
472+
> - Local dependencies with PyPI alternatives → use PyPI package
473+
> - Local dependencies without PyPI alternatives → skipped entirely
474+
> - Dependencies from local packages are still included (from their `requirements.yaml`/`pyproject.toml`)
425475

426476
### Example packages
427477

@@ -468,15 +518,14 @@ build-backend = "hatchling.build"
468518
dynamic = ["dependencies"]
469519
# Additional project configurations
470520
471-
[tool.hatch]
472-
# Additional Hatch configurations
521+
[tool.hatch.metadata.hooks.unidep]
522+
# Enable the unidep plugin
473523
474-
# Allow VCS URLs, local paths, and other direct references in dependencies
475524
[tool.hatch.metadata]
476525
allow-direct-references = true
477526
478-
# Register UniDep as a metadata hook to process dependencies
479-
[tool.hatch.metadata.hooks.unidep]
527+
[tool.unidep]
528+
# Your dependencies configuration
480529
```
481530

482531
## :desktop_computer: As a CLI

0 commit comments

Comments
 (0)