Skip to content

Commit 8bc9851

Browse files
sigmasigma
committed
docs(python): document uv workspace pattern and turnkey.* namespace
Update CLAUDE.md 'Monorepo Dependency Management → Python' section to reflect the new workspace model (single uv.lock, per-member pyproject, turnkey.* PEP 420 namespace) and link to the new workflow doc. Add docs/user-manual/src/workflows/python-workspace.md covering: - Why uv workspaces (parallel to the Cargo workspace pattern) - The turnkey.<name> namespace convention and how downstream monorepos pick their own (e.g., acme.<name>) - hatchling [tool.hatch.build.targets.wheel] packages = ['turnkey'] plus a turnkey/<name>/ subdirectory per member - How uv export → pydeps-gen → pydeps cell composes with the Buck2 track, and the manual 'uv lock && uv export' step before tk sync - Cross-member deps via [tool.uv.sources] { workspace = true } - Non-packaged members (examples) via [tool.uv] package = false Link the new page from docs/user-manual/src/SUMMARY.md. Refs turnkey-991.5
1 parent 1bda5b3 commit 8bc9851

4 files changed

Lines changed: 243 additions & 10 deletions

File tree

‎CLAUDE.md‎

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -260,19 +260,26 @@ my-dep.workspace = true # ✅ Correct
260260

261261
### Python
262262

263-
**Root file**: `pyproject.toml` at repo root
263+
**Layout**: uv workspace. The root `pyproject.toml` aggregates members; each Python package owns its own `pyproject.toml` with its own dependencies. A single `uv.lock` at the root resolves everything together.
264264

265265
```
266266
/turnkey/
267-
├── pyproject.toml # [project.dependencies] declares all deps
268-
├── uv.lock # Lockfile (managed by uv)
269-
├── python-deps.toml # Generated for Nix/Buck2
270-
└── python/mypackage/ # NO pyproject.toml here
271-
```
272-
273-
- All Python code uses the root pyproject.toml
274-
- Use `uv add` from repo root to add dependencies
275-
- Sub-packages are part of the root project
267+
├── pyproject.toml # [tool.uv.workspace] + members as deps
268+
├── uv.lock # Single resolved lockfile
269+
├── pylock.toml # PEP 751 export from uv.lock
270+
├── python-deps.toml # Generated for Nix/Buck2 from pylock.toml
271+
├── src/python/<member>/
272+
│ ├── pyproject.toml # Real package, hatchling backend
273+
│ └── turnkey/<member>/ # Source under shared turnkey.* namespace
274+
└── src/examples/python-<name>/
275+
└── pyproject.toml # Non-packaged ([tool.uv] package = false)
276+
```
277+
278+
- All Python source lives under the shared `turnkey.*` PEP 420 namespace package. Members never define a `turnkey/__init__.py`.
279+
- Cross-member deps are declared with `[tool.uv.sources]` workspace markers, mirroring `Cargo.toml`'s `workspace = true` pattern.
280+
- Externals are declared in the member that consumes them. The lockfile reconciles versions across the workspace.
281+
- Adding/removing deps: edit the member's `pyproject.toml`, then `uv lock && uv export --all-packages --format pylock.toml -o pylock.toml`. `tk sync` regenerates `python-deps.toml` from `pylock.toml`.
282+
- Downstream monorepos that adopt this framework pick their own namespace (e.g. `acme.<name>`); see `docs/user-manual/src/workflows/python-workspace.md`.
276283

277284
### TypeScript/JavaScript
278285

‎docs/user-manual/src/SUMMARY.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@
2828
- [Building Projects](./workflows/building.md)
2929
- [Running Tests](./workflows/testing.md)
3030
- [Managing Dependencies](./workflows/dependencies.md)
31+
- [Python Workspaces](./workflows/python-workspace.md)
3132

3233
# Language Support
3334

‎docs/user-manual/src/languages/python.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
Turnkey provides Python support with Buck2 integration.
44

5+
> Python source in this repo is laid out as a [uv workspace](../workflows/python-workspace.md), with each package owning its own `pyproject.toml` and contributing to a shared `turnkey.*` PEP 420 namespace. This page covers the Buck2 build rules; read the workspace workflow guide first for the overall layout and the uv/Buck2 dual-track model.
6+
57
## Setup
68

79
Add to `toolchain.toml`:

‎docs/user-manual/src/workflows/python-workspace.md‎

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
# Python Workspaces
2+
3+
Turnkey lays out Python source as a [uv workspace][uv-workspaces], in parallel to the Cargo workspace pattern used for Rust. A single `uv.lock` resolves every Python package in the monorepo against a consistent dependency set, while each package keeps its own `pyproject.toml` declaring exactly what it consumes.
4+
5+
Two tracks run side by side over the same source:
6+
7+
- **uv track** — `uv sync`, `uv run`, IDE language servers, REPL. Members are installed editable so source edits are reflected immediately.
8+
- **Buck2 track** — `tk build`, `tk test`. External packages are vendored into the `pydeps` cell built from `python-deps.toml`.
9+
10+
[uv-workspaces]: https://docs.astral.sh/uv/concepts/projects/workspaces/
11+
12+
## Repository Layout
13+
14+
```
15+
/repo/
16+
├── pyproject.toml # Workspace root: members + uv.lock anchor
17+
├── uv.lock # Single resolved lockfile (managed by uv)
18+
├── pylock.toml # PEP 751 export from uv.lock
19+
├── python-deps.toml # Generated for Buck2/Nix from pylock.toml
20+
└── src/python/<member>/
21+
├── pyproject.toml # [project] + hatchling build backend
22+
├── rules.star # Buck2 targets for the member
23+
└── turnkey/<member>/ # Source under shared turnkey.* namespace
24+
├── __init__.py
25+
└── ...
26+
```
27+
28+
Tests live in a sibling `tests/` directory inside each member, kept outside the importable namespace.
29+
30+
## The `turnkey.*` Namespace Convention
31+
32+
Every workspace member contributes a subpackage under the shared `turnkey` [PEP 420 implicit namespace package][pep-420]. No member defines a top-level `turnkey/__init__.py`; Python's import system resolves `turnkey.cargo`, `turnkey.cfg`, etc. by walking every `sys.path` entry that exposes a `turnkey/<name>/` directory.
33+
34+
[pep-420]: https://peps.python.org/pep-0420/
35+
36+
### Downstream Projects: Pick Your Own Namespace
37+
38+
The `turnkey.*` prefix is **this repository's** namespace. If you adopt the same workspace pattern in a different monorepo, choose a namespace specific to your organisation — e.g. `acme.<name>` — to avoid colliding with packages on PyPI or other turnkey-based repos. The mechanics are identical; substitute `turnkey` for your namespace throughout this guide.
39+
40+
## Member `pyproject.toml`
41+
42+
Each library member uses the [hatchling][hatchling] backend and points it at the `turnkey/` directory:
43+
44+
```toml
45+
[build-system]
46+
requires = ["hatchling"]
47+
build-backend = "hatchling.build"
48+
49+
[project]
50+
name = "turnkey-cargo"
51+
version = "0.1.0"
52+
description = "Cargo manifest and feature-graph utilities"
53+
requires-python = ">=3.11"
54+
dependencies = [
55+
"turnkey-cfg", # cross-member dep
56+
]
57+
58+
[tool.uv.sources]
59+
turnkey-cfg = { workspace = true }
60+
61+
[tool.hatch.build.targets.wheel]
62+
packages = ["turnkey"] # everything under turnkey/<name>/ is the wheel content
63+
```
64+
65+
`packages = ["turnkey"]` is the key line: it tells hatchling that the wheel's content is whatever lives under the `turnkey/` directory of this member. Combined with PEP 420 namespace resolution, every member ships only its own `turnkey/<name>/` slice without anyone owning `turnkey/__init__.py`.
66+
67+
[hatchling]: https://hatch.pypa.io/
68+
69+
### Cross-Member Dependencies
70+
71+
Declare the dep under `[project] dependencies` with the bare package name, then pin its source to the workspace under `[tool.uv.sources]`:
72+
73+
```toml
74+
dependencies = ["turnkey-cfg"]
75+
76+
[tool.uv.sources]
77+
turnkey-cfg = { workspace = true }
78+
```
79+
80+
This mirrors `Cargo.toml`'s `serde.workspace = true` pattern — the consumer member doesn't pin a version, the lockfile reconciles it.
81+
82+
### External Dependencies
83+
84+
Declare externals in the member that consumes them, never the workspace root:
85+
86+
```toml
87+
# src/examples/python-hello-deps/pyproject.toml
88+
[project]
89+
name = "turnkey-example-python-hello-deps"
90+
dependencies = ["six>=1.16.0"]
91+
```
92+
93+
The single `uv.lock` at the workspace root resolves every external version-consistently across members.
94+
95+
### Non-Packaged Members
96+
97+
Some members exist only to declare dependencies, not to be installed (typical for application-like entrypoints or examples). Mark them non-packaged:
98+
99+
```toml
100+
[project]
101+
name = "turnkey-example-python-hello-deps"
102+
version = "0.1.0"
103+
dependencies = ["six>=1.16.0"]
104+
105+
[tool.uv]
106+
package = false # uv won't build/install this member
107+
```
108+
109+
No `[build-system]` is required. uv still resolves the member's dependencies as part of the workspace lock.
110+
111+
## Root `pyproject.toml`
112+
113+
The workspace root anchors membership and the shared lockfile:
114+
115+
```toml
116+
[project]
117+
name = "turnkey"
118+
version = "0.1.0"
119+
requires-python = ">=3.11"
120+
121+
# Listing members as dependencies makes the default `uv sync` install all
122+
# of them in one shot — no `--all-packages` flag needed.
123+
dependencies = [
124+
"turnkey-buck",
125+
"turnkey-buildsystem",
126+
"turnkey-cargo",
127+
"turnkey-cfg",
128+
"turnkey-example-python-hello",
129+
"turnkey-example-python-hello-deps",
130+
]
131+
132+
[project.optional-dependencies]
133+
dev = ["pytest>=7.0"]
134+
135+
[tool.uv.workspace]
136+
members = [
137+
"src/python/cargo",
138+
"src/python/buck",
139+
"src/python/buildsystem",
140+
"src/python/cfg",
141+
"src/examples/python-hello",
142+
"src/examples/python-hello-deps",
143+
]
144+
145+
[tool.uv.sources]
146+
turnkey-buck = { workspace = true }
147+
turnkey-buildsystem = { workspace = true }
148+
turnkey-cargo = { workspace = true }
149+
turnkey-cfg = { workspace = true }
150+
turnkey-example-python-hello = { workspace = true }
151+
turnkey-example-python-hello-deps = { workspace = true }
152+
153+
[tool.uv]
154+
package = false # the root itself isn't a packaged project
155+
```
156+
157+
## Buck2 Integration
158+
159+
Member source paths are spelled relative to the member's `rules.star`:
160+
161+
```python
162+
load("@prelude//:rules.bzl", "python_library", "python_test")
163+
164+
python_library(
165+
name = "cargo",
166+
srcs = [
167+
"turnkey/cargo/__init__.py",
168+
"turnkey/cargo/features.py",
169+
"turnkey/cargo/toml.py",
170+
],
171+
base_module = "",
172+
deps = ["//src/python/cfg:cfg"],
173+
visibility = ["PUBLIC"],
174+
)
175+
176+
python_test(
177+
name = "test_toml",
178+
srcs = ["tests/test_toml.py"],
179+
base_module = "tests",
180+
deps = [":cargo"],
181+
)
182+
```
183+
184+
`base_module = ""` tells Buck2 to install sources at their declared `srcs` paths, so files land at `turnkey/cargo/...` in the runtime tree — matching the import prefix the rest of the codebase uses.
185+
186+
## Adding or Updating Dependencies
187+
188+
```bash
189+
# 1. Edit the member that needs the dep
190+
$EDITOR src/python/cargo/pyproject.toml # add to [project] dependencies
191+
192+
# 2. Regenerate the lock
193+
uv lock
194+
195+
# 3. Refresh editable installs (optional but recommended)
196+
uv sync --extra dev
197+
198+
# 4. Export to PEP 751 lock for the Buck2 pipeline
199+
uv export --all-packages --format pylock.toml -o pylock.toml
200+
201+
# 5. Refresh python-deps.toml for the pydeps cell
202+
# (tk sync picks this up automatically when pylock.toml is newer)
203+
tk sync
204+
```
205+
206+
Steps 2–4 are manual today; future work can fold them into `tk sync` as a pre-step.
207+
208+
## Running Code
209+
210+
| Task | uv track | Buck2 track |
211+
| ------------------------------- | --------------------------------------- | ------------------------------------------------- |
212+
| Run all tests | `uv run pytest` | `tk test //src/python/...` |
213+
| Run a single member's tests | `uv run pytest src/python/cargo` | `tk test //src/python/cargo:test_toml` |
214+
| Run an example | `uv run --package <pkg-name> <script>` | `tk run //src/examples/python-hello-deps:python-hello-deps` |
215+
| REPL with members available | `uv run python` | n/a |
216+
| IDE language server | Point at `.venv/bin/python` | n/a |
217+
218+
Both tracks resolve external dependencies the same way (uv.lock is the single source of truth), but the install paths differ: the uv track installs into `.venv/`, the Buck2 track materialises external packages into `.turnkey/pydeps/vendor/<name>/`.
219+
220+
## See Also
221+
222+
- [Managing Dependencies](./dependencies.md) — overall dependency flow across all languages.
223+
- [Python](../languages/python.md) — Buck2 build rules for Python targets.

0 commit comments

Comments
 (0)