Skip to content

Commit 6af5253

Browse files
MattiaSartiMattiaSarti
authored
build: migrate to poetry for Python dependency management (#269)
* docs: add instructions for dependency management * ci: update checkout actions, tox installation and TIOBE schedule * build: migrate to poetry for dependency management in charm 'argo_controller' * build: migrate to poetry for dependency management in base project folder
1 parent 7f02603 commit 6af5253

22 files changed

+3266
-914
lines changed

‎.github/workflows/integrate.yaml‎

Lines changed: 5 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -20,25 +20,17 @@ jobs:
2020
runs-on: ubuntu-24.04
2121
steps:
2222
- name: Check out repo
23-
uses: actions/checkout@v3
24-
- name: Set up Python 3.12
25-
uses: actions/setup-python@v5.6.0
26-
with:
27-
python-version: "3.12"
28-
- run: pip install tox
23+
uses: actions/checkout@v4
24+
- run: pipx install tox
2925
- run: tox -vve argo-controller-lint
3026

3127
unit:
3228
name: Unit Tests
3329
runs-on: ubuntu-24.04
3430
steps:
3531
- name: Check out repo
36-
uses: actions/checkout@v3
37-
- name: Set up Python 3.12
38-
uses: actions/setup-python@v5.6.0
39-
with:
40-
python-version: "3.12"
41-
- run: pip install tox
32+
uses: actions/checkout@v4
33+
- run: pipx install tox
4234
- run: tox -vve argo-controller-unit
4335

4436
terraform-checks:
@@ -54,7 +46,7 @@ jobs:
5446
matrix:
5547
charm: [argo-controller]
5648
steps:
57-
- uses: actions/checkout@v3
49+
- uses: actions/checkout@v4
5850
- name: Setup operator environment
5951
uses: charmed-kubernetes/actions-operator@main
6052
with:

‎.github/workflows/publish.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ jobs:
3232
outputs:
3333
charm_paths_list: ${{ steps.get-charm-paths.outputs.CHARM_PATHS_LIST }}
3434
steps:
35-
- uses: actions/checkout@v3
35+
- uses: actions/checkout@v4
3636
with:
3737
fetch-depth: 0
3838
ref: ${{ inputs.source_branch }}
@@ -52,7 +52,7 @@ jobs:
5252

5353
steps:
5454
- name: Checkout
55-
uses: actions/checkout@v3
55+
uses: actions/checkout@v4
5656
with:
5757
fetch-depth: 0
5858
ref: ${{ inputs.source_branch }}

‎.github/workflows/release.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919
name: Promote charm
2020
runs-on: ubuntu-24.04
2121
steps:
22-
- uses: actions/checkout@v3
22+
- uses: actions/checkout@v4
2323
- name: Release charm to channel
2424
uses: canonical/charming-actions/release-charm@2.6.2
2525
with:

‎.github/workflows/tiobe_scan.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,4 @@ jobs:
4242
branchdir: .
4343
viewerUrl: https://canonical.tiobe.com/tiobeweb/TICS/api/cfg?name=default
4444
ticsAuthToken: ${{ secrets.TICSAUTHTOKEN }}
45-
installTics: true
45+
installTics: true

‎CONTRIBUTING.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
## How to Manage Python Dependencies and Environments
2+
3+
4+
### Prerequisites
5+
6+
`tox` is the only tool required locally, as `tox` internally installs and uses `poetry`, be it to manage Python dependencies or to run `tox` environments. To install it: `pipx install tox`.
7+
8+
Optionally, `poerty` can be additionally installed independently just for the sake of running Python commands locally outside of `tox` during debugging/development. To install it: `pipx install poetry`.
9+
10+
11+
### Updating Dependencies
12+
13+
To add/update/remove any dependencies and/or to upgrade Python, simply:
14+
15+
1. add/update/remove such dependencies to/in/from the desired group(s) below `[tool.poetry.group.<your-group>.dependencies]` in `pyproject.toml`, and/or upgrade Python itself in `requires-python` under `[project]`
16+
17+
_⚠️ dependencies for the charm itself are also defined as dependencies of a dedicated group called `charm`, specifically below `[tool.poetry.group.charm.dependencies]`, and not as project dependencies below `[project.dependencies]` or `[tool.poetry.dependencies]` ⚠️_
18+
19+
2. run `tox -e update-requirements` to update the lock file
20+
21+
by this point, `poerty`, through `tox`, will let you know if there are any dependency conflicts to solve.
22+
23+
3. optionally, if you also want to update your local environment for running Python commands/scripts yourself and not through tox, see [Running Python Environments](#running-python-environments) below
24+
25+
26+
### Running `tox` Environments
27+
28+
To run `tox` environments, either locally for development or in CI workflows for testing, ensure to have `tox` installed first and then simply run your `tox` environments natively (e.g.: `tox -e lint`). `tox` will internally first install `poetry` and then rely on it to install and run its environments.
29+
30+
31+
### Running Python Environments
32+
33+
To run Python commands locally for debugging/development from any environments built from any combinations of dependency groups without relying on `tox`:
34+
1. ensure you have `poetry` installed
35+
2. install any required dependency groups: `poetry install --only <your-group-a>,<your-group-b>` (or all groups, if you prefer: `poetry install --all-groups`)
36+
3. run Python commands via poetry: `poetry run python3 <your-command>`

‎charms/argo-controller/charmcraft.yaml‎

Lines changed: 25 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -14,31 +14,44 @@ platforms:
1414
# (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/services/package.py#L290-L293
1515
# https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/services/package.py#L156-L157)
1616
parts:
17-
# "python-deps" part name is arbitrary; use for consistency
18-
# (but could become a magic constant in the future, similar to "poetry-deps"
19-
# https://github.com/canonical/craft-parts/pull/901)
20-
python-deps:
17+
# "poetry-deps" part name is a magic constant
18+
# https://github.com/canonical/craft-parts/pull/901
19+
poetry-deps:
2120
plugin: nil
21+
build-packages:
22+
- curl
2223
override-build: |
2324
# Use environment variable instead of `--break-system-packages` to avoid failing on older
2425
# versions of pip that do not recognize `--break-system-packages`
2526
# `--user` needed (in addition to `--break-system-packages`) for Ubuntu >=24.04
2627
PIP_BREAK_SYSTEM_PACKAGES=true python3 -m pip install --user --upgrade pip==24.3.1 # renovate: charmcraft-pip-latest
2728
28-
# "charm-python" part name is arbitrary; use for consistency
29+
# Use uv to install poetry so that a newer version of Python can be installed if needed by poetry
30+
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.5.15/uv-installer.sh | sh # renovate: charmcraft-uv-latest
31+
# poetry 2.0.0 requires Python >=3.9
32+
if ! "$HOME/.local/bin/uv" python find '>=3.9'
33+
then
34+
# Use first Python version that is >=3.9 and available in an Ubuntu LTS
35+
# (to reduce the number of Python versions we use)
36+
"$HOME/.local/bin/uv" python install 3.10.12 # renovate: charmcraft-python-ubuntu-22.04
37+
fi
38+
"$HOME/.local/bin/uv" tool install --no-python-downloads --python '>=3.9' poetry==2.0.0 --with poetry-plugin-export==1.8.0 # renovate: charmcraft-poetry-latest
39+
40+
ln -sf "$HOME/.local/bin/poetry" /usr/local/bin/poetry
41+
# "charm-poetry" part name is arbitrary; use for consistency
2942
# Avoid using "charm" part name since that has special meaning to charmcraft
30-
charm-python:
31-
# By default, the `python` plugin creates/stages these directories:
43+
charm-poetry:
44+
# By default, the `poetry` plugin creates/stages these directories:
3245
# - lib, src
33-
# (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/parts/plugins/_python.py#L79-L81)
46+
# (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/parts/plugins/_poetry.py#L76-L78)
3447
# - venv
35-
# (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/parts/plugins/_python.py#L100
48+
# (https://github.com/canonical/charmcraft/blob/9ff19c328e23b50cc06f04e8a5ad4835740badf4/charmcraft/parts/plugins/_poetry.py#L95
3649
# https://github.com/canonical/craft-parts/blob/afb0d652eb330b6aaad4f40fbd6e5357d358de47/craft_parts/plugins/base.py#L270)
37-
plugin: python
50+
plugin: poetry
3851
source: .
3952
after:
40-
- python-deps
41-
python-requirements: [requirements.txt]
53+
- poetry-deps
54+
poetry-export-extra-args: ['--only', 'charm']
4255
build-packages:
4356
- libffi-dev # Needed to build Python dependencies with Rust from source
4457
- libssl-dev # Needed to build Python dependencies with Rust from source
@@ -66,7 +79,6 @@ parts:
6679
craftctl default
6780
# Include requirements.txt in *.charm artifact for easier debugging
6881
cp requirements.txt "$CRAFT_PART_INSTALL/requirements.txt"
69-
7082
# "files" part name is arbitrary; use for consistency
7183
files:
7284
plugin: dump

0 commit comments

Comments
 (0)