Skip to content

Commit d2fdebe

Browse files
glehmannglehmann
authored
Merge pull request #321 from xcp-ng/gln/uv-setup-tests-uzqs
2 parents 39009b2 + 6c4bcc6 commit d2fdebe

File tree

13 files changed

+924
-64
lines changed

13 files changed

+924
-64
lines changed

‎.github/workflows/code-checkers.yml

Lines changed: 10 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -7,34 +7,28 @@ jobs:
77
runs-on: ubuntu-latest
88
steps:
99
- uses: actions/checkout@v2
10-
- name: Set up Python
11-
uses: actions/setup-python@v4
10+
- name: Install uv
11+
uses: astral-sh/setup-uv@v6
1212
with:
13-
python-version: 3.8
13+
version: "0.7.x"
1414
- name: Install dependencies
15-
run: |
16-
python -m pip install --upgrade pip
17-
pip install -r requirements/base.txt
18-
pip install mypy
15+
run: uv sync --frozen
1916
- name: Create a dummy data.py
2017
run: cp data.py-dist data.py
2118
- name: Install additional typing data and check with mypy
22-
run: mypy --install-types --non-interactive lib/ conftest.py pkgfixtures.py tests/
19+
run: uv run mypy --install-types --non-interactive lib/ conftest.py pkgfixtures.py tests/
2320

2421
pyright:
2522
runs-on: ubuntu-latest
2623
steps:
2724
- uses: actions/checkout@v2
28-
- name: Set up Python
29-
uses: actions/setup-python@v4
25+
- name: Install uv
26+
uses: astral-sh/setup-uv@v6
3027
with:
31-
python-version: 3.8
28+
version: "0.7.x"
3229
- name: Install dependencies
33-
run: |
34-
python -m pip install --upgrade pip
35-
pip install -r requirements/base.txt
36-
pip install pyright
30+
run: uv sync --frozen
3731
- name: Create a dummy data.py
3832
run: cp data.py-dist data.py
3933
- name: Check with pyright
40-
run: pyright lib/ conftest.py pkgfixtures.py # tests/
34+
run: uv run pyright lib/ conftest.py pkgfixtures.py # tests/

‎.github/workflows/format.yml

Lines changed: 12 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -7,26 +7,24 @@ jobs:
77
runs-on: ubuntu-latest
88
steps:
99
- uses: actions/checkout@v2
10-
- name: Set up Python
11-
uses: actions/setup-python@v4
10+
- name: Install uv
11+
uses: astral-sh/setup-uv@v6
12+
with:
13+
version: "0.7.x"
1214
- name: Install dependencies
13-
run: |
14-
python -m pip install --upgrade pip
15-
pip install flake8
15+
run: uv sync --frozen
1616
- name: flake8
17-
run: |
18-
flake8
17+
run: uv run flake8
1918

2019
pydocstyle:
2120
runs-on: ubuntu-latest
2221
steps:
2322
- uses: actions/checkout@v2
24-
- name: Set up Python
25-
uses: actions/setup-python@v2
23+
- name: Install uv
24+
uses: astral-sh/setup-uv@v6
25+
with:
26+
version: "0.7.x"
2627
- name: Install dependencies
27-
run: |
28-
python -m pip install --upgrade pip
29-
pip install pydocstyle
28+
run: uv sync --frozen
3029
- name: pydocstyle
31-
run: |
32-
pydocstyle
30+
run: uv run pydocstyle

‎.github/workflows/jobs-check.yml

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,13 @@ jobs:
77
runs-on: ubuntu-latest
88
steps:
99
- uses: actions/checkout@v2
10-
- name: Set up Python
11-
uses: actions/setup-python@v4
10+
- name: Install uv
11+
uses: astral-sh/setup-uv@v6
1212
with:
13-
python-version: 3.8
13+
version: "0.7.x"
1414
- name: Install dependencies
15-
run: |
16-
python -m pip install --upgrade pip
17-
pip install -r requirements/base.txt
15+
run: uv sync --frozen
1816
- name: Create a dummy data.py
1917
run: cp data.py-dist data.py
2018
- name: jobs-check
21-
run: ./jobs.py check
19+
run: uv run ./jobs.py check

‎.github/workflows/requirements-check.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: Check requirements file consistency
2+
3+
on: [push]
4+
5+
jobs:
6+
requirements-check:
7+
runs-on: ubuntu-latest
8+
steps:
9+
- uses: actions/checkout@v2
10+
- name: Install uv
11+
uses: astral-sh/setup-uv@v6
12+
with:
13+
version: "0.7.x"
14+
- name: Install dependencies
15+
run: uv sync --frozen
16+
- run: uv run ./requirements/update_requirements.py
17+
- run: git diff --exit-code

‎.github/workflows/test-sequences.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,19 +7,19 @@ jobs:
77
runs-on: ubuntu-latest
88
steps:
99
- uses: actions/checkout@v2
10-
- name: Set up Python
11-
uses: actions/setup-python@v4
10+
- name: Install uv
11+
uses: astral-sh/setup-uv@v6
12+
with:
13+
version: "0.7.x"
1214
- name: Install dependencies
13-
run: |
14-
python -m pip install --upgrade pip
15-
pip install -r requirements/base.txt
15+
run: uv sync --frozen
1616
- name: Create a dummy data.py
1717
run: cp data.py-dist data.py
1818
- name: jobs-check
1919
run: |
2020
FAILURES=""
2121
for seq in $(find -name "*.lst"); do
22-
if ! pytest @$seq --collect-only --quiet; then
22+
if ! uv run pytest @$seq --collect-only --quiet; then
2323
FAILURES="$FAILURES $seq"
2424
fi
2525
done

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ __pycache__
33
data.py
44
vm_data.py
55
/scripts/guests/windows/id_rsa.pub
6+
.envrc

‎.python-version

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.11.11

‎README.md

Lines changed: 118 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,25 +3,130 @@
33
Note: this is a perpertual work in progress. If you encounter any obstacles or bugs, let us know!
44

55
## Main requirements
6-
* python >= 3.8
7-
* packages as listed in requirements/base.txt
8-
* extra test-specific requirements are documented in the test file
9-
"Requirements" header
6+
7+
This project encourages the use of [uv](https://github.com/astral-sh/uv) to deal with the python dependencies.
8+
9+
Some tools are required on the host running the tests:
10+
* python >= 3.11 (from your distro, or from `uv` as described below)
1011
* `netcat` is required on the machine running the tests.
1112

12-
### Quick install (python requirements)
13+
Extra test-specific requirements are documented in the test file "Requirements" header.
1314

14-
Install the python requirements using pip:
15+
### Quick install
1516

16-
```
17-
$ pip install -r requirements/base.txt
18-
```
17+
Install `uv` with one of these commands
1918

20-
Additionally, for dev dependencies (things like the linter / style checker):
19+
~~~sh
20+
pip install uv # for Fedora-based and older distros
21+
pipx install uv # for newer distros
22+
curl -LsSf https://astral.sh/uv/install.sh | sh # if you don't mind an installer modifying your environment
23+
~~~
2124

22-
```
23-
$ pip install -r requirements/dev.txt
24-
```
25+
You have two options to run the tests:
26+
27+
* with the `uv run` command
28+
29+
You can use the dependencies managed by `uv` by prefixing the commands documented in the next chapters with `uv run`.
30+
For example
31+
32+
~~~sh
33+
uv run pytest
34+
~~~
35+
36+
or
37+
38+
~~~sh
39+
uv run ./jobs.py
40+
~~~
41+
42+
When running the command with `uv run`, `uv` checks that your dependencies are
43+
up to date and updates them if required, before running your command. You're
44+
always sure to have the expected dependencies, even when switching branch!
45+
46+
* by activating the virtual environment
47+
48+
You need to run `uv sync` to install the dependencies managed by uv, and then
49+
activate the virtual environment with
50+
51+
~~~sh
52+
source .venv/bin/activate
53+
~~~
54+
55+
You can then run the commands directly in your shell: `pytest`, `./jobs.py`, …
56+
57+
Note you have to explicitly run `uv sync` when the dependencies are modified in
58+
the project, as `uv` won't ensure that the dependencies are up to date when run
59+
this way.
60+
61+
Running `deactivate` restores your shell configuration.
62+
63+
### Advanced usages
64+
65+
> [!CAUTION]
66+
> These methods don't strictly guarantee the version of the dependencies.
67+
68+
#### Use a specific python interpreter with `uv`
69+
70+
`uv` is able to use any python version accessible on your system. Many `uv`
71+
commands accept a `-p`/`--python` option which takes either the path of the
72+
python interpreter to use, or a python version. `uv` also uses the value of
73+
the `UV_PYTHON` environment variable to set the python interpreter (or version
74+
to use).
75+
76+
So if you want to run this project with your system's interpreter, you can
77+
78+
~~~sh
79+
export UV_PYTHON=$(which python3)
80+
~~~
81+
82+
and then use `uv` as shown in the previous section.
83+
84+
> [!TIP]
85+
> [direnv](https://direnv.net/) may be used to automatically deal with the
86+
> environment variables when you enter the project directory with your shell.
87+
88+
#### Dealing with dependencies without `uv`
89+
90+
This project can be used without `uv`.
91+
92+
If you desire, you can install a virtual environment with
93+
94+
~~~sh
95+
python3 -m venv .venv
96+
source .venv/bin/activate
97+
~~~
98+
99+
or skip the virtual environment entirely and just go with a user or system
100+
installation of the dependencies.
101+
102+
The base dependencies can be installed with
103+
104+
~~~sh
105+
./pip_install_pyproject.py
106+
~~~
107+
108+
Optionally, you can install the development dependencies with
109+
110+
~~~sh
111+
./pip_install_pyproject.py dev
112+
~~~
113+
114+
### Adding python dependencies
115+
116+
When adding a dependency, run
117+
118+
~~~sh
119+
uv add --raw my_dep
120+
~~~
121+
122+
This command installs the dependency locally and update the `uv.lock` file.
123+
124+
> [!NOTE]
125+
> In order to keep this project usable with dependency packaged by the system,
126+
> we always use `uv add` with the `--raw` command line option.
127+
128+
The lock file allows `uv` to install the dependencies at the exact same versions
129+
on all the environments.
25130

26131
## Other requirements
27132
* XCP-ng hosts that you can ssh to using a SSH key, non-interactively

‎pyproject.toml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,32 @@
1+
[project]
2+
name = "xcp-ng-tests"
3+
version = "0.1.0"
4+
description = "Testing scripts for XCP-ng"
5+
readme = "README.md"
6+
requires-python = "~=3.11"
7+
dependencies = [
8+
"cryptography>=3.3.1",
9+
"gitpython",
10+
"legacycrypt",
11+
"packaging>=20.7",
12+
"pluggy>=1.1.0",
13+
"pytest>=8.0.0",
14+
"pytest-dependency",
15+
"requests",
16+
]
17+
18+
[dependency-groups]
19+
dev = [
20+
"ansible>=5.0.1",
21+
"bs4>=0.0.1",
22+
"mypy",
23+
"flake8",
24+
"pydocstyle",
25+
"pyright",
26+
"pyyaml>=6.0",
27+
"types-requests",
28+
"typing-extensions",
29+
]
30+
131
[tool.pyright]
232
typeCheckingMode = "standard"

‎requirements/base.txt

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1+
# generated with update_requirements.py, do not edit manually
12
cryptography>=3.3.1
2-
GitPython
3+
gitpython
4+
legacycrypt
35
packaging>=20.7
4-
pytest>=8.0.0
56
pluggy>=1.1.0
6-
requests
7-
legacycrypt
7+
pytest>=8.0.0
88
pytest-dependency
9+
requests

0 commit comments

Comments
 (0)