You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* implement whl, whl_no_aio, sdist, mindependency, latestdependency in azpysdk
* add support to azpysdk for --injected-packages + --mark-arg to dispatch checks
* Update all docs referencing `tox` to reference our `azpysdk` tool
* Update all tests to run using the new `azpysdk` to invoke the checks intead of `tox`. Both live and recorded.
* follow-up PRs should begin removing tox scripts entirely
Co-authored-by: jennypng <63012604+JennyPng@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: mccoyp <mcpatino@microsoft.com>
@@ -11,9 +11,7 @@ If you want to contribute to a file that is generated (header contains `Code gen
11
11
12
12
We utilize a variety of tools to ensure smooth development, testing, and code quality for the Azure Python SDK. Below is a list of key tools and their purpose in the workflow:
13
13
14
-
- Tox: [Tox](https://tox.wiki/en/latest/) is our primary tool for managing test environments. It allows us to distribute tests to virtual environments, install dependencies, and maintain consistency between local and CI builds. Tox is configured to handle various testing scenarios, including linting, type checks, and running unit tests.
15
-
16
-
- Virtualenv: [Virtualenv](https://virtualenv.pypa.io/en/latest/) is leveraged by Tox to create isolated environments for each test suite, ensuring consistent dependencies and reducing conflicts.
14
+
- azpysdk: The `azpysdk` CLI is our primary tool for running checks locally and in CI. It is an entrypoint provided by the `eng/tools/azure-sdk-tools` package and abstracts all checks (linting, type checking, tests, doc generation, etc.) behind a single command. See the [Tool Usage Guide](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/tool_usage_guide.md) for full details.
17
15
18
16
- UV: [UV](https://docs.astral.sh/uv/) is a fast package manager that can manage Python versions, run and install Python packages, and be used instead of pip, virtualenv, and more.
19
17
@@ -31,140 +29,65 @@ We utilize a variety of tools to ensure smooth development, testing, and code qu
31
29
32
30
## Building and Testing
33
31
34
-
The Azure SDK team's Python CI leverages the tool `tox` to distribute tests to virtual environments, handle test dependency installation, and coordinate tooling reporting during PR/CI builds. This means that a dev working locally can reproduce _exactly_ what the build machine is doing.
35
-
36
-
[A Brief Overview of Tox](https://tox.wiki/en/latest/)
37
-
38
-
#### A Monorepo and Tox in Harmony
39
-
40
-
Traditionally, the `tox.ini` file for a package sits _alongside the setup.py_ in source code. The `azure-sdk-for-python` necessarily does not adhere to this policy. There are over one-hundred packages contained here-in. That's a lot of `tox.ini` files to maintain!
41
-
42
-
Instead, the CI system leverages the `--root` argument which is new to `tox4`. The `--root` argument allows `tox` to act as if the `tox.ini` is located in whatever directory you specify!
43
-
44
-
#### Tox Environments
45
-
46
-
A given `tox.ini` works on the concept of `test environments`. A given test environment is a combination of:
47
-
48
-
1. An identifier (or identifiers)
49
-
2. A targeted Python version
50
-
1.`tox` will default to base python executing the `tox` command if no Python environment is specified
51
-
3. (optionally) an OS platform
52
-
53
-
Internally `tox` leverages `virtualenv` to create each test environment's virtual environment.
32
+
The Azure SDK team's Python CI leverages the `azpysdk` CLI tool to run checks, tests, and linters during PR/CI builds. This means that a dev working locally can reproduce _exactly_ what the build machine is doing.
54
33
55
-
This means that once the `tox` workflow is in place, all tests will be executed _within a virtual environment._
34
+
The `azpysdk` entrypoint is provided by the `eng/tools/azure-sdk-tools` package. For full setup instructions and the list of available checks, see the [Tool Usage Guide](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/tool_usage_guide.md).
56
35
57
-
You can use the command `tox list` to list all the environments provided by a `tox.ini` file. You can either use that command in the
58
-
same directory as the file itself, or use the `--conf` argument to specify the path to it directly.
36
+
### Quick Setup
59
37
60
-
61
-
Sample output of `tox list`:
38
+
From the root of your target package:
62
39
63
40
```
64
-
sdk-for-python/eng/tox> tox list
65
-
default environments:
66
-
whl -> Builds a wheel and runs tests
67
-
sdist -> Builds a source distribution and runs tests
68
-
69
-
additional environments:
70
-
pylint -> Lints a package with a pinned version of pylint
71
-
next-pylint -> Lints a package with pylint (version 2.15.8)
72
-
mypy -> Typechecks a package with mypy (version 1.9.0)
73
-
next-mypy -> Typechecks a package with the latest version of mypy
74
-
pyright -> Typechecks a package with pyright (version 1.1.287)
75
-
next-pyright -> Typechecks a package with the latest version of static type-checker pyright
76
-
verifytypes -> Verifies the "type completeness" of a package with pyright
77
-
whl_no_aio -> Builds a wheel without aio and runs tests
78
-
develop -> Tests a package
79
-
sphinx -> Builds a package's documentation with sphinx
80
-
depends -> Ensures all modules in a target package can be successfully imported
81
-
verifywhl -> Verify directories included in whl and contents in manifest file
82
-
verifysdist -> Verify directories included in sdist and contents in manifest file. Also ensures that py.typed configuration is correct within the setup.py
83
-
devtest -> Tests a package against dependencies installed from a dev index
84
-
latestdependency -> Tests a package against the released, upper-bound versions of its azure dependencies
85
-
mindependency -> Tests a package against the released, lower-bound versions of its azure dependencies
86
-
apistub -> Generate an api stub of a package ( for https://apiview.dev )
87
-
bandit -> Runs bandit, a tool to find common security issues, against a package
88
-
samples -> Runs a package's samples
89
-
breaking -> Runs the breaking changes checker against a package
41
+
pip install -r dev_requirements.txt
90
42
```
91
43
92
-
### Example Usage of the common Azure SDK For Python `tox.ini`
93
-
94
-
Basic usage of `tox` within this monorepo is:
95
-
96
-
1.`pip install "tox<5"`
97
-
2. Run `tox run -e ENV_NAME -c path/to/tox.ini --root path/to/python_package`
98
-
***Note**: You can use environment variables to provide defaults for tox config values
99
-
* With `TOX_CONFIG_FILE` set to the absolute path of `tox.ini`, you can avoid needing `-c path/to/tox.ini` in your tox invocations
100
-
* With `TOX_ROOT_DIR` set to the absolute path to your python package, you can avoid needing `--root path/to/python_package`
101
-
102
-
The common `tox.ini` location is `eng/tox/tox.ini` within the repository.
103
-
104
-
If at any time you want to blow away the tox created virtual environments and start over, simply append `-r` to any tox invocation!
105
-
106
-
#### Example `azure-core` mypy
107
-
108
-
1. Run `tox run -e mypy -c ./eng/tox/tox.ini --root sdk/core/azure-core`
44
+
This installs `azure-sdk-tools` (which provides `azpysdk`) along with the package's dev dependencies.
109
45
110
-
#### Example `azure-storage-blob` tests
46
+
###Available Checks
111
47
112
-
2. Execute `tox run -c ./eng/tox/tox.ini --root sdk/storage/azure-storage-blob`
113
-
114
-
Note that we didn't provide an `environment` argument for this example. Reason here is that the _default_ environment selected by our common `tox.ini` file is one that runs `pytest`.
115
-
116
-
#### `whl` environment
117
-
Used for test execution across the spectrum of all the platforms we want to support. Maintained at a `platform specific` level just in case we run into platform-specific bugs.
118
-
119
-
* Installs the wheel, runs tests using the wheel
48
+
You can discover all available checks by running `azpysdk --help`. Some common checks:
120
49
121
50
```
122
-
\> tox run -e whl -c <path to tox.ini> --root <path to python package>
123
-
51
+
azpysdk pylint . # Lint with pylint
52
+
azpysdk mypy . # Type check with mypy
53
+
azpysdk pyright . # Type check with pyright
54
+
azpysdk verifytypes . # Verify type completeness
55
+
azpysdk sphinx . # Build documentation
56
+
azpysdk bandit . # Security analysis
57
+
azpysdk black . # Code formatting
58
+
azpysdk verifywhl . # Verify wheel contents
59
+
azpysdk verifysdist . # Verify sdist contents
60
+
azpysdk import_all . # Verify all imports resolve
61
+
azpysdk apistub . # Generate API stub
62
+
azpysdk samples . # Run samples
63
+
azpysdk breaking . # Check for breaking changes
64
+
azpysdk devtest . # Test against dev feed dependencies
124
65
```
125
66
126
-
#### `sdist` environment
127
-
Used for the local dev loop.
67
+
### Running from the repo root
128
68
129
-
* Installs package in editable mode
130
-
* Runs tests using the editable mode installation, not the wheel
69
+
`azpysdk` also supports globbing and comma-separated package names when invoked from the repo root:
131
70
132
71
```
133
-
134
-
\> tox run -e sdist -c <path to tox.ini> --root <path to python package>
\> tox run -e pylint -c <path to tox.ini> --root <path to python package>
143
-
```
76
+
### Isolated environments
144
77
145
-
146
-
#### `mypy` environment
147
-
Mypy install and run.
148
-
149
-
```
150
-
\> tox run -e mypy -c <path to tox.ini> --root <path to python package>
151
-
```
152
-
153
-
#### `sphinx` environment
154
-
Generate sphinx doc for this package.
78
+
To run a check in a completely fresh virtual environment, add `--isolate`:
155
79
156
80
```
157
-
\> tox run -e sphinx -c <path to tox.ini> --root <path to python package>
81
+
azpysdk pylint . --isolate
158
82
```
159
83
160
84
### Custom Pytest Arguments
161
85
162
-
`tox` supports custom arguments, and the defined pytest environments within the common `tox.ini` also allow these. Essentially, separate the arguments you want passed to `pytest` by a `--` in your tox invocation.
163
-
164
-
[Tox Documentation on Positional Arguments](https://tox.wiki/en/latest/config.html#substitutions-for-positional-arguments-in-commands)
86
+
When running test-related checks, you can pass additional arguments to `pytest` after `--`:
165
87
166
-
**Example: Invoke tox, breaking into the debugger on failure**
167
-
`tox run -e whl -c <path to tox.ini> --root <path to python package> -- --pdb`
88
+
```
89
+
azpysdk devtest . -- --pdb
90
+
```
168
91
169
92
### Performance Testing
170
93
@@ -175,7 +98,7 @@ SDK performance testing is supported via the custom `perfstress` framework. For
175
98
We maintain an [additional document](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/eng_sys_checks.md) that has a ton of detail as to what is actually _happening_ in these executions.
176
99
177
100
### Dev Feed
178
-
Daily dev build version of Azure sdk packages for python are available and are uploaded to Azure devops feed daily. We have also created a tox environment to test a package against dev built version of dependent packages. Below is the link to Azure devops feed.
101
+
Daily dev build version of Azure sdk packages for python are available and are uploaded to Azure devops feed daily. Below is the link to Azure devops feed.
Copy file name to clipboardExpand all lines: doc/dev/dev_setup.md
+1-3Lines changed: 1 addition & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -33,11 +33,10 @@ or execute the various commands available in the toolbox.
33
33
34
34
4. Setup your development environment
35
35
36
-
Install the development requirements for a specific library (located in the `dev_requirements.txt` file at the root of the library), [Tox][tox] and an editable install of your library. For example, to install requirements for `azure-ai-formrecognizer`:
36
+
Install the development requirements for a specific library (located in the `dev_requirements.txt` file at the root of the library)and an editable install of your library. This will also install `azure-sdk-tools` which provides the `azpysdk` CLI for running checks. For example, to install requirements for `azure-ai-formrecognizer`:
37
37
```
38
38
azure-sdk-for-python> cd sdk/formrecognizer/azure-ai-formrecognizer
If you don't want to use tox, you can also install and run pylint on its own:
29
+
If you don't want to use `azpysdk`, you can also install and run pylint on its own:
30
30
31
31
- If taking this approach, in order to run with the pylintrc formatting and the custom pylint checkers you must also install the custom checkers and `SET` the pylintrc path.
32
32
@@ -36,8 +36,7 @@ If you don't want to use tox, you can also install and run pylint on its own:
Note that you may see different errors if running a different [version of pylint or azure-pylint-guidelines-checker](https://github.com/Azure/azure-sdk-for-python/blob/fdf7c49ea760b1e1698ebbbac48794e8382d8de5/eng/tox/tox.ini#L90) than the one in CI.
40
-
39
+
Note that you may see different errors if running a different version of [pylint](https://github.com/Azure/azure-sdk-for-python/blob/main/eng/tools/azure-sdk-tools/azpysdk/pylint.py#L17) or [azure-pylint-guidelines-checker](https://github.com/Azure/azure-sdk-for-python/blob/main/eng/tools/azure-sdk-tools/azpysdk/pylint.py#L61) than the one in CI.
41
40
42
41
# Ignoring Pylint Checkers
43
42
@@ -58,12 +57,12 @@ In addition to being a part of the CI, the custom pylint checkers are also integ
58
57
59
58
There is now a new step on the CI pipeline called `Run Pylint Next`. This is merely a duplicate of the `Run Pylint` step with the exception that `Run Pylint Next` uses the latest version of pylint and the latest version of the custom pylint checkers.
60
59
61
-
This next-pylint environment can also be run locally through tox:
60
+
This next-pylint check can also be run locally:
62
61
63
-
tox run -e next-pylint -c ../../../eng/tox/tox.ini --root <path to python package>
62
+
azpysdk pylint --next=True <path to python package>
64
63
65
64
The errors generated by the `Run Pylint Next` step will not break your weekly test pipelines, but make sure to fix the warnings so that your client library is up to date for the next pylint release.
66
65
67
66
# How to prepare your SDK for a new pylint update?
68
67
69
-
Check each client library's `Run Pylint Next` output in the [test-weekly CI pipeline](https://dev.azure.com/azure-sdk/internal/_build?pipelineNameFilter=python%20*%20tests-weekly). If there is no corresponding test-weekly pipeline, run `next-pylint` locally with `tox` as described in [How to run Pylint?](#how-to-run-pylint). In order to ensure that the SDK pipeline will not break when pylint is updated, make sure to address all pylint warnings present.
68
+
Check each client library's `Run Pylint Next` output in the [test-weekly CI pipeline](https://dev.azure.com/azure-sdk/internal/_build?pipelineNameFilter=python%20*%20tests-weekly). If there is no corresponding test-weekly pipeline, run `next-pylint` locally with `azpysdk pylint --next=True .` as described in [How to run Pylint?](#how-to-run-pylint). In order to ensure that the SDK pipeline will not break when pylint is updated, make sure to address all pylint warnings present.
0 commit comments