|
1 | 1 | # For Developers |
2 | 2 |
|
3 | | -## Updating internal dependencies |
| 3 | +This document covers tips and guidance for working on the rules_python code |
| 4 | +base. A primary audience for it is first time contributors. |
4 | 5 |
|
5 | | -1. Modify the `./python/private/pypi/requirements.txt` file and run: |
6 | | - ``` |
7 | | - bazel run //private:whl_library_requirements.update |
8 | | - ``` |
9 | | -1. Run the following target to update `twine` dependencies: |
10 | | - ``` |
11 | | - bazel run //private:requirements.update |
12 | | - ``` |
13 | | -1. Bump the coverage dependencies using the script using: |
14 | | - ``` |
15 | | - bazel run //tools/private/update_deps:update_coverage_deps <VERSION> |
16 | | - # for example: |
17 | | - # bazel run //tools/private/update_deps:update_coverage_deps 7.6.1 |
18 | | - ``` |
| 6 | +## Running tests |
| 7 | + |
| 8 | +Running tests is particularly easy thanks to Bazel, simply run: |
| 9 | + |
| 10 | +``` |
| 11 | +bazel test //... |
| 12 | +``` |
19 | 13 |
|
20 | | -## Releasing |
| 14 | +And it will run all the tests it can find. The first time you do this, it will |
| 15 | +probably take long time because various dependencies will need to be downloaded |
| 16 | +and setup. Subsequent runs will be faster, but there are many tests, and some of |
| 17 | +them are slow. If you're working on a particular area of code, you can run just |
| 18 | +the tests in those directories instead, which can speed up your edit-run cycle. |
21 | 19 |
|
22 | | -Start from a clean checkout at `main`. |
| 20 | +## Writing Tests |
23 | 21 |
|
24 | | -Before running through the release it's good to run the build and the tests locally, and make sure CI is passing. You can |
25 | | -also test-drive the commit in an existing Bazel workspace to sanity check functionality. |
| 22 | +Most code should have tests of some sort. This helps us have confidence that |
| 23 | +refactors didn't break anything and that releases won't have regressions. |
26 | 24 |
|
27 | | -### Releasing from HEAD |
| 25 | +We don't require 100% test coverage, testing certain Bazel functionality is |
| 26 | +difficult, and some edge cases are simply too hard to test or not worth the |
| 27 | +extra complexity. We try to judiciously decide when not having tests is a good |
| 28 | +idea. |
28 | 29 |
|
29 | | -#### Steps |
30 | | -1. [Determine the next semantic version number](#determining-semantic-version) |
31 | | -1. Create a tag and push, e.g. `git tag 0.5.0 upstream/main && git push upstream --tags` |
32 | | - NOTE: Pushing the tag will trigger release automation. |
33 | | -1. Watch the release automation run on https://github.com/bazelbuild/rules_python/actions |
34 | | -1. Add missing information to the release notes. The automatic release note |
35 | | - generation only includes commits associated with issues. |
| 30 | +Tests go under `tests/`. They are loosely organized into directories for the |
| 31 | +particular subsystem or functionality they are testing. If an existing directory |
| 32 | +doesn't seem like a good match for the functionality being testing, then it's |
| 33 | +fine to create a new directory. |
36 | 34 |
|
37 | | -#### Determining Semantic Version |
| 35 | +Re-usable test helpers and support code go in `tests/support`. Tests don't need |
| 36 | +to be perfectly factored and not every common thing a test does needs to be |
| 37 | +factored into a more generally reusable piece. Copying and pasting is fine. It's |
| 38 | +more important for tests to balance understandability and maintainability. |
38 | 39 |
|
39 | | -**rules_python** is currently using [Zero-based versioning](https://0ver.org/) and thus backwards-incompatible API |
40 | | -changes still come under the minor-version digit. So releases with API changes and new features bump the minor, and |
41 | | -those with only bug fixes and other minor changes bump the patch digit. |
| 40 | +### sh_py_run_test |
42 | 41 |
|
43 | | -To find if there were any features added or incompatible changes made, review |
44 | | -the commit history. This can be done using github by going to the url: |
45 | | -`https://github.com/bazelbuild/rules_python/compare/<VERSION>...main`. |
| 42 | +The [`sh_py_run_test`](tests/support/sh_py_run_test.bzl) rule is a helper to |
| 43 | +make it easy to run a Python program with custom build settings using a shell |
| 44 | +script to perform setup and verification. This is best to use when verifying |
| 45 | +behavior needs certain environment variables or directory structures to |
| 46 | +correctly and reliably verify behavior. |
46 | 47 |
|
47 | | -### Patch release with cherry picks |
| 48 | +When adding a test, you may find the flag you need to set isn't supported by |
| 49 | +the rule. To have it support setting a new flag, see the py_reconfig_test docs |
| 50 | +below. |
48 | 51 |
|
49 | | -If a patch release from head would contain changes that aren't appropriate for |
50 | | -a patch release, then the patch release needs to be based on the original |
51 | | -release tag and the patch changes cherry-picked into it. |
| 52 | +### py_reconfig_test |
52 | 53 |
|
53 | | -In this example, release `0.37.0` is being patched to create release `0.37.1`. |
54 | | -The fix being included is commit `deadbeef`. |
| 54 | +The `py_reconfig_test` and `py_reconfig_binary` rules are helpers for running |
| 55 | +Python binaries and tests with custom build flags. This is best to use when |
| 56 | +verifying behavior that requires specific flags to be set and when the program |
| 57 | +itself can verify the desired state. |
55 | 58 |
|
56 | | -1. `git checkout -b release/0.37 0.37.0` |
57 | | -1. `git push upstream release/0.37` |
58 | | -1. `git cherry-pick -x deadbeef` |
59 | | -1. Fix merge conflicts, if any. |
60 | | -1. `git cherry-pick --continue` (if applicable) |
61 | | -1. `git push upstream` |
| 59 | +When adding a test, you may find the flag you need to set isn't supported by |
| 60 | +the rule. To have it support setting a new flag: |
62 | 61 |
|
63 | | -If multiple commits need to be applied, repeat the `git cherry-pick` step for |
64 | | -each. |
| 62 | +* Add an attribute to the rule. It should have the same name as the flag |
| 63 | + it's for. It should be a string, string_list, or label attribute -- this |
| 64 | + allows distinguishing between if the value was specified or not. |
| 65 | +* Modify the transition and add the flag to both the inputs and outputs |
| 66 | + list, then modify the transition's logic to check the attribute and set |
| 67 | + the flag value if the attribute is set. |
65 | 68 |
|
66 | | -Once the release branch is in the desired state, use `git tag` to tag it, as |
67 | | -done with a release from head. Release automation will do the rest. |
| 69 | +### Integration tests |
68 | 70 |
|
69 | | -#### After release creation in Github |
| 71 | +An integration test is one that runs a separate Bazel instance inside the test. |
| 72 | +These tests are discouraged unless absolutely necessary because they are slow, |
| 73 | +require much memory and CPU, and are generally harder to debug. Integration |
| 74 | +tests are reserved for things that simple can't be tested otherwise, or for |
| 75 | +simple high level verification tests. |
70 | 76 |
|
71 | | -1. Announce the release in the #python channel in the Bazel slack (bazelbuild.slack.com). |
| 77 | +Integration tests live in `tests/integration`. When possible, add to an existing |
| 78 | +integration test. |
72 | 79 |
|
73 | | -## Secrets |
| 80 | +## Updating internal dependencies |
| 81 | + |
| 82 | +1. Modify the `./python/private/pypi/requirements.txt` file and run: |
| 83 | + ``` |
| 84 | + bazel run //private:whl_library_requirements.update |
| 85 | + ``` |
| 86 | +1. Run the following target to update `twine` dependencies: |
| 87 | + ``` |
| 88 | + bazel run //private:requirements.update |
| 89 | + ``` |
| 90 | +1. Bump the coverage dependencies using the script using: |
| 91 | + ``` |
| 92 | + bazel run //tools/private/update_deps:update_coverage_deps <VERSION> |
| 93 | + # for example: |
| 94 | + # bazel run //tools/private/update_deps:update_coverage_deps 7.6.1 |
| 95 | + ``` |
74 | 96 |
|
75 | | -### PyPI user rules-python |
| 97 | +## Updating tool dependencies |
76 | 98 |
|
77 | | -Part of the release process uploads packages to PyPI as the user `rules-python`. |
78 | | -This account is managed by Google; contact [email protected] if |
79 | | -something needs to be done with the PyPI account. |
| 99 | +It's suggested to routinely update the tool versions within our repo - some of the |
| 100 | +tools are using requirement files compiled by `uv` and others use other means. In order |
| 101 | +to have everything self-documented, we have a special target - |
| 102 | +`//private:requirements.update`, which uses `rules_multirun` to run in sequence all |
| 103 | +of the requirement updating scripts in one go. This can be done once per release as |
| 104 | +we prepare for releases. |
0 commit comments