Skip to content

Commit 1ae06cb

Browse files
committed
docs: update docs for integration tests
1 parent e4aeacb commit 1ae06cb

File tree

1 file changed

+22
-43
lines changed

1 file changed

+22
-43
lines changed

tests_integration/README.md

Lines changed: 22 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -26,41 +26,39 @@ definitions/
2626
│ ├── resources/
2727
│ │ ├── feedstock
2828
│ │ └── ... (entirely custom)
29-
│ ├── version_update.py
30-
│ ├── aarch_migration.py
31-
│ ├── some_other_test_case.py
32-
│ └── ...
29+
│ └── __init__.py
3330
├── llvmdev/
3431
│ ├── resources/
35-
│ └── test_case.py
32+
│ └── __init__.py
3633
└── ...
3734
```
3835

39-
Each feedstock has its own directory containing the test cases for that feedstock. The test cases are defined in
40-
Python files in the feedstock directory, where each file contains a single test case.
36+
Each feedstock has its own Python module containing the test cases for that feedstock.
37+
**The test cases are always defined in the top-level `__init__.py` file of the feedstock directory.**
4138

4239
For storing resources, a `resources` directory is used for each feedstock directory.
4340
Inside the `resources` directory, you can use an arbitrary directory structure to store the resources.
4441

4542
Usually, we include a specific revision of the original feedstock as a submodule in the `resources` directory.
4643

4744
A test case always tests the entire pipeline of the bot and not any intermediate states that could be checked
48-
in the cf-graph. See the [workflow file](../.github/workflows/test-integration.yml) for more details.
45+
in the cf-graph. See the [pytest test definition](test_integration.py) for more details.
4946
Also, a test case is always bound to one specific feedstock.
5047

5148
### Test Case Definition
52-
To define a test case, create a Python file in the definitions dir of the feedstock. You can name it arbitrarily.
53-
Referring to
54-
[this](definitions/pydantic/version_update.py) minimal test case,
55-
you have to define three things:
49+
To define a test case, create a subclass of `tests_integration.lib.test_case.TestCase` in the `__init__.py` file of
50+
your feedstock. You can name it arbitrarily.
51+
Referring to the minimal `VersionUpdate` test case in the
52+
[pydantic module](definitions/pydantic/__init__.py),
53+
your class has to implement three methods:
5654

57-
1. A function `prepare(helper: IntegrationTestHelper)` for setting up your test case. Usually, you will want to
55+
1. `get_router()` should return an `APIRouter` object to define mock responses for specific HTTP requests. All web requests are intercepted by an HTTP proxy.
56+
Refer to `tests_integration.lib.shared.get_transparent_urls` to define URLs that should not be intercepted.
57+
58+
2. `prepare(helper: IntegrationTestHelper)` for setting up your test case. Usually, you will want to
5859
overwrite the feedstock repository in the test environment. The `IntegrationTestHelper` provides methods to interact
5960
with the test environment.
6061

61-
2. A `router` object to define mock responses for specific HTTP requests. All web requests are intercepted by an HTTP proxy.
62-
Consult `tests_integration.lib.shared.get_transparent_urls` to define URLs that should not be intercepted.
63-
6462
3. A function `validate(helper: IntegrationTestHelper)` for validating the state after the bot has run.
6563
The `IntegrationTestHelper` provides convenience methods such as `assert_version_pr_present` to check for the presence
6664
of a version update PR.
@@ -119,44 +117,25 @@ For example, scenario 1 uses `test_case_3.py` from Feedstock A and `test_case_2.
119117
## Environment Variables
120118
The tests expect the following environment variables:
121119

122-
| Variable | Description |
123-
|--------------------|----------------------------------------------------------------------------------------------------------------------------------|
124-
| `TEST_SETUP_TOKEN` | Classic PAT for `cf-regro-autotick-bot-staging` used to setup the test environment. Typically, this is identical to `BOT_TOKEN`. |
125-
| `GITHUB_RUN_ID` | Set by GitHub. ID of the current run. Used as random seed. |
120+
| Variable | Description |
121+
|--------------------|-----------------------------------------------------------------------------------------------------------------------------------|
122+
| `BOT_TOKEN` | Classic PAT for `cf-regro-autotick-bot-staging`. Used to interact with the test environment. |
123+
| `TEST_SETUP_TOKEN` | Classic PAT for `cf-regro-autotick-bot-staging` used to setup the test environment. Typically, this is identical to `BOT_TOKEN`. |
124+
| `GITHUB_RUN_ID` | Set by GitHub. ID of the current run. Used as random seed. |
126125

127126

128127
We do not use `BOT_TOKEN` instead of `TEST_SETUP_TOKEN` for setting up the test environment to allow for future separation of the two tokens.
129128
Furthermore, `BOT_TOKEN` is hidden by the sensitive env logic of `conda_forge_tick` and we want the test environment to not need to rely on this logic.
130129

131130

132131
### GitHub Token Permissions
133-
The token should have the following scopes: `repo`, `workflow`, `delete_repo`.
132+
The bot token (which you can should use as the test setup token) should have the following scopes: `repo`, `workflow`, `delete_repo`.
134133

135134
## Running the Integration Tests Locally
136135

137136
To run the integration tests locally, you currently need to have a valid token for the `cf-regro-autotick-bot-staging` account.
138-
Besides that, the further setup is described below.
139-
140-
### Generate CA Certificates
141-
142-
For mitmproxy to properly intercept HTTPS traffic, you need to generate a CA certificate:
143-
144-
```bash
145-
cd .mitmproxy
146-
openssl genrsa -out mitmproxy-ca.key 4096
147-
openssl req -x509 -new -nodes -key mitmproxy-ca.key -sha256 -out mitmproxy-ca.crt -addext keyUsage=critical,keyCertSign -subj "/C=US/ST=cf-scripts/L=cf-scripts/O=cf-scripts/OU=cf-scripts/CN=cf-scripts"
148-
cat mitmproxy-ca.key mitmproxy-ca.crt > mitmproxy-ca.pem
149-
```
150-
151-
After running the commands above, trust the CA certificate in your system's keychain.
152-
153-
On macOS, you do this by dragging the `mitmproxy-ca.pem` file into the "Keychain Access" app while having the
154-
"Login" keychain selected. Then, double-click the certificate in the keychain and set "Always Trust" in the "Trust" section.
155-
156-
Then, build the certificate bundle to pass to Python:
137+
Besides that, run the following setup wizard to set up self-signed certificates for the HTTP proxy:
157138

158139
```bash
159-
cd .mitmproxy # if not already there
160-
cp $(python -m certifi) mitmproxy-cert-bundle.pem
161-
cat mitmproxy-ca.pem >> mitmproxy-cert-bundle.pem
140+
./mitmproxy_setup_wizard.sh
162141
```

0 commit comments

Comments
 (0)