Skip to content

Commit c39a5d3

Browse files
ytauschpavelzwbeckermrjaimergpregro-cf-autotick-bot
authored
Integration tests (#3417)
* run update_upstream_versions in tests * gather feedstocks in tests + make graph * update upstream versions in tests * add other make-graph step * enable debugging, deploy, don't update pinnings every time * refactor and fix env vars * add pydantic feedstock as a resource * add IntegrationTestHelper supporting overwriting feedstock contents * relock, fix install bot code * fix subprocess call * set git initial branch * fix tests * set proxy in update-upstream-versions * global router + overwrite cf-graph repo * reorder steps * fix install bot code * disable random skipping in integration tests * use correct env name for container image * (todo) don't overwrite container image env variables * reorder action steps * fix * don't create pinning repo * debug * fix * debug * debug * also capture stderr * use older version for pydantic * use lowercase proxy * set DEPLOY_REPO correctly * tmate action * dont use proxy for tmate * fix * fix proxy in container * fix * git CLI: don't use proxy * add make-migrators step * fix * relock * don't use with sensitive_env * fix * fix * make GitHub API transparent * install_bot_code does not need BOT_TOKEN * separate bot user token and regro org token * apply fixes * relock * fix token * fix URL request handler * fix URL request handler * Revert "Merge branch 'separate-bot-tokens' into integration-tests" This reverts commit 63fec86, reversing changes made to 56f9caa. * Revert "Merge branch 'separate-bot-tokens' into integration-tests" This reverts commit 56f9caa, reversing changes made to d1f213a. * use classic PAT * remove tmate * use TEST_SETUP_TOKEN * fix: forward headers * fix proxy * make PyPI tarfiles transparent * route transparent URLs directly by mitmproxy, configure conda-forge org * use conda-forge-bot-staging everywhere * fix setup * add llvmdev via submodule * add pydantic via submodule * run make-migrators twice * successfully validate new pydantic version * remove llvmdev from integration tests * relock * fix conda forge org override * fix typo * fix todo: override env variables in install_bot_code.sh * fix: don't divide by zero * finish docs for integration testing * docs: refer to TRANSPARENT_URLs * try to run make-migrators a third time * review: fix branch name Co-authored-by: Pavel Zwerschke <[email protected]> * pin GitHub Actions correctly * fix: we don't want to change num_try * fix: use CONDA_FORGE_ORG * move ENV_OVERRIDE_CONDA_FORGE_ORG to settings.py * use debug log level if running workflow in debug mode * refactor: move env variable name to constant * DRY: use identical env variable for cf-graph repository * Revert "try to run make-migrators a third time" This reverts commit 973d95c. did not solve the issue * docs: fix reference to env variables * use Docker image name consistently * refactor: rename DEPLOY_REPO to GRAPH_REPO to make it consistent with the env variable naming * fix f-string * clarify bug comment * move RANDOM_FRAC_TO_UPDATE to settings * resolve TODO * fix docstring * fix: reorder try clause * fix: add pip install * fix: move imports to top-level * add debug logging to proxy * add CODEOWNERS file to automatically request further reviews * add CODEOWNERS file * fix: push 1.x branch to pydantic feedstock * relock * move proxy mode to conda-forge-feedstock-ops see conda-forge/conda-forge-feedstock-ops#34 * fix reponses from PyPI API * retrigger ci * ruff * run integration tests in merge queue * add dummy check for setup-repositories * run integration tests on PRs from branches * remove dummy workflow * skip on push * review suggestion: reformat Co-authored-by: Matthew R. Becker <[email protected]> * Apply suggestions from code review Co-authored-by: Matthew R. Becker <[email protected]> * separate RANDOM_FRAC_TO_UPDATE for make-graph and update-upstream-versions * fix comment * add docstring for settings.py * add type annotations for settings * prevent duplicate kwarg errors * use deferred log expressions Co-authored-by: jaimergp <[email protected]> * clarify mimicked ClonedFeedstockContext warning * remove "environment variables" comment again * make graph repo default branch configurable * defer log expression * add comment about GITHUB_OUTPUT * use working-directory instead of cd * fix env var * clarify docstring * docs: random.shuffle is in-place * add docs to explain test scenario collection * fix images * fix except AttributeError * add shebang Co-authored-by: jaimergp <[email protected]> * Update tests_integration/resources/empty-graph/README.md Co-authored-by: jaimergp <[email protected]> * docs: clarify arbitrary test case name * docs: improve wording wro test scenario * begin refactor integration tests to pytest * continue refactor integration tests to pytest * update submodule * document mitmproxy * migrate to xprocess for mitmproxy * make dir switches benign * edit gitignore * make integration tests pass * chore: relock * fix: adjust for new settings module ir * feat: add mitmproxy cert wizard, move mitmproxy dir * docs: update docs for integration tests * feat: run new test concept everywhere * chore: disable Docker caching because it's faster * fix: condarc file in CI * fix: ci-driven development * fix: heredoc * ci: fix script path * ci: debug * fix: trust script and ci * ci: disable debug mode * fix: set git identity in integration tests * debug ci * fix: do proxy alive check via port probing * debug ci * fix: pass PYTHONPATH to proxy script * ci: enable debug mode * ci: output proxy logs * chore: formatting * debug ci * debug: more timeout * ci: debug ci with updated dependencies * test(integration): don't run deploy to github in proxy env * ci debug * test(integration): trust crt file * ci: debug ci * chore: add mitmproxy trust script to gitignore * fix(integration-tests): add more URLs to transparent URLs * chore: relock conda * fix: don't reassign os.environ * fix(integration tests): we also need the http versions of the transparent URLs * fix: typo in integration tests, pytest nodist * complete integration tests * ci: debug * fix: add CA cert to git * docs: add comment for Python 3.13 * remove debugger * chore: relock * chore: add conda-lock to .gitattributes * feat(integration tests): dynamically parametrize tests * docs(integration-tests): update graphics * chore: remove explicit truststore dependency * docs(integration-tests): add --dist=no flag to pytest * chore: remove clear runner script * chore: add original draw.io file * fix: typo * fix: formatting Co-authored-by: Pavel Zwerschke <[email protected]> * fix: spelling Co-authored-by: Pavel Zwerschke <[email protected]> * fix: remove pydantic-settings note Co-authored-by: Pavel Zwerschke <[email protected]> * feat: improve instructions in setup wizard * fix: formatting * docs: add note about cloning depth * rename home_directory to initial_working_dir * fix: dir link * fix: get_test_scenario * docs: explain PyCharm warning * docs: clarify lifetime of TestCase objects * docs: refer to README in module docstring * refactor: use monkeypatch.setenv everywhere * docs: add docstring to test_scenario * refactor: make coherent lib module for integration testing * fix: you cannot monkeypatch in module level * fix: imports * fix: circular imports * fix method call * fix filesystem path * relock w/ conda-lock * relock w/ conda-lock * fix integration test 1 * relock * fix: lower pin for conda_forge_feedstock_ops * relock w/ conda-lock * relock * replace svg with markdown * hotfix: pass settings env manually this should eventually replaced by conda-forge/conda-forge-feedstock-ops#49 * fix: base_url setting * slahes in URLs are a good thing * run ruff --------- Co-authored-by: Pavel Zwerschke <[email protected]> Co-authored-by: Matthew R. Becker <[email protected]> Co-authored-by: jaimergp <[email protected]> Co-authored-by: regro-cf-autotick-bot <[email protected]>
1 parent 41ddbe2 commit c39a5d3

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+3780
-180
lines changed

.dockerignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
docs/
2+
tests/
3+
tests_integration/

.github/CODEOWNERS

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
.github/workflows/test-integration.yml @ytausch
2+
conda_forge_tick/contexts.py @ytausch
3+
conda_forge_tick/git_utils.py @ytausch
4+
conda_forge_tick/models/* @ytausch
5+
tests/github_api/* @ytausch
6+
tests/conda_forge_tick/* @ytausch
7+
tests/model/* @ytausch
8+
tests/test_contexts.py @ytausch
9+
tests/test_git_utils.py @ytausch
10+
tests_integration/* @ytausch
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
name: Integration Tests
2+
on:
3+
pull_request:
4+
branches:
5+
- main
6+
merge_group: null
7+
8+
9+
# Integration tests interact with GitHub resources in the integration test infrastructure and therefore
10+
# cannot run concurrently with other integration tests.
11+
concurrency:
12+
group: cf-scripts-integration-tests
13+
cancel-in-progress: false
14+
15+
defaults:
16+
run:
17+
shell: bash -leo pipefail {0}
18+
19+
jobs:
20+
integration-tests:
21+
name: Run Integration Tests
22+
# if triggered by pull_request, only run on non-fork PRs (secrets access needed)
23+
# Nevertheless, this check is always run in the merge queue.
24+
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
25+
runs-on: ubuntu-latest
26+
steps:
27+
- name: Set up Docker Buildx
28+
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
29+
30+
- name: Checkout
31+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
32+
with:
33+
path: cf-scripts
34+
submodules: 'true'
35+
36+
- name: Build Docker Image
37+
uses: docker/build-push-action@471d1dc4e07e5cdedd4c2171150001c434f0b7a4 # v6.15.0
38+
with:
39+
context: cf-scripts
40+
push: false
41+
load: true
42+
tags: conda-forge-tick:test
43+
44+
- name: Setup micromamba
45+
uses: mamba-org/setup-micromamba@0dea6379afdaffa5d528b3d1dabc45da37f443fc # v2.0.4
46+
with:
47+
environment-file: cf-scripts/conda-lock.yml
48+
environment-name: cf-scripts
49+
condarc-file: cf-scripts/autotick-bot/condarc
50+
51+
- name: Run pip install
52+
working-directory: cf-scripts
53+
run: |
54+
pip install --no-deps --no-build-isolation -e .
55+
56+
- name: Run mitmproxy certificate setup wizard
57+
working-directory: cf-scripts
58+
run: |
59+
# place a script in the mitmproxy directory that will be run by the setup wizard
60+
# to trust the mitmproxy certificate
61+
cat <<EOF > ./tests_integration/.mitmproxy/${{ env.MITMPROXY_WIZARD_HEADLESS_TRUST_SCRIPT }}
62+
#!/usr/bin/env bash
63+
set -euo pipefail
64+
sudo cp "\$1" /usr/local/share/ca-certificates/mitmproxy.crt
65+
sudo update-ca-certificates
66+
EOF
67+
68+
./tests_integration/mitmproxy_setup_wizard.sh
69+
env:
70+
MITMPROXY_WIZARD_HEADLESS: true
71+
MITMPROXY_WIZARD_HEADLESS_TRUST_SCRIPT: mitmproxy_trust_script.sh
72+
73+
- name: Set up git identity
74+
run: |
75+
git config --global user.name "regro-cf-autotick-bot-staging"
76+
git config --global user.email "[email protected]"
77+
78+
- name: Run Integration Tests with pytest
79+
working-directory: cf-scripts
80+
run: |
81+
pytest -s -v \
82+
--dist=no \
83+
tests_integration
84+
env:
85+
BOT_TOKEN: ${{ secrets.GH_TOKEN_STAGING_BOT_USER }}
86+
TEST_SETUP_TOKEN: ${{ secrets.GH_TOKEN_STAGING_BOT_USER }}
87+
88+
- name: Print Proxy Logs
89+
run: cat /tmp/mitmproxy.log
90+
if: always()

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ docs/_build/
2121

2222
node_attrs/*
2323
graph.json
24+
!tests_integration/resources/empty-graph/graph.json
2425
pr_json/*
2526
pr_status/*
2627
status/*
@@ -61,3 +62,4 @@ pixi.toml
6162
.repodata_cache/
6263
venv
6364
oryx-build-commands.txt
65+
.env

.gitmodules

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
[submodule "tests_integration/definitions/conda_forge_pinning/resources/feedstock"]
2+
path = tests_integration/lib/_definitions/conda_forge_pinning/resources/feedstock
3+
url = https://github.com/conda-forge/conda-forge-pinning-feedstock.git
4+
[submodule "tests_integration/definitions/pydantic/resources/feedstock"]
5+
path = tests_integration/lib/_definitions/pydantic/resources/feedstock
6+
url = https://github.com/conda-forge/pydantic-feedstock.git
7+
[submodule "tests_integration/definitions/pydantic/resources/feedstock_v1"]
8+
path = tests_integration/lib/_definitions/pydantic/resources/feedstock_v1
9+
url = https://github.com/conda-forge/pydantic-feedstock.git

.pre-commit-config.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,11 @@ repos:
2727
rev: v1
2828
hooks:
2929
- id: typos
30+
exclude: |
31+
(?x)^(
32+
^tests_integration\/definitions\/.*\/resources\/.*|
33+
^docs\/assets\/.*
34+
)$
3035
3136
- repo: https://github.com/astral-sh/ruff-pre-commit
3237
rev: v0.11.8

README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,6 +237,8 @@ If your migrator needs special configuration, you should write a new factory fun
237237
- `CF_FEEDSTOCK_OPS_CONTAINER_NAME`: the name of the container to use in the bot, otherwise defaults to `ghcr.io/regro/conda-forge-tick`
238238
- `CF_FEEDSTOCK_OPS_CONTAINER_TAG`: set this to override the default container tag used in production runs, otherwise the value of `__version__` is used
239239

240+
Additional environment variables are described in [the settings module](conda_forge_tick/settings.py).
241+
240242
### Getting a Working Environment
241243

242244
The bot has an abstract set of requirements stored in the `environment.yml` file in this repo.
@@ -299,6 +301,10 @@ Currently, the following commands are supported and tested:
299301

300302
- `update-upstream-versions`
301303

304+
### Integration Tests
305+
306+
See [tests_integration/README.md](tests_integration/README.md).
307+
302308
### Structure of the Bot's Jobs
303309

304310
#### History

autotick-bot/install_bot_code.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ done
4343
if [[ "${clone_graph}" == "true" ]]; then
4444
cf_graph_repo=${CF_TICK_GRAPH_GITHUB_BACKEND_REPO:-"regro/cf-graph-countyfair"}
4545
cf_graph_remote="https://github.com/${cf_graph_repo}.git"
46+
# please make sure the cloning depth is always identical to the one used in the integration tests (test_integration.py)
4647
git clone --depth=5 "${cf_graph_remote}" cf-graph
4748
else
4849
echo "Skipping cloning of cf-graph"

0 commit comments

Comments
 (0)