Skip to content

gh-125669: Do not run -ugui tests by default on make test #125730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 30, 2024
Merged

gh-125669: Do not run -ugui tests by default on make test #125730

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
62ae2d4
Address review
sobolevn Oct 21, 2024
3bf42ab
Add `make ci` target and a NEWS entry
sobolevn Oct 21, 2024
aaa8257
Merge branch 'main' into issue-125669
sobolevn Oct 30, 2024
37e9527
Address review
sobolevn Oct 30, 2024
c7d91a2
Merge branch 'main' into issue-125669
sobolevn Oct 30, 2024
d5ed9a1
Address review
sobolevn Oct 30, 2024
93d2982
Revert windows changes
sobolevn Oct 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -468,7 +468,7 @@ jobs:
- name: Display build info
run: make pythoninfo
- name: Tests
run: xvfb-run make test
run: xvfb-run make ci

build_tsan:
name: 'Thread sanitizer'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/reusable-macos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,4 +69,4 @@ jobs:
--fail-on-improvement
--path-prefix="./"
- name: Tests
run: make test
run: make ci
2 changes: 1 addition & 1 deletion .github/workflows/reusable-ubuntu.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,4 +105,4 @@ jobs:
run: sudo mount "$CPYTHON_RO_SRCDIR" -oremount,rw
- name: Tests
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: xvfb-run make test
run: xvfb-run make ci
2 changes: 1 addition & 1 deletion .github/workflows/reusable-windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,5 +43,5 @@ jobs:
run: >-
.\PCbuild\rt.bat
-p ${{ inputs.arch }}
-d -q --fast-ci
-d -q --fast-ci -ugui
${{ fromJSON(inputs.free-threading) && '--disable-gil' || '' }}
8 changes: 7 additions & 1 deletion Doc/using/configure.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1142,13 +1142,19 @@ make test
^^^^^^^^^

Build the ``all`` target and run the Python test suite with the
``--fast-ci`` option. Variables:
``--fast-ci`` option without GUI tests. Variables:

* ``TESTOPTS``: additional regrtest command-line options.
* ``TESTPYTHONOPTS``: additional Python command-line options.
* ``TESTTIMEOUT``: timeout in seconds (default: 10 minutes).


make ci
^^^^^^^

This is similar to ``make test``, but uses the ``-ugui`` to also run GUI tests.


make buildbottest
^^^^^^^^^^^^^^^^^

Expand Down
6 changes: 6 additions & 0 deletions Makefile.pre.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2104,6 +2104,12 @@ cleantest: all
# Similar to buildbottest, but use --fast-ci option, instead of --slow-ci.
.PHONY: test
test: all
$(TESTRUNNER) --fast-ci -u-gui --timeout=$(TESTTIMEOUT) $(TESTOPTS)

# Run a basic set of regression tests inside the CI.
# This excludes some tests that are particularly resource-intensive.
# Similar to test, but also runs GUI tests.
ci: all
$(TESTRUNNER) --fast-ci --timeout=$(TESTTIMEOUT) $(TESTOPTS)

# Run the test suite for both architectures in a Universal build on OSX.
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Tests/2024-10-21-14-10-56.gh-issue-125730.kcWbvI.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Change ``make test`` to not run GUI tests by default. Use ``make ci`` to run
tests with GUI tests instead.
2 changes: 1 addition & 1 deletion PCbuild/rt.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ set pyname=python
set suffix=
set qmode=
set dashO=
set regrtestargs=--fast-ci
set regrtestargs=--fast-ci -u-gui
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Buildbots run Tools/buildbot/test.bat which runs PCbuild/rt.bat. I'm not sure that this change is correct. Maybe it will disable GUI tests on buildbots.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, thanks for this question. I was not also sure about that. Since I don't ever use windows and not quite sure about how it should work. I will revert this.

set exe=

:CheckOpts
Expand Down
Loading