Skip to content

[tests] Made selenium tests resilient to failures #990

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 19 commits into from
Mar 20, 2025
Merged

[tests] Made selenium tests resilient to failures #990

merged 19 commits into from
Mar 20, 2025

Conversation

@pandafy
Copy link
Member

@pandafy pandafy commented Mar 13, 2025

Checklist

  • I have read the OpenWISP Contributing Guidelines.
  • I have manually tested the changes proposed in this pull request.
  • I have written new test cases for new code and/or updated existing tests for changes to existing code.
  • I have updated the documentation.

@pandafy pandafy force-pushed the update-selenium-tests branch from 1259e7e to 3be9dc9 Compare March 13, 2025 17:49
@pandafy pandafy force-pushed the update-selenium-tests branch from 3be9dc9 to 1989ad6 Compare March 13, 2025 19:10
nemesifier
nemesifier reviewed Mar 13, 2025
View reviewed changes
openwisp_controller/tests/utils.py Outdated
self.web_driver.refresh()
except UnexpectedAlertPresentException:
self.web_driver.switch_to_alert().accept()
self.web_driver.switch_to_alert.accept()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why twice?

openwisp_controller/connection/tests/test_selenium.py Outdated
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from django.test import tag
from django.urls import reverse
import ipdb
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

@pandafy pandafy force-pushed the update-selenium-tests branch 2 times, most recently from 7aa1b0b to c957448 Compare March 14, 2025 06:17
@pandafy pandafy force-pushed the update-selenium-tests branch from c957448 to 990bf08 Compare March 14, 2025 06:24
@pandafy pandafy force-pushed the update-selenium-tests branch from 28fddb8 to 15ed708 Compare March 14, 2025 14:29
@coveralls
Copy link

coveralls commented Mar 14, 2025 •
edited
Loading

Coverage Status

coverage: 98.884% (-0.02%) from 98.9%
when pulling 3071ea3 on update-selenium-tests
into d84383c on master.

pandafy
pandafy commented Mar 14, 2025
View reviewed changes
.github/workflows/ci.yml Outdated
pip install -U pip wheel setuptools
pip install -U -r requirements-test.txt
pip install -U -e .
pip install -UI --no-deps https://github.com/openwisp/openwisp-utils/tarball/browser-logs
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Remove this before merging

Suggested change
pip install -UI --no-deps https://github.com/openwisp/openwisp-utils/tarball/browser-logs

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can remove this now.

nemesifier
nemesifier requested changes Mar 14, 2025
View reviewed changes
openwisp_controller/tests/utils.py Outdated
self._assert_loading_overlay_hidden()

def _assert_loading_overlay_hidden(self):
self.wait_for_invisibility(By.CSS_SELECTOR, '#loading-overlay')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we want to centralize this in openwisp-utils as I added something similar in openwisp-monitoring, but I had to hide the overlay explicitly as doing this was not enough.

@github-project-automation github-project-automation bot moved this from To do (general) to In progress in OpenWISP Contributor's Board Mar 14, 2025
nemesifier
nemesifier requested changes Mar 14, 2025
View reviewed changes
Copy link
Member

@nemesifier nemesifier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Please make ./runtests.py consistent with the firmware-upgrader module (allow running ./runtests.py to run everything).
  • Make sure selenium tests are skipped in SAMPLE_APP=1

@pandafy pandafy force-pushed the update-selenium-tests branch 2 times, most recently from 8c010a8 to 2ea4fe5 Compare March 17, 2025 11:35
@pandafy pandafy force-pushed the update-selenium-tests branch from 2ea4fe5 to 87497fd Compare March 17, 2025 11:45
nemesifier
nemesifier requested changes Mar 17, 2025
View reviewed changes
openwisp_controller/config/tests/test_selenium.py
with self.subTest('Alert should be displayed after making changes'):
# The WebDriver automatically accepts the
# beforeunload confirmation dialog. To verify the message,
# we log it to the console and check its content.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# we log it to the console and check its content.
# we log it to the console and check its content.
#
# our own JS code sets e.returnValue when triggered
# so we just need to ensure it's set as expected

@pandafy pandafy force-pushed the update-selenium-tests branch from ee1b3a5 to fa4d7e9 Compare March 17, 2025 19:58
@pandafy pandafy force-pushed the update-selenium-tests branch 7 times, most recently from 705c03d to 1e4e8dc Compare March 18, 2025 17:41
@pandafy pandafy force-pushed the update-selenium-tests branch from 1e4e8dc to ffef8c1 Compare March 18, 2025 17:54
@nemesifier nemesifier mentioned this pull request Mar 18, 2025
Merged
4 tasks
@pandafy pandafy force-pushed the update-selenium-tests branch 3 times, most recently from 9938eb0 to f70dcab Compare March 19, 2025 19:20
@pandafy pandafy force-pushed the update-selenium-tests branch from f70dcab to 6795f36 Compare March 19, 2025 19:59
@pandafy
Copy link
Member Author

pandafy commented Mar 20, 2025 •
edited
Loading

The random failures that we observed in the following builds were due to port collision in geckodriver.

There's already a PR in selenium which addresses this issue

@pandafy pandafy force-pushed the update-selenium-tests branch from 0df75f8 to 930d094 Compare March 20, 2025 13:44
@nemesifier nemesifier added the bug label Mar 20, 2025
@nemesifier nemesifier merged commit 63eaadf into master Mar 20, 2025
15 checks passed
@nemesifier nemesifier deleted the update-selenium-tests branch March 20, 2025 20:31
@github-project-automation github-project-automation bot moved this from In progress to Done in OpenWISP Contributor's Board Mar 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nemesifier nemesifier nemesifier approved these changes

Assignees

@pandafy pandafy

Labels

bug

Projects

OpenWISP Contributor's Board
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@pandafy @coveralls @nemesifier