Skip to content

Conversation

@titusfortner
Copy link
Member

@titusfortner titusfortner commented Jan 11, 2026

User description

🔗 Related Issues

Implements #16879 for Java

💥 What does this PR do?

Turns on debugging for reruns

🔧 Implementation Notes

I'm honestly not sure if this is needed since it already buffers the logging, but checking to see if this gives more output.

Turns out that enabling debug mode gave us 833 lines of debug vs 286, so it's worth doing in Java as well. (https://github.com/SeleniumHQ/selenium/actions/runs/20899827993/job/60044011581). Looks like it is giving remote server information when running against grid.


PR Type

Enhancement, Tests


Description

  • Enable debug mode for Java test reruns to capture detailed logs

  • Reduce initial flaky test attempts from 3 to 2 across CI jobs

  • Add rerun-with-debug flag to browser and remote test workflows

  • Improve debugging visibility for grid-related test failures


Diagram Walkthrough

flowchart LR
  A["CI Workflow Jobs"] -->|Add rerun-with-debug flag| B["Debug Mode Enabled"]
  A -->|Reduce flaky attempts| C["Faster Test Cycles"]
  B -->|Capture detailed logs| D["Better Failure Analysis"]
  C -->|2 attempts instead of 3| D
Loading

File Walkthrough

Relevant files
B-build
ci-java.yml
Enable debug mode and optimize flaky test attempts             

.github/workflows/ci-java.yml

  • Added rerun-with-debug: true parameter to three test job
    configurations
  • Reduced --flaky_test_attempts from 3 to 2 in all affected jobs
  • Applied changes to Chrome browser tests on Windows and macOS
  • Applied changes to remote Chrome tests on macOS
+6/-3     

@selenium-ci selenium-ci added C-java Java Bindings B-build Includes scripting, bazel and CI integrations labels Jan 11, 2026
@titusfortner titusfortner marked this pull request as ready for review January 12, 2026 15:54
@titusfortner titusfortner requested a review from diemol January 12, 2026 15:55
@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Jan 12, 2026

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🟡
🎫 #1234
🔴 Ensure that click() triggers JavaScript in a link href (e.g., javascript: URL) in Selenium
2.48.x as it did in 2.47.1 when using Firefox (reported with Firefox 42.0).
Provide/enable a fix or regression resolution so the provided test case alerts as
expected.
🟡
🎫 #5678
🔴 Address repeated ChromeDriver instantiation failures that produce "Error: ConnectFailure
(Connection refused)" after the first instance on Ubuntu (as described).
Provide a fix or guidance change that prevents or mitigates the connection refused errors
for subsequent driver instantiations.
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status:
Debug output exposure: Enabling rerun-with-debug: true may increase CI log verbosity and potentially surface
internal stack traces or environment details, which should be verified as not user-facing
and not externally exposed beyond intended CI logs.

Referred Code
rerun-with-debug: true
run: >
  bazel test --flaky_test_attempts 2
  //java/test/org/openqa/selenium/chrome:ChromeDriverFunctionalTest

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status:
Possible sensitive logs: The added rerun-with-debug: true and increased test verbosity could emit sensitive
infrastructure details (e.g., grid endpoints/remote server info) into CI logs, requiring
verification that no secrets/tokens/PII are logged at debug level.

Referred Code
    rerun-with-debug: true
    run: >
      bazel test --flaky_test_attempts 2
      //java/test/org/openqa/selenium/chrome:ChromeDriverFunctionalTest
      //java/test/org/openqa/selenium/federatedcredentialmanagement:FederatedCredentialManagementTest
      //java/test/org/openqa/selenium/firefox:FirefoxDriverBuilderTest
      //java/test/org/openqa/selenium/grid/router:RemoteWebDriverDownloadTest
      //java/test/org/openqa/selenium/remote:RemoteWebDriverBuilderTest
      //java/test/org/openqa/selenium/grid/router:RemoteWebDriverDownloadTest
      //java/test/org/openqa/selenium/devtools:NetworkInterceptorRestTest

browser-tests-macos:
  name: Browser Tests
  uses: ./.github/workflows/bazel.yml
  strategy:
    fail-fast: false
    matrix:
      include:
        - os: macos
  with:
    name: Browser Tests (chrome, ${{ matrix.os }})


 ... (clipped 34 lines)

Learn more about managing compliance generic rules or creating your own custom rules

  • Update
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Jan 12, 2026

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Stream failing test logs

Add the --test_output=errors flag to the bazel test command to stream detailed
failure logs for easier debugging.

.github/workflows/ci-java.yml [28]

-bazel test --flaky_test_attempts 2
+bazel test --flaky_test_attempts 2 --test_output=errors
  • Apply / Chat
Suggestion importance[1-10]: 7

__

Why: This is a valuable suggestion for improving debuggability by adding --test_output=errors to the bazel test command, which complements the PR's goal of handling flaky tests.

Medium
Learned
best practice
Gate debug behavior by input

Avoid hard-coding debug reruns on every CI run; gate it behind a workflow input
(with a safe default) so it’s only enabled when explicitly requested.

.github/workflows/ci-java.yml [26]

-rerun-with-debug: true
+rerun-with-debug: ${{ inputs.rerun_with_debug == true }}
  • Apply / Chat
Suggestion importance[1-10]: 5

__

Why:
Relevant best practice - Add explicit validation/guards at integration boundaries (GitHub Actions contexts/inputs) instead of assuming presence/intent.

Low
  • Update

@titusfortner titusfortner merged commit 7b98b24 into trunk Jan 13, 2026
21 checks passed
@titusfortner titusfortner deleted the java_rerun branch January 13, 2026 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B-build Includes scripting, bazel and CI integrations C-java Java Bindings Review effort 2/5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants