[🐛 Bug]: Cannot use BiDi with Selenium Grid trying multiple configurations and the RemoteWebDriver builder. This bug seems related to #16263

[MohabMohie]

Description

I can confirm similar behavior to that mentioned in #16263 on my end.

Summary:

• The code compiles properly.
• The session gets assigned from the grid side.
• The java binding throws an Exception org.openqa.selenium.remote.http.ConnectionFailedException: JdkWebSocket initial request execution error

Other observations:

• When webSocketUrl=true is not passed, the same behavior happens.
• when webSocketUrl=true is not passed, and SE_NODE_GRID_URL=http://selenium-hub:4444 is removed, the same behavior happens.
• When the builder is not used, and instead is replaced by legacy code new RemoteWebDriver(new java.net.URI(targetExecutionUrl).toURL(), capabilities); the session is created successfully in all cases, YET if I try to check if (webDriver instanceOf HasBiDi) it is always FALSE. I tried with webSocketUrl=true or not, and with the SE_NODE_GRID_URL set or not.
• In all the above (and the main issue) there is one constant: I am unable to use BiDi with Selenium Grid.

Code:

RemoteWebDriver.builder().address(targetExecutionUrl).oneOf(capabilities).build();

Actual Behavior:

Session gets created from the grid's side, meaning that the capabilities have matched and were sent correctly. But then the exception is thrown.

Grid config:

NOTE This item is added to grid config as per the official user guide SE_NODE_GRID_URL=http://selenium-hub:4444

services:
  chrome:
    image: selenium/node-chrome:4.36.0-20251001
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_NODE_GRID_URL=http://selenium-hub:4444

  edge:
    image: selenium/node-edge:4.36.0-20251001
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_NODE_GRID_URL=http://selenium-hub:4444

  firefox:
    image: selenium/node-firefox:4.36.0-20251001
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_NODE_GRID_URL=http://selenium-hub:4444

  selenium-hub:
    image: selenium/hub:4.36.0-20251001
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

Grid logs:

selenium-hub  | 20:21:38.705 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
selenium-hub  |  [Capabilities {browserName: chrome, goog:chromeOptions: {args: [--headless, --remote-debugging-port=55261, --window-position=0,0, --window-size=1920,1080], extensions: []}, pageLoadStrategy: none, platformName: linux, timeouts: {pageLoad: 30000, script: 30000}, webSocketUrl: true}]
chrome-1      | 20:21:39.728 INFO [LocalNode.newSession] - Session created by the Node. Id: 4e2dd6e757623de4031771d7aa277d3f, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 141.0.7390.54, chrome: {chromedriverVersion: 141.0.7390.54 (b95610d5c4a5..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:55261}, networkConnectionEnabled: false, pageLoadStrategy: none, platformName: linux, proxy: Proxy(), se:cdp: ws://selenium-hub:4444/sess..., se:cdpVersion: 141.0.7390.54, se:containerName: 5734b3e3714c, se:deleteSessionOnUi: true, se:downloadsEnabled: true, se:gridWebSocketUrl: ws://localhost:23735/sessio..., se:noVncPort: 7900, se:vnc: ws://selenium-hub:4444/sess..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.3:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 30000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webSocketUrl: ws://selenium-hub:4444/sess..., webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
selenium-hub  | 20:21:39.744 INFO [LocalSessionMap.add] - Added session to local Session Map, Id: 4e2dd6e757623de4031771d7aa277d3f, Node: http://172.18.0.3:5555
selenium-hub  | 20:21:39.747 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 4e2dd6e757623de4031771d7aa277d3f 
selenium-hub  |  Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 141.0.7390.54, chrome: {chromedriverVersion: 141.0.7390.54 (b95610d5c4a5..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:55261}, networkConnectionEnabled: false, pageLoadStrategy: none, platformName: linux, proxy: {}, se:cdp: ws://selenium-hub:4444/sess..., se:cdpVersion: 141.0.7390.54, se:containerName: 5734b3e3714c, se:deleteSessionOnUi: true, se:downloadsEnabled: true, se:gridWebSocketUrl: ws://localhost:23735/sessio..., se:noVncPort: 7900, se:vnc: ws://selenium-hub:4444/sess..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.3:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 30000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webSocketUrl: ws://selenium-hub:4444/sess..., webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}

Exception stacktrace:

org.openqa.selenium.remote.http.ConnectionFailedException: JdkWebSocket initial request execution error
Build info: version: '4.36.0', revision: '6d115cfd87'
System info: os.name: 'Windows 11', os.arch: 'amd64', os.version: '10.0', java.version: '21.0.8'
Driver info: driver.version: SHAFT$GUI$WebDriver
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.openSocket(JdkHttpClient.java:253)
	at org.openqa.selenium.bidi.Connection.(Connection.java:90)
	at org.openqa.selenium.bidi.BiDiProvider.getImplementation(BiDiProvider.java:53)
	at org.openqa.selenium.bidi.BiDiProvider.getImplementation(BiDiProvider.java:31)
	at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:202)
	at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:173)
	at org.openqa.selenium.remote.RemoteWebDriverBuilder.build(RemoteWebDriverBuilder.java:382)
	at com.shaft.driver.internal.DriverFactory.DriverFactoryHelper.connectToRemoteServer(DriverFactoryHelper.java:241)
	at com.shaft.driver.internal.DriverFactory.DriverFactoryHelper.attemptRemoteServerConnection(DriverFactoryHelper.java:176)
	at com.shaft.driver.internal.DriverFactory.DriverFactoryHelper.setRemoteDriverInstance(DriverFactoryHelper.java:512)
	... 44 more
Caused by: java.net.ConnectException
	at java.net.http/jdk.internal.net.http.common.Utils.toConnectException(Utils.java:1065)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:227)
	at java.net.http/jdk.internal.net.http.Http1Exchange.sendHeadersAsync(Http1Exchange.java:312)
	at java.net.http/jdk.internal.net.http.Exchange.lambda$responseAsyncImpl0$8(Exchange.java:581)
	at java.net.http/jdk.internal.net.http.Exchange.checkFor407(Exchange.java:452)
	at java.net.http/jdk.internal.net.http.Exchange.lambda$responseAsyncImpl0$9(Exchange.java:585)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
	at java.base/java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:950)
	at java.base/java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2372)
	at java.net.http/jdk.internal.net.http.Exchange.responseAsyncImpl0(Exchange.java:585)
	at java.net.http/jdk.internal.net.http.Exchange.responseAsyncImpl(Exchange.java:428)
	at java.net.http/jdk.internal.net.http.Exchange.responseAsync(Exchange.java:420)
	at java.net.http/jdk.internal.net.http.MultiExchange.responseAsyncImpl(MultiExchange.java:435)
	at java.net.http/jdk.internal.net.http.MultiExchange.lambda$responseAsyncImpl$7(MultiExchange.java:476)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
	at java.base/java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:950)
	at java.base/java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2372)
	at java.net.http/jdk.internal.net.http.MultiExchange.responseAsyncImpl(MultiExchange.java:466)
	at java.net.http/jdk.internal.net.http.MultiExchange.lambda$responseAsync0$2(MultiExchange.java:355)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run$$$capture(CompletableFuture.java:1773)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java)
	at --- Async.Stack.Trace --- (captured by IntelliJ IDEA debugger)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.(CompletableFuture.java:1754)
	at java.base/java.util.concurrent.CompletableFuture.completeAsync(CompletableFuture.java:2719)
	at java.net.http/jdk.internal.net.http.MultiExchange.responseAsync(MultiExchange.java:308)
	at java.net.http/jdk.internal.net.http.HttpClientImpl.sendAsync(HttpClientImpl.java:1053)
	at java.net.http/jdk.internal.net.http.HttpClientImpl.sendAsync(HttpClientImpl.java:991)
	at java.net.http/jdk.internal.net.http.HttpClientImpl.sendAsync(HttpClientImpl.java:983)
	at java.net.http/jdk.internal.net.http.HttpClientFacade.sendAsync(HttpClientFacade.java:143)
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.lambda$send$0(OpeningHandshake.java:192)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:319)
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.send(OpeningHandshake.java:194)
	at java.net.http/jdk.internal.net.http.websocket.WebSocketImpl.newInstanceAsync(WebSocketImpl.java:143)
	at java.net.http/jdk.internal.net.http.websocket.BuilderImpl.buildAsync(BuilderImpl.java:120)
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.openSocket(JdkHttpClient.java:177)
	... 53 more
Caused by: java.nio.channels.UnresolvedAddressException
	at java.base/sun.nio.ch.Net.checkAddress(Net.java:137)
	at java.base/sun.nio.ch.Net.checkAddress(Net.java:145)
	at java.base/sun.nio.ch.SocketChannelImpl.checkRemote(SocketChannelImpl.java:842)
	at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:865)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$1(PlainHttpConnection.java:210)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:571)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:212)
	... 88 more

Reproducible Code

RemoteWebDriver.builder().address(targetExecutionUrl).oneOf(capabilities).build();

[selenium-ci]

@MohabMohie, thank you for creating this issue. We will troubleshoot it as soon as we can.

Selenium Triage Team: remember to follow the Triage Guide

[MohabMohie]

several ai-bot chats later, there seems to be some issues with my grid configuration. this is the correct grid configuration that should be used for regular grids.

services:
  chrome:
    image: selenium/node-chrome:4.36.0-20251001
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_NODE_GRID_URL=http://localhost:4444
      - SE_NODE_CONNECTION_LIMIT_PER_SESSION=1000

  edge:
    image: selenium/node-edge:4.36.0-20251001
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_NODE_GRID_URL=http://localhost:4444
      - SE_NODE_CONNECTION_LIMIT_PER_SESSION=1000

  firefox:
    image: selenium/node-firefox:4.36.0-20251001
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_NODE_GRID_URL=http://localhost:4444
      - SE_NODE_CONNECTION_LIMIT_PER_SESSION=1000

  selenium-hub:
    image: selenium/hub:4.36.0-20251001
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

Note these two:

• SE_NODE_GRID_URL=http://localhost:4444 => needs to be explicitly set to localhost for selenium grid instances that are hosted using GitHub Actions or locally without a public-ip.
• SE_NODE_CONNECTION_LIMIT_PER_SESSION=1000 => needs to be increased from the default 10 so I went for an overkill.

This code block works as expected to add BiDi augmentation to a RemoteWebDriver object that was created the old (constructor not builder) way:

var driver = new RemoteWebDriver(new URI(targetExecutionUrl).toURL(), capabilities);
var augmenter = new Augmenter();
augmenter.addDriverAugmentation(new BiDiProvider());
return augmenter.augment(driver);

I did not test the new builder yet with this approach. And I believe that having "A way" to get BiDi working with the Grid is what this issue is about, so I believe it's safe to close it now.

[MohabMohie]

@pujagani quick tag for your reference. Maybe this can help you out with other similar issues.