[🐛 Bug]: OpenQA.Selenium.WebDriverException : Unexpected error creating WebSocket DevTools session. ----> System.NullReferenceException : Object reference not set to an instance of an object.

[MJB222398]

What happened?

This exception was thrown when setting up my Firefox RemoteWebDriver:

Message: 
OpenQA.Selenium.WebDriverException : Unexpected error creating WebSocket DevTools session.
  ----> System.NullReferenceException : Object reference not set to an instance of an object.

  Stack Trace: 
RemoteWebDriver.GetDevToolsSession(DevToolsOptions options)
RemoteWebDriver.GetDevToolsSession()
<.ctor>b__0()
Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
Lazy`1.CreateValue()
JavaScriptEngine.StartEventMonitoring()

--NullReferenceException
DevToolsSession.SendCommand(String commandName, String sessionId, JsonNode commandParameters, CancellationToken cancellationToken, Nullable`1 millisecondsTimeout, Boolean throwExceptionIfResponseNotReceived)
DevToolsSession.SendCommand(String commandName, JsonNode commandParameters, CancellationToken cancellationToken, Nullable`1 millisecondsTimeout, Boolean throwExceptionIfResponseNotReceived)
DevToolsSession.SendCommand[TCommand,TCommandResponse](TCommand command, CancellationToken cancellationToken, Nullable`1 millisecondsTimeout, Boolean throwExceptionIfResponseNotReceived)
V85Log.Clear()
DevToolsSession.StartSession()
<<GetDevToolsSession>b__0>d.MoveNext()
--- End of stack trace from previous location ---
RemoteWebDriver.GetDevToolsSession(DevToolsOptions options)

How can we reproduce the issue?

This does happen every time since updating from 4.23. It only affects Firefox, Chrome and Edge are working fine as before. This is the code that I have:

var seleniumRemoteWebDriver = new RemoteWebDriver(
        remoteAddress: new Uri(_driverConfiguration.Url),
        capabilities: _driverOptionsFactory.Create(browser).ToCapabilities(),
        commandTimeout: TimeSpan.FromSeconds(_driverConfiguration.CommandTimeout));
seleniumRemoteWebDriver.Manage().Window.Maximize();

var jsEngine = new JavaScriptEngine(seleniumRemoteWebDriver)
jsEngine.JavaScriptConsoleApiCalled += (_, e) => { lock (_consoleLogsLock) { _consoleLogs.Add(e); } };
await jsEngine.StartEventMonitoring();

Not sure how much this matters. If you need more detail of the code let me know. Was hoping the stack trace would be enough.

Relevant log output

Driver logs:

2024-09-10 13:02:30.373 DEBUG HttpCommandExecutor: Executing command: []: newSession {"capabilities":{"firstMatch":[{"browserName":"firefox","moz:firefoxOptions":{"args":["-profile","home/seluser/.mozilla/firefox/seluser_profile"],"prefs":{"remote.active-protocols":3}},"moz:debuggerAddress":true}]}}
2024-09-10 13:02:30.373 TRACE HttpCommandExecutor: >> Method: POST, RequestUri: 'http://localhost:4444/wd/hub/session', Version: 1.1, Content: System.Net.Http.ByteArrayContent, Headers:
{
  Accept: application/json; charset=utf-8
  User-Agent: selenium/4.24.0
  User-Agent: (.net windows)
  Content-Type: application/json; charset=utf-8
  Content-Length: 215
}
{"capabilities":{"firstMatch":[{"browserName":"firefox","moz:firefoxOptions":{"args":["-profile","home/seluser/.mozilla/firefox/seluser_profile"],"prefs":{"remote.active-protocols":3}},"moz:debuggerAddress":true}]}}
2024-09-10 13:02:32.396 TRACE HttpCommandExecutor: << StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
  Cache-Control: no-cache
  Content-Length: 1816
  Content-Type: application/json; charset=utf-8
}
2024-09-10 13:02:32.396 DEBUG HttpCommandExecutor: Response: (ece3e5ae-5545-4f1d-b152-fac12dd8e0b9 Success: System.Collections.Generic.Dictionary`2[System.String,System.Object])
2024-09-10 13:02:32.396 DEBUG HttpCommandExecutor: Executing command: [ece3e5ae-5545-4f1d-b152-fac12dd8e0b9]: maximizeWindow {}
2024-09-10 13:02:32.396 TRACE HttpCommandExecutor: >> Method: POST, RequestUri: 'http://localhost:4444/wd/hub/session/ece3e5ae-5545-4f1d-b152-fac12dd8e0b9/window/maximize', Version: 1.1, Content: System.Net.Http.ByteArrayContent, Headers:
{
  Accept: application/json; charset=utf-8
  User-Agent: selenium/4.24.0
  User-Agent: (.net windows)
  Content-Type: application/json; charset=utf-8
  Content-Length: 2
}
{}
2024-09-10 13:02:32.700 TRACE HttpCommandExecutor: << StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
  Cache-Control: no-cache
  Content-Length: 50
  Content-Type: application/json; charset=utf-8
}
2024-09-10 13:02:32.700 DEBUG HttpCommandExecutor: Response: ( Success: System.Collections.Generic.Dictionary`2[System.String,System.Object])


Logs from Docker container:

2024-09-10 12:01:55,443 INFO Included extra file "/etc/supervisor/conf.d/firefox-cleanup.conf" during parsing
2024-09-10 12:01:55,444 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2024-09-10 12:01:55,446 INFO RPC interface 'supervisor' initialized
2024-09-10 12:01:55,446 INFO supervisord started with pid 8
2024-09-10 12:01:56,448 INFO spawned: 'xvfb' with pid 1189
2024-09-10 12:01:56,449 INFO spawned: 'vnc' with pid 1190
2024-09-10 12:01:56,450 INFO spawned: 'novnc' with pid 1191
2024-09-10 12:01:56,451 INFO spawned: 'selenium-node' with pid 1192
2024-09-10 12:01:56,462 INFO success: selenium-node entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
Appending Selenium options: --log-level CONFIG
Appending Selenium option: --session-timeout 180
Appending Selenium option: --heartbeat-period 30
Selenium option: --log-level already set in env variable SE_OPTS. Ignore new option: --log-level INFO
Appending Selenium option: --http-logs false
Appending Selenium option: --structured-logs false
Generating Selenium Config
Configuring server...
Setting up SE_NODE_HOST...
Setting up SE_NODE_PORT...
Tracing is enabled
Classpath will be enriched with these external jars :  --ext /external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-exporter-otlp/1.40.0/opentelemetry-exporter-otlp-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-netty/1.65.1/grpc-netty-1.65.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-codec-http/4.1.111.Final/netty-codec-http-4.1.111.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-trace/1.40.0/opentelemetry-sdk-trace-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-metrics/1.40.0/opentelemetry-sdk-metrics-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-logs/1.40.0/opentelemetry-sdk-logs-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-exporter-otlp-common/1.40.0/opentelemetry-exporter-otlp-common-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-exporter-sender-okhttp/1.40.0/opentelemetry-exporter-sender-okhttp-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-extension-autoconfigure-spi/1.40.0/opentelemetry-sdk-extension-autoconfigure-spi-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-api/1.65.1/grpc-api-1.65.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-codec-http2/4.1.100.Final/netty-codec-http2-4.1.100.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-core/1.65.1/grpc-core-1.65.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-handler-proxy/4.1.100.Final/netty-handler-proxy-4.1.100.Final.jar:/external_jars/https/repo1.maven.org/maven2/com/google/guava/guava/32.1.3-android/guava-32.1.3-android.jar:/external_jars/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.23.0/error_prone_annotations-2.23.0.jar:/external_jars/https/repo1.maven.org/maven2/io/perfmark/perfmark-api/0.26.0/perfmark-api-0.26.0.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-transport-native-unix-common/4.1.111.Final/netty-transport-native-unix-common-4.1.111.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-util/1.65.1/grpc-util-1.65.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-common/4.1.111.Final/netty-common-4.1.111.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-buffer/4.1.111.Final/netty-buffer-4.1.111.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-transport/4.1.111.Final/netty-transport-4.1.111.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-codec/4.1.111.Final/netty-codec-4.1.111.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-handler/4.1.111.Final/netty-handler-4.1.111.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-api/1.40.0/opentelemetry-api-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk-common/1.40.0/opentelemetry-sdk-common-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-api-incubator/1.40.0-alpha/opentelemetry-api-incubator-1.40.0-alpha.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-exporter-common/1.40.0/opentelemetry-exporter-common-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/com/squareup/okhttp3/okhttp/4.12.0/okhttp-4.12.0.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-sdk/1.40.0/opentelemetry-sdk-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar:/external_jars/https/repo1.maven.org/maven2/com/google/code/gson/gson/2.10.1/gson-2.10.1.jar:/external_jars/https/repo1.maven.org/maven2/com/google/android/annotations/4.1.1.4/annotations-4.1.1.4.jar:/external_jars/https/repo1.maven.org/maven2/org/codehaus/mojo/animal-sniffer-annotations/1.23/animal-sniffer-annotations-1.23.jar:/external_jars/https/repo1.maven.org/maven2/io/grpc/grpc-context/1.65.1/grpc-context-1.65.1.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-codec-socks/4.1.100.Final/netty-codec-socks-4.1.100.Final.jar:/external_jars/https/repo1.maven.org/maven2/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/external_jars/https/repo1.maven.org/maven2/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/external_jars/https/repo1.maven.org/maven2/org/checkerframework/checker-qual/3.37.0/checker-qual-3.37.0.jar:/external_jars/https/repo1.maven.org/maven2/com/google/j2objc/j2objc-annotations/2.8/j2objc-annotations-2.8.jar:/external_jars/https/repo1.maven.org/maven2/io/netty/netty-resolver/4.1.111.Final/netty-resolver-4.1.111.Final.jar:/external_jars/https/repo1.maven.org/maven2/io/opentelemetry/opentelemetry-context/1.40.0/opentelemetry-context-1.40.0.jar:/external_jars/https/repo1.maven.org/maven2/com/squareup/okio/okio/3.6.0/okio-3.6.0.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-jdk8/1.9.10/kotlin-stdlib-jdk8-1.9.10.jar:/external_jars/https/repo1.maven.org/maven2/com/squareup/okio/okio-jvm/3.6.0/okio-jvm-3.6.0.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib/1.9.10/kotlin-stdlib-1.9.10.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-jdk7/1.9.10/kotlin-stdlib-jdk7-1.9.10.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-common/1.9.10/kotlin-stdlib-common-1.9.10.jar:/external_jars/https/repo1.maven.org/maven2/org/jetbrains/annotations/13.0/annotations-13.0.jar
List arguments for OpenTelemetry:  -Dotel.resource.attributes=service.name=selenium-node-firefox -Dotel.traces.exporter=otlp -Dotel.java.global-autoconfigure.enabled=true
Selenium Grid Node configuration: 
[events]
publish = "tcp://selenium-hub:4442"
subscribe = "tcp://selenium-hub:4443"

[node]
grid-url = "http://localhost:4444"
session-timeout = "180"
override-max-sessions = false
detect-drivers = false
drain-after-session-count = 0
max-sessions = 1

[[node.driver-configuration]]
display-name = "firefox"
stereotype = '{"browserName": "firefox", "browserVersion": "130.0", "platformName": "Linux", "moz:firefoxOptions": {"binary": "/usr/bin/firefox"}, "se:containerName": ""}'
max-sessions = 1
    
Starting Selenium Grid Node...
12:01:57.169 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
12:01:57.179 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
12:01:57.403 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://selenium-hub:4442 and tcp://selenium-hub:4443
12:01:57.450 INFO [UnboundZmqEventBus.<init>] - Sockets created
2024-09-10 12:01:57,450 INFO success: xvfb entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-09-10 12:01:57,450 INFO success: vnc entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-09-10 12:01:57,450 INFO success: novnc entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
12:01:58.451 INFO [UnboundZmqEventBus.<init>] - Event bus ready
12:01:58.511 INFO [NodeServer.createHandlers] - Reporting self as: http://172.18.0.4:5555
12:01:58.527 INFO [NodeOptions.getSessionFactories] - Detected 20 available processors
12:01:58.555 INFO [NodeOptions.report] - Adding firefox for {"browserName": "firefox","browserVersion": "130.0","moz:firefoxOptions": {"binary": "\u002fusr\u002fbin\u002ffirefox"},"platformName": "linux","se:containerName": "","se:noVncPort": 7900,"se:vncEnabled": true} 1 times
12:01:58.568 INFO [Node.<init>] - Binding additional locator mechanisms: relative
12:01:58.685 INFO [NodeServer$1.start] - Starting registration process for Node http://172.18.0.4:5555
12:01:58.687 INFO [NodeServer.execute] - Started Selenium node 4.24.0 (revision 748ffc9bc3): http://172.18.0.4:5555
12:01:58.698 INFO [NodeServer$1.lambda$start$1] - Sending registration event...
12:01:58.785 INFO [NodeServer.lambda$createHandlers$2] - Node has been added
12:02:02.375 WARN [ThrottlingLogger.doLog] - Failed to export spans. Server responded with gRPC status code 2. Error message: Failed to connect to localhost/[0:0:0:0:0:0:0:1]:4317
12:02:32.379 WARN [ThrottlingLogger.doLog] - Failed to export spans. Server responded with gRPC status code 2. Error message: Failed to connect to localhost/[0:0:0:0:0:0:0:1]:4317
12:02:32.385 INFO [LocalNode.newSession] - Session created by the Node. Id: ece3e5ae-5545-4f1d-b152-fac12dd8e0b9, Caps: Capabilities {acceptInsecureCerts: false, browserName: firefox, browserVersion: 130.0, moz:accessibilityChecks: false, moz:buildID: 20240829075237, moz:debuggerAddress: 127.0.0.1:23450, moz:firefoxOptions: {args: [-profile, home/seluser/.mozilla/firef...], prefs: {remote.active-protocols: 3}}, moz:geckodriverVersion: 0.35.0, moz:headless: false, moz:platformVersion: 5.15.146.1-microsoft-standa..., moz:processID: 1662, moz:profile: /home/seluser/.mozilla/fire..., moz:shutdownTimeout: 60000, moz:webdriverClick: true, moz:windowless: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://localhost:4444/session..., se:cdpVersion: 85.0, se:containerName: , se:noVncPort: 7900, se:vnc: ws://localhost:4444/session..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.4:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, userAgent: Mozilla/5.0 (X11; Linux x86...}
12:02:32.726 INFO [ProxyNodeWebsockets.createWsEndPoint] - Establishing connection to ws://127.0.0.1:23450/devtools/browser/b8e7f9ae-d105-46cb-b830-d8c57e83f515
12:02:37.381 WARN [ThrottlingLogger.doLog] - Failed to export spans. Server responded with gRPC status code 2. Error message: Failed to connect to localhost/[0:0:0:0:0:0:0:1]:4317

Operating System

Windows 10

Selenium version

4.24

What are the browser(s) and version(s) where you see this issue?

Firefox

What are the browser driver(s) and version(s) where you see this issue?

Firefox

Are you using Selenium Grid?

Yes 4.24.0-20240907

[github-actions]

@MJB222398, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[MJB222398]

Hi @nvborisenko is there any update on this? I cannot upgrade to 4.24 as it stands

[nvborisenko]

I see the following even locally

1727099359594   CDP     WARN    Support for the Chrome DevTools Protocol (CDP) in Firefox will be deprecated after Firefox 128 (ESR) and will be removed in a later release. CDP users should consider migrating to WebDriver BiDi. See https://bugzilla.mozilla.org/show_bug.cgi?id=1872254
1727099359594   CDP     WARN    Invalid browser preferences for CDP. Set "fission.webContentIsolationStrategy"to 0 and "fission.bfcacheInParent" to false before Firefox starts.
1727099359629   CDP     WARN    Support for the Chrome DevTools Protocol (CDP) in Firefox will be deprecated after Firefox 128 (ESR) and will be removed in a later release. CDP users should consider migrating to WebDriver BiDi. See https://bugzilla.mozilla.org/show_bug.cgi?id=1872254
1727099359629   CDP     WARN    Invalid browser preferences for CDP. Set "fission.webContentIsolationStrategy"to 0 and "fission.bfcacheInParent" to false before Firefox starts.
console.error: (new UnknownMethodError("Log.clear", "chrome://remote/content/cdp/Error.sys.mjs", 18))
1727099359638   CDP     ERROR   UnknownMethodError: Log.clear:
        RemoteAgentError@chrome://remote/content/cdp/Error.sys.mjs:18:5
        UnknownMethodError@chrome://remote/content/cdp/Error.sys.mjs:101:7
        execute@chrome://remote/content/cdp/domains/DomainCache.sys.mjs:92:13
        receiveMessage@chrome://remote/content/cdp/sessions/ContentProcessSession.sys.mjs:79:45
caused by: 
Unhandled exception. OpenQA.Selenium.WebDriverException: Unexpected error creating WebSocket DevTools session.
 ---> System.NullReferenceException: Object reference not set to an instance of an object.

I think your options here are:

• Stick to Firefox 128
• Or migrate to BiDi

The first preview is available in Selenium v4.25, something like the following:

var options = new FirefoxOptions() { UseWebSocketUrl = true };

using var driver = new FirefoxDriver(options);

var bidi = await driver.AsBiDiAsync();
await bidi.Log.OnEntryAddedAsync(e => Console.WriteLine(e));

driver.Url = "https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html";
driver.FindElement(By.Id("consoleLog")).Click();

Please try it out and leave feedback.

[MJB222398]

@nvborisenko Would like to do the BiDi upgrade, wasn't aware it was ready enough yet. Trying out the above for FF is working well on first glance. However for Edge and Chrome it is not:

System.InvalidOperationException : Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: unknown error: Failed to initialize BiDi Mapper: TypeError: Failed to set the 'innerHTML' property on 'Element': This document requires 'TrustedHTML' assignment.
    at generatePage (<anonymous>:888:1056)
    at <anonymous>:906:295
    at <anonymous>:906:1073
  (Session info: chrome=129.0.6668.58) 
Host info: host: '20e30ca0b129', ip: '172.18.0.5'
Build info: version: '4.25.0', revision: '030fcf7918'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.146.1-microsoft-standard-WSL2', java.version: '17.0.12'
Driver info: driver.version: unknown
Build info: version: '4.25.0', revision: '030fcf7918'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.146.1-microsoft-standard-WSL2', java.version: '17.0.12'
Driver info: driver.version: unknown
Build info: version: '4.25.0', revision: '030fcf7918'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.146.1-microsoft-standard-WSL2', java.version: '17.0.12'
Driver info: driver.version: unknown (SessionNotCreated)

[nvborisenko]

Like this one: #14299

[nvborisenko]

I adjusted code slightly to avoid NullReferenceExcception which helps to understand what was wrong in terms of CDP commands.

[MJB222398]

@nvborisenko Thanks you're right thats the issue. Yes definitely makes sense to guard against the NRE, thanks

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.