[🐛 Bug]: Cannot run selenium java on Linux/Windows against HTTPS based authentication enabled selenium grid

[ambirag]

What happened?

Cannot run selenium java on AWS Linux EC2 / Windows box against HTTPS based authentication enabled selenium grid (this works when authentication is not enabled in the selenium grid)

1. Run sample Selenium Java automation against https authentication enabled selenum grid
2. https://user:password@seleniumgrid.domain.com/wd/hub
3. Obervations:
4. Mac: Same automation code works with the same https authentication enabled selenium grid when automation code run on Mac
5. AWS Linux: Same automation code works against the same selenium grid which does not have authentication enabled on Linux EC2 / Windows

How can we reproduce the issue?

1. Run sample Selenium Java automation on AWS EC2 Linux (amzn2-ami-kernel-5.10-hvm-2.0.20241031.0-x86_64-gp2) OR Windows box, against HTTPS + authentication enabled selenum grid (Selenium grid runs on EKS cluster) 
2. https://user:password@seleniumgrid.domain.com/wd/hub
3. selenium-java - 4.8.2
4. Selenium Grid 4.28.0
5. AWS EC2 - amzn2-ami-kernel-5.10-hvm-2.0.20241031.0-x86_64-gp2 / Windows

Relevant log output

``` INFO: Using OpenTelemetry for tracing
2025-01-02 20:07:22,176 +0000 [main] ERROR (WebDriverInitialize.java:65) - Webdriver initialization failed.
org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. 
Host info: host: 'ip-xx.ec2.internal', ip: 'xx'
Build info: version: '4.8.3', revision: 'e5e76298c3'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.227-219.884.amzn2.x86_64', java.version: '11.0.25'
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*], extensions: []}, goog:loggingPrefs: org.openqa.selenium.logging...}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*], extensions: []}, goog:loggingPrefs: org.openqa.selenium.logging...}}]
Capabilities {browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*], extensions: []}, goog:loggingPrefs: org.openqa.selenium.logging...}
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.cucumber.java.Invoker.doInvoke(Invoker.java:66)
	at io.cucumber.java.Invoker.invokeStatic(Invoker.java:80)
	at io.cucumber.java.AbstractGlueDefinition.invokeMethod(AbstractGlueDefinition.java:45)
	at io.cucumber.java.JavaStaticHookDefinition.execute(JavaStaticHookDefinition.java:49)
	at io.cucumber.core.runner.Runner.executeHook(Runner.java:112)
	at io.cucumber.core.runner.Runner.lambda$executeHooks$2(Runner.java:99)
	at io.cucumber.core.runner.ThrowableCollector.execute(ThrowableCollector.java:21)
	at io.cucumber.core.runner.Runner.executeHooks(Runner.java:99)
	at io.cucumber.core.runner.Runner.runBeforeAllHooks(Runner.java:89)
	at io.cucumber.core.runtime.RethrowingThrowableCollector.executeAndThrow(RethrowingThrowableCollector.java:23)
	at io.cucumber.core.runtime.CucumberExecutionContext.runBeforeAllHooks(CucumberExecutionContext.java:107)
	at io.cucumber.testng.TestNGCucumberRunner.<init>(TestNGCucumberRunner.java:124)
	at io.cucumber.testng.AbstractTestNGCucumberTests.setUpClass(AbstractTestNGCucumberTests.java:27)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:133)
	at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:62)
	at org.testng.internal.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:385)
	at org.testng.internal.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:321)
	at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:176)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:122)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at org.testng.TestRunner.privateRun(TestRunner.java:794)
	at org.testng.TestRunner.run(TestRunner.java:596)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:377)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:371)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:332)
	at org.testng.SuiteRunner.run(SuiteRunner.java:276)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1212)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1134)
	at org.testng.TestNG.runSuites(TestNG.java:1063)
	at org.testng.TestNG.run(TestNG.java:1031)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:284)
	at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:119)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
Caused by: java.io.UncheckedIOException: java.io.IOException: EOF reached while reading
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:296)
	at org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:55)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:124)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:106)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:67)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:165)
	at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:543)
	... 55 more
Caused by: java.io.IOException: EOF reached while reading
	at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:570)
	at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:119)
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:292)
	... 62 more
Caused by: java.io.EOFException: EOF reached while reading
	at java.net.http/jdk.internal.net.http.Http2Connection$Http2TubeSubscriber.onComplete(Http2Connection.java:1608)
	at java.net.http/jdk.internal.net.http.common.SSLTube$DelegateWrapper.onComplete(SSLTube.java:268)
	at java.net.http/jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper.complete(SSLTube.java:432)
	at java.net.http/jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper.onComplete(SSLTube.java:562)
	at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.checkCompletion(SubscriberWrapper.java:443)
	at java.net.http/jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher.run1(SubscriberWrapper.java:322)
	at java.net.http/jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher.run(SubscriberWrapper.java:261)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
	at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.outgoing(SubscriberWrapper.java:234)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader.processData(SSLFlowDelegate.java:434)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher.run(SSLFlowDelegate.java:264)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.592 s <<< FAILURE! - in TestSuite
Time elapsed: 9.436 s  <<< FAILURE!
org.openqa.selenium.SessionNotCreatedException: 
Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. 
Host info: host: 'ip-172-31-43-177.ec2.internal', ip: '172.31.43.177'
Build info: version: '4.8.3', revision: 'e5e76298c3'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.227-219.884.amzn2.x86_64', java.version: '11.0.25'
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*], extensions: []}, goog:loggingPrefs: org.openqa.selenium.logging...}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*], extensions: []}, goog:loggingPrefs: org.openqa.selenium.logging...}}]

### Operating System

AWS Linux/UNIX
Windows

### Selenium version

4.8.2 (Selenium Grid 4.28.0)
selenium-http-jdk-client - 4.8 3

### What are the browser(s) and version(s) where you see this issue?

Chrome 130.0

### What are the browser driver(s) and version(s) where you see this issue?

Selenium Grid

### Are you using Selenium Grid?

Selenium Grid 4.28.0

[github-actions]

@ambirag, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[diemol]

What happens if you try on a local Linux machine? What happens if you use the Docker images?

[VietND96]

I understand that automation code is running on a local machine. In case Grid setup on Mac, those 2 are in the same machine. And in the case of Grid on Linux EC2, local code connects to cloud. Is this correct?

[aragavan1]

I understand that automation code is running on a local machine. In case Grid setup on Mac, those 2 are in the same machine. And in the case of Grid on Linux EC2, local code connects to cloud. Is this correct?
@VietND96
automation code is running on AWS EC2 Linux machine connecting to Grid setup running on AWS EKS using EC2 Linux, both are in AWS, none is running locally except when i run the automation on Mac, it runs on my local Mac against the same grid in AWS. In this case it works without any issues.

[aragavan1]

What happens if you try on a local Linux machine? What happens if you use the Docker images?
@diemol
I don't have a local Linux box, only tried on AWS Linux EC2. The HTTPS auth enabled selenium grid is already using the docker images from docker-selenium for hub and nodes. Its working when i run the automation on Mac against the same selenium grid. i'll update if the same works when running on Windows against the same grid.

[ambirag]

This is happening on Windows as well.

[VietND96]

How did you create the RemoteWebDriver? Are you using Java binding? If yes, refer to this https://www.selenium.dev/documentation/webdriver/drivers/http_client/
By using ClientConfig, set the version HTTP_1_1 to see any help