[🐛 Bug]: CfT and locally installed Chrome behave differently

[dmke]

Description

Problem Description

I have Chrome installed locally (/usr/bin/google-chrome --version yields "Google Chrome 135.0.7049.84").

When running specs (via rspec → capybara → selenium-webdriver) I found differences between the following commands:

1. SE_BROWSER_VERSION=135 rspec - uses local installation
2. SE_FORCE_BROWSER_DOWNLOAD=true rspec - downloads and uses Chrome-for-Testing v135.x

The problem with the local installed Chrome is that it reproducibly fails to close the browser windows at the end when the page embeds a PDF file (via <object type="application/pdf" data="/path/to.pdf" />), yielding a Selenium::WebDriver::Error::UnknownError: failed to close window in 20 seconds.

SE_BROWSER_VERSION=135 rspec

rspec log

$ SE_BROWSER_VERSION=135 rspec spec/features/some_browser_spec.rb
Capybara starting Puma...
* Version 6.6.0, codename: Return to Forever
* Min threads: 0, max threads: 4
* Listening on http://127.0.0.1:3952
F

Failures:

  1) Some Browser
     Failure/Error: example.run
     
     Selenium::WebDriver::Error::UnknownError:
       unknown error: failed to close window in 20 seconds
         (Session info: chrome=135.0.7049.84)
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/response.rb:63:in `add_cause'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/response.rb:41:in `error'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/response.rb:52:in `assert_ok'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/response.rb:34:in `initialize'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/http/common.rb:103:in `new'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/http/common.rb:103:in `create_response'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/http/default.rb:103:in `request'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/http/common.rb:68:in `call'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/bridge.rb:625:in `execute'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/remote/bridge.rb:219:in `close'
     # ./.gems/3.2/gems/selenium-webdriver-4.31.0/lib/selenium/webdriver/common/driver.rb:192:in `close'
     # ./spec/support/rack_attack.rb:11:in `block (2 levels) in <main>'
     # ./.gems/3.2/gems/webmock-3.25.1/lib/webmock/rspec.rb:39:in `block (2 levels) in <main>'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `loop'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # ./spec/spec_helper.rb:30:in `block (2 levels) in <top (required)>'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `loop'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # ./.gems/3.2/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block (2 levels) in setup'
     # ------------------
     # --- Caused by: ---
     # Selenium::WebDriver::Error::WebDriverError:
     #   #0 0x558199d1fd0a <unknown>
#1 0x5581997d05f0 <unknown>
#2 0x5581997a951f <unknown>
#3 0x5581997a958c <unknown>
#4 0x55819985919a <unknown>
#5 0x558199847628 <unknown>
#6 0x558199813ea3 <unknown>
#7 0x558199814b01 <unknown>
#8 0x558199ce4b5b <unknown>
#9 0x558199ce8a41 <unknown>
#10 0x558199ccbc52 <unknown>
#11 0x558199ce95b4 <unknown>
#12 0x558199caff0f <unknown>
#13 0x558199d0ddb8 <unknown>
#14 0x558199d0df96 <unknown>
#15 0x558199d1eb56 <unknown>
#16 0x7faa2da731f5 <unknown>


Finished in 23.96 seconds (files took 1.55 seconds to load)
1 example, 1 failure

selenium log (excerpt)

2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Selenium Manager binary found at $PROJECT_ROOT/web/.gems/3.2/gems/selenium-webdriver-4.31.0/bin/linux/selenium-manager 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Executing Process ["$PROJECT_ROOT/.gems/3.2/gems/selenium-webdriver-4.31.0/bin/linux/selenium-manager", "--browser", "chrome", "--browser-version", "135", "--language-binding", "ruby", "--output", "json", "--debug"] 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Sending stats to Plausible: Props { browser: "chrome", browser_version: "135", os: "linux", arch: "x86_64", lang: "ruby", selenium_version: "4.31" } 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] chromedriver not found in PATH 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] chrome detected at /usr/bin/google-chrome 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Running command: /usr/bin/google-chrome --version 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Output: "Google Chrome 135.0.7049.84 " 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Detected browser: chrome 135.0.7049.84 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Required driver: chromedriver 135.0.7049.84 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] chromedriver 135.0.7049.84 already in the cache 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Driver path: $HOME/.cache/selenium/chromedriver/linux64/135.0.7049.84/chromedriver 
2025-04-14 10:40:06 DEBUG Selenium [:selenium_manager] Browser path: /usr/bin/google-chrome 
2025-04-14 10:40:06 DEBUG Selenium [:driver_service] Executing Process ["$HOME/.cache/selenium/chromedriver/linux64/135.0.7049.84/chromedriver", "--port=9515"] 
2025-04-14 10:40:06 DEBUG Selenium [:process] Starting process: ["$HOME/.cache/selenium/chromedriver/linux64/135.0.7049.84/chromedriver", "--port=9515"] with {[:out, :err]=>#<File:$PROJECT_ROOT/web/log/selenium.log>, :pgroup=>true} 
2025-04-14 10:40:06 DEBUG Selenium [:process]   -> pid: 80533 
2025-04-14 10:40:06 DEBUG Selenium [:driver_service] polling for socket on ["127.0.0.1", 9515] 
Starting ChromeDriver 135.0.7049.84 (6c019e56001911b3fd467e03bf68c435924d62f4-refs/branch-heads/7049@{#1778}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully on port 9515.

...

2021-07-15 00:00:00 DEBUG Selenium [:command] -> GET session/3195b1f3306c45c14436b264c349a71a/window/handles 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":["C98B509BA8E0DC57AAFE2CCF3102BCDF","653EED4BB7E8FC5A3DE3992781C80ECC"]} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> POST session/3195b1f3306c45c14436b264c349a71a/window 
2021-07-15 00:00:00 DEBUG Selenium [:command]    >>> http://127.0.0.1:9515/session/3195b1f3306c45c14436b264c349a71a/window | {"handle":"C98B509BA8E0DC57AAFE2CCF3102BCDF"} 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":null} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> GET session/3195b1f3306c45c14436b264c349a71a/window/handles 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":["C98B509BA8E0DC57AAFE2CCF3102BCDF","653EED4BB7E8FC5A3DE3992781C80ECC"]} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> GET session/3195b1f3306c45c14436b264c349a71a/window/handles 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":["C98B509BA8E0DC57AAFE2CCF3102BCDF","653EED4BB7E8FC5A3DE3992781C80ECC"]} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> GET session/3195b1f3306c45c14436b264c349a71a/window 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":"C98B509BA8E0DC57AAFE2CCF3102BCDF"} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> POST session/3195b1f3306c45c14436b264c349a71a/window 
2021-07-15 00:00:00 DEBUG Selenium [:command]    >>> http://127.0.0.1:9515/session/3195b1f3306c45c14436b264c349a71a/window | {"handle":"653EED4BB7E8FC5A3DE3992781C80ECC"} 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":null} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> DELETE session/3195b1f3306c45c14436b264c349a71a/window 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":{"error":"unknown error","message":"unknown error: failed to close window in 20 seconds\n  (Session info: chrome=135.0.7049.84)","stacktrace":"#0 0x558199d1fd0a \u003Cunknown>\n#1 0x5581997d05f0 \u003Cunknown>\n#2 0x5581997a951f \u003Cunknown>\n#3 0x5581997a958c \u003Cunknown>\n#4 0x55819985919a \u003Cunknown>\n#5 0x558199847628 \u003Cunknown>\n#6 0x558199813ea3 \u003Cunknown>\n#7 0x558199814b01 \u003Cunknown>\n#8 0x558199ce4b5b \u003Cunknown>\n#9 0x558199ce8a41 \u003Cunknown>\n#10 0x558199ccbc52 \u003Cunknown>\n#11 0x558199ce95b4 \u003Cunknown>\n#12 0x558199caff0f \u003Cunknown>\n#13 0x558199d0ddb8 \u003Cunknown>\n#14 0x558199d0df96 \u003Cunknown>\n#15 0x558199d1eb56 \u003Cunknown>\n#16 0x7faa2da731f5 \u003Cunknown>\n"}} 
2025-04-14 10:40:30 DEBUG Selenium [:command] -> DELETE session/3195b1f3306c45c14436b264c349a71a 
2025-04-14 10:40:30 DEBUG Selenium [:command] <- {"value":null} 
2025-04-14 10:40:30 DEBUG Selenium [:process] Checking if 80533 is exited: 
2025-04-14 10:40:30 DEBUG Selenium [:driver_service] Sending shutdown request to server 
2025-04-14 10:40:30 DEBUG Selenium [:process] Polling 20 seconds for exit of 80533 
2025-04-14 10:40:30 DEBUG Selenium [:process] Checking if 80533 is exited: 
2025-04-14 10:40:30 DEBUG Selenium [:process] Checking if 80533 is exited: 
2025-04-14 10:40:30 DEBUG Selenium [:process]   -> exit code is 0 
2025-04-14 10:40:30 DEBUG Selenium [:process] Checking if 80533 is exited: 
2025-04-14 10:40:30 DEBUG Selenium [:process]   -> exit code is 0 
2025-04-14 10:40:30 DEBUG Selenium [:process] Checking if 80533 is exited: 
2025-04-14 10:40:30 DEBUG Selenium [:process]   -> exit code is 0 
2025-04-14 10:40:30 DEBUG Selenium [:process] Checking if 80533 is exited: 
2025-04-14 10:40:30 DEBUG Selenium [:process]   -> exit code is 0 
2025-04-14 10:40:30 DEBUG Selenium [:process] Checking if 80533 is exited: 
2025-04-14 10:40:30 DEBUG Selenium [:process]   -> exit code is 0 

(Not sure why the :command log entries have a fixed date (2021-07-15 00:00:00), but that's a separate issue.)

SE_FORCE_BROWSER_DOWNLOAD=true rspec

For comparison, forcing to download CfT yields no such error.

rspec log

$ rspec spec/features/some_browser_spec.rb
Capybara starting Puma...
* Version 6.6.0, codename: Return to Forever
* Min threads: 0, max threads: 4
* Listening on http://127.0.0.1:3952
.

Finished in 3.7 seconds (files took 1.52 seconds to load)
1 example, 0 failures

selenium log (excerpt)

2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] Selenium Manager binary found at $PROJECT_ROOT/web/.gems/3.2/gems/selenium-webdriver-4.31.0/bin/linux/selenium-manager 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] Executing Process ["$PROJECT_ROOT/web/.gems/3.2/gems/selenium-webdriver-4.31.0/bin/linux/selenium-manager", "--browser", "chrome", "--language-binding", "ruby", "--output", "json", "--debug"] 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] chromedriver not found in PATH 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] Required browser: chrome 135.0.7049.84 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] chrome 135.0.7049.84 already exists 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] chrome 135.0.7049.84 is available at $HOME/.cache/selenium/chrome/linux64/135.0.7049.84/chrome 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] Required driver: chromedriver 135.0.7049.84 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] chromedriver 135.0.7049.84 already in the cache 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] Driver path: $HOME/.cache/selenium/chromedriver/linux64/135.0.7049.84/chromedriver 
2025-04-14 10:45:59 DEBUG Selenium [:selenium_manager] Browser path: $HOME/.cache/selenium/chrome/linux64/135.0.7049.84/chrome 
2025-04-14 10:45:59 DEBUG Selenium [:driver_service] Executing Process ["$HOME/.cache/selenium/chromedriver/linux64/135.0.7049.84/chromedriver", "--port=9515"] 
2025-04-14 10:45:59 DEBUG Selenium [:process] Starting process: ["$HOME/.cache/selenium/chromedriver/linux64/135.0.7049.84/chromedriver", "--port=9515"] with {[:out, :err]=>#<File:$PROJECT_ROOT/web/log/selenium.log>, :pgroup=>true} 
2025-04-14 10:45:59 DEBUG Selenium [:process]   -> pid: 84091 
2025-04-14 10:45:59 DEBUG Selenium [:driver_service] polling for socket on ["127.0.0.1", 9515] 
Starting ChromeDriver 135.0.7049.84 (6c019e56001911b3fd467e03bf68c435924d62f4-refs/branch-heads/7049@{#1778}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully on port 9515.

...

2021-07-15 00:00:00 DEBUG Selenium [:command] -> GET session/0916593ae1958385eab60214baabc65f/window/handles 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":["93CD8770B73D0CE2566000356A0C3853"]} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> POST session/0916593ae1958385eab60214baabc65f/window 
2021-07-15 00:00:00 DEBUG Selenium [:command]    >>> http://127.0.0.1:9515/session/0916593ae1958385eab60214baabc65f/window | {"handle":"93CD8770B73D0CE2566000356A0C3853"} 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":null} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> GET session/0916593ae1958385eab60214baabc65f/window/handles 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":["93CD8770B73D0CE2566000356A0C3853"]} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> POST session/0916593ae1958385eab60214baabc65f/url 
2021-07-15 00:00:00 DEBUG Selenium [:command]    >>> http://127.0.0.1:9515/session/0916593ae1958385eab60214baabc65f/url | {"url":"about:blank"} 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":null} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> POST session/0916593ae1958385eab60214baabc65f/elements 
2021-07-15 00:00:00 DEBUG Selenium [:command]    >>> http://127.0.0.1:9515/session/0916593ae1958385eab60214baabc65f/elements | {"using":"xpath","value":"/html/body/*"} 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":[]} 
2021-07-15 00:00:00 DEBUG Selenium [:command] -> POST session/0916593ae1958385eab60214baabc65f/goog/cdp/execute 
2021-07-15 00:00:00 DEBUG Selenium [:command]    >>> http://127.0.0.1:9515/session/0916593ae1958385eab60214baabc65f/goog/cdp/execute | {"cmd":"Storage.clearDataForOrigin","params":{"origin":"*","storageTypes":"cookies,local_storage"}} 
2021-07-15 00:00:00 DEBUG Selenium [:command] <- {"value":{}} 
2025-04-14 10:46:03 DEBUG Selenium [:command] -> DELETE session/0916593ae1958385eab60214baabc65f 
2025-04-14 10:46:03 DEBUG Selenium [:command] <- {"value":null} 
2025-04-14 10:46:03 DEBUG Selenium [:process] Checking if 84091 is exited: 
2025-04-14 10:46:03 DEBUG Selenium [:driver_service] Sending shutdown request to server 
2025-04-14 10:46:03 DEBUG Selenium [:process] Polling 20 seconds for exit of 84091 
2025-04-14 10:46:03 DEBUG Selenium [:process] Checking if 84091 is exited: 
2025-04-14 10:46:03 DEBUG Selenium [:process] Checking if 84091 is exited: 
2025-04-14 10:46:03 DEBUG Selenium [:process]   -> exit code is 0 
2025-04-14 10:46:03 DEBUG Selenium [:process] Checking if 84091 is exited: 
2025-04-14 10:46:03 DEBUG Selenium [:process]   -> exit code is 0 
2025-04-14 10:46:03 DEBUG Selenium [:process] Checking if 84091 is exited: 
2025-04-14 10:46:03 DEBUG Selenium [:process]   -> exit code is 0 
2025-04-14 10:46:03 DEBUG Selenium [:process] Checking if 84091 is exited: 
2025-04-14 10:46:03 DEBUG Selenium [:process]   -> exit code is 0 
2025-04-14 10:46:03 DEBUG Selenium [:process] Checking if 84091 is exited: 
2025-04-14 10:46:03 DEBUG Selenium [:process]   -> exit code is 0 

Environment

• Debian 12 (Bookworm), 64 Bit, on a amd64 CPU
• Ruby 3.2.6 (via rbenv)
• Chrome 135.0.7049.84 (both local installation and as CfT in Selenium Cache)

Reproduction

Extracting reproducible code from my application is a bit hard, but I think I've condensed it more or less to the following code (although the error is slightly different). You need:

• a locally installed Chrome
• Ruby v3.2 (3.3 or 3.4 ought to work as well)

Run it with:

1. ruby ./repro.rb
2. SE_FORCE_BROWSER_DOWNLOAD=true ruby ./repro.rb

Reproducible Code

#!/usr/bin/env ruby

require "bundler/inline"

gemfile do
  source "https://rubygems.org"
  ruby ">= 3.2.0"
  gem "selenium-webdriver", "~> 4.31.0"
end

log_file = File.join(__dir__, "selenium.log")
File.delete(log_file) if File.exist?(log_file)

Selenium::WebDriver.logger.tap { |logger|
  logger.level  = :debug
  logger.output = log_file
}

driver = Selenium::WebDriver.for :chrome
driver.get "https://hexapdf.gettalong.org/examples/hello_world.html"

Selenium::WebDriver::Wait.new(timeout: 5, interval: 0.5).until {
  driver.find_element(tag_name: "object").displayed?
}

driver.close
puts "ok"

Debugging Logs

See above.

[selenium-ci]

@dmke, thank you for creating this issue. We will troubleshoot it as soon as we can.

Selenium Triage Team: remember to follow the Triage Guide

[dmke]

N.B. Using SE_FORCE_BROWSER_DOWNLOAD=true regularly fails with:

2025-04-14 13:38:24 ERROR Selenium [:selenium_manager] error sending request for url (https://googlechromelabs.github.io/chrome-for-testing/last-known-good-versions-with-downloads.json) 
2025-04-14 13:38:24 ERROR Selenium Exception occurred: Unsuccessful command executed: ["$PROJECT_ROOT/.gems/3.2/gems/selenium-webdriver-4.31.0/bin/linux/selenium-manager", "--browser", "chrome", "--language-binding", "ruby", "--output", "json"] - Code 65
{"code"=>65, "message"=>"error sending request for url (https://googlechromelabs.github.io/chrome-for-testing/last-known-good-versions-with-downloads.json)", "driver_path"=>"", "browser_path"=>""}

(This seems to be an IPv6 connectivity issue with Github Pages, not with Selenium Manager, but is annoying nonetheless.)

I'm currently working around that issue with an additional SE_TTL=604800 (7d; I'm OK with slightly out-of-date versions).

[cgoldberg]

@dmke have you filed a Chrome/ChromeDriver issue for this?

Selenium doesn't treat regular Chrome any different than CfT in how it interacts with the driver/browser... so this doesn't sound like something that can be fixed on our end.

[dmke]

have you filed a Chrome/ChromeDriver issue for this?

No, not yet. I guess was hoping you'd say something like "yes, we treat CfT differently" :)

I'll keep you posted.

[dmke]

Chromium/Chromedriver issue: https://issues.chromium.org/issues/410533047