[🐛 Bug]: Failed to get some website on Linux but success on Windows

[dongrixinyu]

What happened?

I tried to use selenium to get some pages on Windows as below:

from selenium import webdriver

url = "https://vip.stock.finance.sina.com.cn/mkt/#stock_hs_up"
option = webdriver.ChromeOptions()
option.add_argument('headless')
driver = webdriver.Chrome(options=option)
driver.get(url)
width = driver.execute_script("return document.documentElement.scrollWidth")
height = driver.execute_script("return document.documentElement.scrollHeight")
driver.set_window_size(width, height)
driver.get_screenshot_as_file("webpage.png")

driver.get_screenshot_as_file(r"D:\webpage.png")

• Chrome version: Google Chrome 134.0.6998.88
• selenium version: 4.29.0

This snippet above achieves what I want.

but when I execute it on Linux as below, it failed.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service

url = "https://vip.stock.finance.sina.com.cn/mkt/#stock_hs_up"

option = webdriver.ChromeOptions()
option.add_argument('headless')

# here I need to set the correct binary path of chrome driver. It is a little bit different from code on Windows.
service = Service(executable_path='./chromedriver-linux64/chromedriver')

driver = webdriver.Chrome(options=option, service=service)
# driver = webdriver.Chrome()
driver.get(url)
width = driver.execute_script("return document.documentElement.scrollWidth")
height = driver.execute_script("return document.documentElement.scrollHeight")

driver.set_window_size(width, height)

driver.get_screenshot_as_file("webpage.png")

• Chrome version: Google Chrome 134.0.6998.88
• selenium version: 4.29.0
• the network is OK

I failed to get the result.

Traceback (most recent call last):
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 534, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connection.py", line 516, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/http/client.py", line 1395, in getresponse
    response.begin()
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/http/client.py", line 286, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/socket.py", line 718, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cuichengyu/quantrading/test.py", line 17, in <module>
    driver.get(url)
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 454, in get
    self.execute(Command.GET, {"url": url})
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 427, in execute
    response = self.command_executor.execute(driver_command, params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/selenium/webdriver/remote/remote_connection.py", line 404, in execute
    return self._request(command_info[0], url, body=data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/selenium/webdriver/remote/remote_connection.py", line 428, in _request
    response = self._conn.request(method, url, body=body, headers=headers, timeout=self._client_config.timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/_request_methods.py", line 143, in request
    return self.request_encode_body(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/_request_methods.py", line 278, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/poolmanager.py", line 443, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/util/retry.py", line 474, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=57885): Read timed out. (read timeout=120)

As you can see, it report timeout error, but when I request is by curl https://vip.stock.finance.sina.com.cn/mkt/#stock_hs_up, I get the result below immedietly, for sure the network is OK.

<script type="text/javascript" src="//n.sinaimg.cn/finance/stock/hq/src/addSima.js"></script>
<!--ima end-->
<script type="text/javascript" src="http://www.sinaimg.cn/unipro/pub/suda_m_v629.js"></script>
<script type="text/javascript">suds_init(1523,20.00,915,2);</script>
<script type="text/javascript" src="http://hq.sinajs.cn/?format=json&func=window.StandardBJTime=hq_json_sys_time;if%28typeof%28StandardBJTime_Callback%29==%27function%27%29StandardBJTime_Callback%28%29;&list=sys_time"></script>
<script type="text/javascript" src="http://finance.sina.com.cn/globaltimezone.js"></script>
<!--<script type="text/javascript" src="http://finance.sina.com.cn/basejs/MarketTS.js "></script>-->
<script type="text/javascript" src="//n.sinaimg.cn/finance/hqzxpclrr/MarketTS.js"></script>
<script type="text/javascript">
        (new MarketTS()).bind("stock",'nsq',function () { },function (__stringStatus,__allStatus)
        {
                document.getElementById("notice_us").innerHTML = __stringStatus;
                if(__allStatus[0] == 2)
                {
                        document.getElementById("notice_us").style.color = 'red';
                }
                else
                {
                        document.getElementById("notice_us").style.color = '#888';
                }
        });
</script>
<script charset="UTF-8" src="//n.sinaimg.cn/finance/hqcenter_right_ad/hqcenter_finapp_ad_0205.js"></script>
<!--<script charset="utf-8" src="http://n.sinaimg.cn/finance/xcapp/js/hqCenter_xcapp.js?ts=2.5"></script>-->
</html>

How can we reproduce the issue?

from selenium import webdriver
from selenium.webdriver.chrome.service import Service

url = "https://vip.stock.finance.sina.com.cn/mkt/#stock_hs_up"

option = webdriver.ChromeOptions()
option.add_argument('headless')

# here I need to set the correct binary path of chrome driver. It is a little bit different from code on Windows.
service = Service(executable_path='./chromedriver-linux64/chromedriver')

driver = webdriver.Chrome(options=option, service=service)
# driver = webdriver.Chrome()
driver.get(url)
width = driver.execute_script("return document.documentElement.scrollWidth")
height = driver.execute_script("return document.documentElement.scrollHeight")

driver.set_window_size(width, height)

driver.get_screenshot_as_file("webpage.png")


- OS: linux ubuntu
- Chrome version: Google Chrome 134.0.6998.88
- selenium version: 4.29.0
- the network is OK

Relevant log output

Traceback (most recent call last):
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 534, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connection.py", line 516, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/http/client.py", line 1395, in getresponse
    response.begin()
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/http/client.py", line 286, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/socket.py", line 718, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cuichengyu/quantrading/test.py", line 17, in <module>
    driver.get(url)
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 454, in get
    self.execute(Command.GET, {"url": url})
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 427, in execute
    response = self.command_executor.execute(driver_command, params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/selenium/webdriver/remote/remote_connection.py", line 404, in execute
    return self._request(command_info[0], url, body=data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/selenium/webdriver/remote/remote_connection.py", line 428, in _request
    response = self._conn.request(method, url, body=body, headers=headers, timeout=self._client_config.timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/_request_methods.py", line 143, in request
    return self.request_encode_body(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/_request_methods.py", line 278, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/poolmanager.py", line 443, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/util/retry.py", line 474, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/home/cuichengyu/anaconda3/envs/quantrading/lib/python3.11/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=57885): Read timed out. (read timeout=120)

Operating System

Ubuntu

Selenium version

4.29.0

What are the browser(s) and version(s) where you see this issue?

Chrome 134.0.6998.88

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 134.0.6998.88

Are you using Selenium Grid?

No response

[github-actions]

@dongrixinyu, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[github-actions]

We need more information about this issue in order to troubleshoot.

Please turn on logging and re-run your code. Information on how to adjust logs for your language can be found in our
Troubleshooting documentation.

[github-actions]

This issue was closed because we did not receive any additional information after 14 days.

[cgoldberg]

@dongrixinyu if you want this issue investigated, please add debug logs and we will re-open it. The error you posted shows it is timing out communicating with chromedriver, not in reaching the remote URL.

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.