[🐛 Bug]: Unable to access shadow root via Selenium

[GaurangTandon]

Description

Steps to repro issue:

1. Open the page https://trailhead.salesforce.com/trailblazer-community/feed/0D54S00000QLJIySAP yourself in Chrome
2. Run this in DevTools: document.querySelector('TBC-FEED-ITEM-DETAIL-PAGE').shadowRoot.querySelector('TBC-FEED-V2')
3. Observe a non-null element is returned, as shown in this image:

4. Now goto the Terminal and run the Python script using python file.py
5. Let the script finish.

Actual

The script fails to find the shadow root that we were able to find using JavaScript above.

Expected

The script will be able to find the shadow root.

Stacktrace

Traceback (most recent call last):
  File "/Users/blazegt/test-shadow-root-selenium/runtest.py", line 21, in <module>
    root = element.shadow_root
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/blazegt/test-shadow-root-selenium/venv/lib/python3.12/site-packages/selenium/webdriver/remote/webelement.py", line 327, in shadow_root
    return self._execute(Command.GET_SHADOW_ROOT)["value"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/blazegt/test-shadow-root-selenium/venv/lib/python3.12/site-packages/selenium/webdriver/remote/webelement.py", line 572, in _execute
    return self._parent.execute(command, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/blazegt/test-shadow-root-selenium/venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/Users/blazegt/test-shadow-root-selenium/venv/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 232, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchShadowRootException: Message: no such shadow root: shadow root not found
  (Session info: chrome=135.0.7049.96)
Stacktrace:
0   chromedriver                        0x0000000104ad6a54 cxxbridge1$str$ptr + 2803960
1   chromedriver                        0x0000000104acecf0 cxxbridge1$str$ptr + 2771860
2   chromedriver                        0x000000010461a864 cxxbridge1$string$len + 93028
3   chromedriver                        0x0000000104656e58 cxxbridge1$string$len + 340312
4   chromedriver                        0x0000000104656aa8 cxxbridge1$string$len + 339368
5   chromedriver                        0x00000001046a2480 cxxbridge1$string$len + 649088
6   chromedriver                        0x00000001046557ec cxxbridge1$string$len + 334572
7   chromedriver                        0x0000000104a9bccc cxxbridge1$str$ptr + 2562928
8   chromedriver                        0x0000000104a9ef98 cxxbridge1$str$ptr + 2575932
9   chromedriver                        0x0000000104a7c2c4 cxxbridge1$str$ptr + 2433384
10  chromedriver                        0x0000000104a9f810 cxxbridge1$str$ptr + 2578100
11  chromedriver                        0x0000000104a6d2f0 cxxbridge1$str$ptr + 2371988
12  chromedriver                        0x0000000104abf57c cxxbridge1$str$ptr + 2708512
13  chromedriver                        0x0000000104abf708 cxxbridge1$str$ptr + 2708908
14  chromedriver                        0x0000000104ace93c cxxbridge1$str$ptr + 2770912
15  libsystem_pthread.dylib             0x000000019885bfa8 _pthread_start + 148
16  libsystem_pthread.dylib             0x0000000198856da0 thread_start + 8

Reproducible Code

import time
import logging
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from selenium.webdriver.common.by import By

logger = logging.getLogger('selenium')
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler()
logger.addHandler(handler)

driver = webdriver.Chrome(service=ChromeService())
driver.implicitly_wait(5)

driver.get('chrome://version')
time.sleep(1)
driver.get('https://trailhead.salesforce.com/trailblazer-community/feed/0D54S00000QLJIySAP')

time.sleep(10) # wait for all dynamic elements in the page to load
element = driver.find_element(By.CSS_SELECTOR, 'TBC-FEED-ITEM-DETAIL-PAGE')
root = element.shadow_root
child = root.find_element(By.CSS_SELECTOR, 'TBC-FEED-V2')

Debugging Logs

Selenium Manager binary found at: /Users/blazegt/test-shadow-root-selenium/venv/lib/python3.12/site-packages/selenium/webdriver/common/macos/selenium-manager
Executing process: /Users/blazegt/test-shadow-root-selenium/venv/lib/python3.12/site-packages/selenium/webdriver/common/macos/selenium-manager --browser chrome --debug --language-binding python --output json
chromedriver not found in PATH
chrome detected at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
Running command: /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --version
Output: "Google Chrome 135.0.7049.96 "
Detected browser: chrome 135.0.7049.96
Required driver: chromedriver 135.0.7049.114
chromedriver 135.0.7049.114 already in the cache
Driver path: /Users/blazegt/.cache/selenium/chromedriver/mac-arm64/135.0.7049.114/chromedriver
Browser path: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
Started executable: `/Users/blazegt/.cache/selenium/chromedriver/mac-arm64/135.0.7049.114/chromedriver` in a child process with pid: 92289 using 0 to output -3
POST http://localhost:49867/session {'capabilities': {'firstMatch': [{}], 'alwaysMatch': {'browserName': 'chrome', 'pageLoadStrategy': <PageLoadStrategy.normal: 'normal'>, 'browserVersion': None, 'goog:chromeOptions': {'extensions': [], 'binary': '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', 'args': []}}}}
Remote response: status=200 | data={"value":{"capabilities":{"acceptInsecureCerts":false,"browserName":"chrome","browserVersion":"135.0.7049.96","chrome":{"chromedriverVersion":"135.0.7049.114 (63fd8a7d9d09e41ba37b84386c85d5f249f848f7-refs/branch-heads/7049@{#2175})","userDataDir":"/var/folders/rq/5y4dyhqd29x5qsqt32tdl2s40000gn/T/.org.chromium.Chromium.ekaIX7"},"fedcm:accounts":true,"goog:chromeOptions":{"debuggerAddress":"localhost:49880"},"networkConnectionEnabled":false,"pageLoadStrategy":"normal","platformName":"mac","proxy":{},"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify","webauthn:extension:credBlob":true,"webauthn:extension:largeBlob":true,"webauthn:extension:minPinLength":true,"webauthn:extension:prf":true,"webauthn:virtualAuthenticators":true},"sessionId":"138ada247048a893026239cfabb70c41"}} | headers=HTTPHeaderDict({'Content-Length': '891', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request
POST http://localhost:49867/session/138ada247048a893026239cfabb70c41/timeouts {'implicit': 5000}
Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Content-Length': '14', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request
POST http://localhost:49867/session/138ada247048a893026239cfabb70c41/url {'url': 'chrome://version'}
Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Content-Length': '14', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request
POST http://localhost:49867/session/138ada247048a893026239cfabb70c41/url {'url': 'https://trailhead.salesforce.com/trailblazer-community/feed/0D54S00000QLJIySAP'}
Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Content-Length': '14', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request
POST http://localhost:49867/session/138ada247048a893026239cfabb70c41/element {'using': 'css selector', 'value': 'TBC-FEED-ITEM-DETAIL-PAGE'}
Remote response: status=200 | data={"value":{"element-6066-11e4-a52e-4f735466cecf":"f.579FB7A7DB62C86830DD3F73FD95BC20.d.0312384BADEDEE5946065BBFCEA5ADEC.e.2041"}} | headers=HTTPHeaderDict({'Content-Length': '128', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request
GET http://localhost:49867/session/138ada247048a893026239cfabb70c41/element/f.579FB7A7DB62C86830DD3F73FD95BC20.d.0312384BADEDEE5946065BBFCEA5ADEC.e.2041/shadow {}
Remote response: status=404 | data={"value":{"error":"no such shadow root","message":"no such shadow root: shadow root not found\n  (Session info: chrome=135.0.7049.96)","stacktrace":"0   chromedriver                        0x0000000104ad6a54 cxxbridge1$str$ptr + 2803960\n1   chromedriver                        0x0000000104acecf0 cxxbridge1$str$ptr + 2771860\n2   chromedriver                        0x000000010461a864 cxxbridge1$string$len + 93028\n3   chromedriver                        0x0000000104656e58 cxxbridge1$string$len + 340312\n4   chromedriver                        0x0000000104656aa8 cxxbridge1$string$len + 339368\n5   chromedriver                        0x00000001046a2480 cxxbridge1$string$len + 649088\n6   chromedriver                        0x00000001046557ec cxxbridge1$string$len + 334572\n7   chromedriver                        0x0000000104a9bccc cxxbridge1$str$ptr + 2562928\n8   chromedriver                        0x0000000104a9ef98 cxxbridge1$str$ptr + 2575932\n9   chromedriver                        0x0000000104a7c2c4 cxxbridge1$str$ptr + 2433384\n10  chromedriver                        0x0000000104a9f810 cxxbridge1$str$ptr + 2578100\n11  chromedriver                        0x0000000104a6d2f0 cxxbridge1$str$ptr + 2371988\n12  chromedriver                        0x0000000104abf57c cxxbridge1$str$ptr + 2708512\n13  chromedriver                        0x0000000104abf708 cxxbridge1$str$ptr + 2708908\n14  chromedriver                        0x0000000104ace93c cxxbridge1$str$ptr + 2770912\n15  libsystem_pthread.dylib             0x000000019885bfa8 _pthread_start + 148\n16  libsystem_pthread.dylib             0x0000000198856da0 thread_start + 8\n"}} | headers=HTTPHeaderDict({'Content-Length': '1654', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request

[selenium-ci]

@GaurangTandon, thank you for creating this issue. We will troubleshoot it as soon as we can.

Selenium Triage Team: remember to follow the Triage Guide

[shbenzer]

Shadow roots can either be in 'open' or 'closed' states. Selenium requires JavaScript to access shadow roots in the closed state. The following works for me:

driver = webdriver.Chrome()
driver.get('https://trailhead.salesforce.com/trailblazer-community/feed/0D54S00000QLJIySAP')
element = driver.find_element(By.CSS_SELECTOR, 'TBC-FEED-ITEM-DETAIL-PAGE')
root = driver.execute_script("return arguments[0].shadowRoot", element)
child = root.find_element(By.CSS_SELECTOR, 'TBC-FEED-V2')

[titusfortner]

@shbenzer what is the action you want to take on this?

[shbenzer]

@titusfortner I was originally waiting for a response from the creator, but I think it can be closed on reflection.

[GaurangTandon]

Hi @shbenzer Thanks for the response.

To be clear, closed shadow roots are directly accessible in Selenium since version 131 of ChromeDriver. I myself use it in other test files (ChromeDriver commit)

I found a possible reason after some more debugging. The Trailblazer site has its custom implementation of attachShadow:

attachShadow: {
        value: function(e) {
            return e["$$lwc-synthetic-mode"] ? Xn(this, e) : we.call(this, e)
           //                                  ^^^^^^^^^^^---- I confirmed this branch is triggered 
        },
        enumerable: !0,
        writable: !0,
        configurable: !0
    },

which is from their Synthetic Shadow DOM polyfill So, the .shadowRoot property in JavaScript works fine, but Selenium ChromeDriver does not recognize it, (I assume) because it's not a "real" shadow root.

Either way it seems this is not an issue with Selenium. I will leave this post up as reference for future readers (who may also find this behavior confusing)

[shbenzer]

Hi @shbenzer Thanks for the response.

To be clear, closed shadow roots are directly accessible in Selenium since version 131 of ChromeDriver. I myself use it in other test files (ChromeDriver commit)

Very cool! Thanks for letting us know

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.