re-opening issue regarding shadowDOM access

[dornech]

Sorry, with all due respect for your efforts @titusfortner I slightly disagree there is no issue regarding shadowDOM access.

While the code works (also when I use CSS_SELECTOR access) the issue remain that old code does not work. So there is at least a regression issue.

As mentioned, the issue is that the Python shadowroot object has no attribute/properties "id" set.
This is strange to the extent that Python webelement has both attributes/properties i. e. id and _id set or deifned as property funktion.
Moreover: manually setting "id = _id" solves the problem and makes old code work.

After looking into webelement.py and finding this

    @property
    def id(self) -> str:
        return self._id

I do not think anymore it is an issue related to evaluation webdriver response but a missing property in the shadow-root class in the same way as defined for webelement.

This is the recommended way to interact with Shadow DOM content. I've verified it works.

    driver = webdriver.Chrome()
    driver.get("https://www.consorsbank.de/home")

    shadow_host = driver.find_element(By.TAG_NAME, 'wb-website-header')
    shadow_root = shadow_host.shadow_root
    shadow_content = shadow_root.find_element(By.ID, 'wemo_theme')

Originally posted by @titusfortner in #15732

[selenium-ci]

Hi, @dornech.
This issue has been determined to require fixes in ChromeDriver.

You can see if the feature is passing in the Web Platform Tests.

If it is something new, please create an issue with the ChromeDriver team.
Feel free to comment the issues that you raise back in this issue. Thank you.

[selenium-ci]

Hi, @dornech.
This issue has been determined to require a change to the
WebDriver W3C Specification for Selenium to be able to support it.

Please create an issue with the WebDriver project.

Feel free to comment the issues that you raise back in this issue. Thank you.

[dornech]

Well, I am afraid I am not conviced it is an issue with the Webdriver itself even this was a first guess by myself as well. The reason is that I am using also SeleniumVBA which is a Selenium binding / COM-wrapper for VBA and did some tests.
Actually accessing shadow DOM element with SeleniumVBA works different ways which in Python would match to

webdriver.execute_script(f"return "arguments[0].shadowRoot.querySelectorAll('div > button#header-login-button')", shadowhost)
webdriver.execute_script(f"return "arguments[0].querySelectorAll('div > button#header-login-button')", shadowhost.shadow_root)
shadowhost.shadow_root.find_Element(By.CSS_SELECTOR, "div > button#header-login-button")

What makes me think it is an Python binding problem:

• in Selenium VBA access via JavaScript works; SeleniumVBA provides two id fields for a shadow root object, on marked with an underscore
• Python binding only provides the _id attribute for a shadow root object whereas Python bindings provide an _id and an id attribute/property for "normal" webelement object.
• The Python exception message indicates that there is an issue with the missing id attribute/property when working with shadow root object at least in javascript / execute_script method.
• The problem in Pytohn occurs wether I retrieve shadow root object via webdriver.execute_script("return arguments[0].shadowRoot", shadowhost) or via shadow_host.shadow_root
• Adding id attribute in Python before working with a shadow root object via execute_script avoids the error in all cases

So I still think there is an issue in the Python Selenium bindings but not the Webdriver itself.

[cgoldberg]

yea, I think this is a valid bug. I am able to reproduce with this code:

import time

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("https://www.consorsbank.de/home")
time.sleep(6)
shadow_host = driver.find_element(By.TAG_NAME, 'wb-website-header')
driver.execute_script(f"return arguments[0].shadowRoot.querySelectorAll('div > button#header-login-button')", shadow_host)
driver.execute_script(f"return arguments[0].querySelectorAll('div > button#header-login-button')", shadow_host.shadow_root)

(click the cookie banner out of the way while it's paused)

The error occurs on the last line, and it is AttributeError: 'ShadowRoot' object has no attribute 'id'

It is occurring here:

File "/home/cgoldberg617/code/selenium/py/selenium/webdriver/remote/webdriver.py", line 376, in _wrap_value
    return {"shadow-6066-11e4-a52e-4f735466cecf": value.id}

The method in webdriver.py it breaks on is:

    def _wrap_value(self, value):
        if isinstance(value, dict):
            converted = {}
            for key, val in value.items():
                converted[key] = self._wrap_value(val)
            return converted
        if isinstance(value, self._web_element_cls):
            return {"element-6066-11e4-a52e-4f735466cecf": value.id}
        if isinstance(value, self._shadowroot_cls):
            return {"shadow-6066-11e4-a52e-4f735466cecf": value.id}
        if isinstance(value, list):
            return list(self._wrap_value(item) for item in value)
        return value

When value is an instance of ShadowRoot, it tries to access value.id, which doesn't exist.

The fix would be to expose the id attribute in the ShadowRoot class (py/selenium/webdriver/remote/shadowroot.py). Currently it is a private variable named _id.

@dornech Are you able to provide a PR to fix this? If not, I can probably do it this week.

[dornech]

Thank @cgoldberg for your investigation and confirmation.

Well, I had a look in the webelement class and there is the property id defined.
What is about just copying this to shadow root class?
So fix would be three lines (without the docstring)

    @property
    def id(self) -> str:
        return self._id

What do you think?

[cgoldberg]

yes, I think that would fix it.

[cgoldberg]

I added this in #15739

[cgoldberg]

@dornech
This fix will be in the 4.33 release. If you want to try it early, we do a nightly release to TestPyPI. You can install it tomorrow with:

pip install -i https://test.pypi.org/simple/ selenium

(use -U to upgrade an existing installation)

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.