Merge pull request #249 from autoscrape-labs/feat/new-public-api

New public methods and some fixes

Author: thalissonvs

README.md

@@ -47,6 +47,40 @@ We believe that powerful automation shouldn't require you to become an expert in
 
 ## What's New
 
+### WebElement: state waiting and new public APIs
+
+- New `wait_until(...)` on `WebElement` to await element states with minimal code:
+
+```python
+# Wait until it becomes visible OR the timeout expires
+await element.wait_until(is_visible=True, timeout=5)
+
+# Wait until it becomes interactable (visible, on top, receiving pointer events)
+await element.wait_until(is_interactable=True, timeout=10)
+```
+
+- Methods now public on `WebElement`:
+  - `is_visible()`
+    - Checks that the element has a visible area (> 0), isn’t hidden by CSS and is in the viewport (after `scroll_into_view()` when needed). Useful pre-check before interactions.
+  - `is_interactable()`
+    - “Click-ready” state: combines visibility, enabledness and pointer-event hit testing. Ideal for robust flows that avoid lost clicks.
+  - `is_on_top()`
+    - Verifies the element is the top hit-test target at the intended click point, avoiding overlays.
+  - `execute_script(script: str, return_by_value: bool = False)`
+    - Executes JavaScript in the element’s own context (where `this` is the element). Great for fine-tuning and quick inspections.
+
+```python
+# Visually outline the element via JS
+await element.execute_script("this.style.outline='2px solid #22d3ee'")
+
+# Confirm states
+visible = await element.is_visible()
+interactable = await element.is_interactable()
+on_top = await element.is_on_top()
+```
+
+These additions simplify waiting and state validation before clicking/typing, reducing flakiness and making automations more predictable.
+
 ### Browser-context HTTP requests - game changer for hybrid automation!
 Ever wished you could make HTTP requests that automatically inherit all your browser's session state? **Now you can!**<br>
 The `tab.request` property gives you a beautiful `requests`-like interface that executes HTTP calls directly in the browser's JavaScript context. This means every request automatically gets cookies, authentication headers, CORS policies, and session state, just as if the browser made the request itself.

README_zh.md

@@ -49,6 +49,40 @@ Pydoll 采用全新设计理念，从零构建，直接对接 Chrome DevTools Pr
 
 ## 最新功能
 
+### WebElement：状态等待与新的公共 API
+
+- 新增 `wait_until(...)` 用于等待元素状态，使用更简单：
+
+```python
+# 等待元素变为可见，直到超时
+await element.wait_until(is_visible=True, timeout=5)
+
+# 等待元素变为可交互（可见、位于顶层并可接收事件）
+await element.wait_until(is_interactable=True, timeout=10)
+```
+
+- 以下 `WebElement` 方法现已公开：
+  - `is_visible()`
+    - 判断元素是否具有可见区域、未被 CSS 隐藏，并在需要时滚动进入视口。适用于交互前的快速校验。
+  - `is_interactable()`
+    - “可点击”状态：综合可见性、启用状态与指针事件命中等条件，适合构建更稳健的交互流程。
+  - `is_on_top()`
+    - 检查元素在点击位置是否为顶部命中目标，避免被覆盖导致点击失效。
+  - `execute_script(script: str, return_by_value: bool = False)`
+    - 在元素上下文中执行 JavaScript（this 指向该元素），便于细粒度调整与快速检查。
+
+```python
+# 使用 JS 高亮元素
+await element.execute_script("this.style.outline='2px solid #22d3ee'")
+
+# 校验状态
+visible = await element.is_visible()
+interactable = await element.is_interactable()
+on_top = await element.is_on_top()
+```
+
+以上新增能力能显著简化“等待+验证”场景，降低自动化过程中的不稳定性，使用例更可预测。
+
 ### 浏览器上下文 HTTP 请求 - 混合自动化的游戏规则改变者！
 你是否曾经希望能够发出自动继承浏览器所有会话状态的 HTTP 请求？**现在你可以了！**<br>
 `tab.request` 属性为你提供了一个美观的 `requests` 风格接口，可在浏览器的 JavaScript 上下文中直接执行 HTTP 调用。这意味着每个请求都会自动获得 cookies、身份验证标头、CORS 策略和会话状态，就像浏览器本身发出请求一样。

public/docs/deep-dive/webelement-domain.md

@@ -217,7 +217,7 @@ async def click(
     if self._is_option_tag():
         return await self.click_option_tag()
 
-    if not await self._is_element_visible():
+    if not await self.is_visible():
         raise exceptions.ElementNotVisible(
             'Element is not visible on the page.'
         )
@@ -341,16 +341,16 @@ WebElement provides seamless integration with JavaScript for operations that req
 
 ```python
 # Execute JavaScript in the context of this element
-await element._execute_script("this.style.border = '2px solid red';")
+await element.execute_script("this.style.border = '2px solid red';")
 
 # Get result from JavaScript execution
-visibility = await element._is_element_visible()
+visibility = await element.is_visible()
 ```
 
 The implementation uses the CDP Runtime domain to execute JavaScript with the element as the context:
 
 ```python
-async def _execute_script(
+async def execute_script(
     self, script: str, return_by_value: bool = False
 ):
     """
@@ -369,10 +369,13 @@ WebElement provides methods to check the element's visibility and interactabilit
 
 ```python
 # Check if element is visible
-is_visible = await element._is_element_visible()
+is_visible = await element.is_visible()
 
 # Check if element is the topmost at its position
-is_on_top = await element._is_element_on_top()
+is_on_top = await element.is_on_top()
+
+# Check if element is interactable
+is_interactable = await element.is_interactable()
 ```
 
 These verifications are crucial for reliable automation, ensuring that elements can be interacted with before attempting operations.

public/docs/zh/deep-dive/webelement-domain.md

@@ -219,7 +219,7 @@ async def click(
     if self._is_option_tag():
         return await self.click_option_tag()
 
-    if not await self._is_element_visible():
+    if not await self.is_visible():
         raise exceptions.ElementNotVisible(
             'Element is not visible on the page.'
         )
@@ -342,16 +342,16 @@ WebElement 为需要直接操作DOM的操作提供了与JavaScript的无缝集
 
 ```python
 # Execute JavaScript in the context of this element
-await element._execute_script("this.style.border = '2px solid red';")
+await element.execute_script("this.style.border = '2px solid red';")
 
 # Get result from JavaScript execution
-visibility = await element._is_element_visible()
+visibility = await element.is_visible()
 ```
 
 该实现通过CDP Runtime域执行JavaScript，并将元素作为上下文：
 
 ```python
-async def _execute_script(
+async def execute_script(
     self, script: str, return_by_value: bool = False
 ):
     """
@@ -371,10 +371,13 @@ WebElement 提供了检查元素可见性和可交互性的方法：
 
 ```python
 # Check if element is visible
-is_visible = await element._is_element_visible()
+is_visible = await element.is_visible()
 
 # Check if element is the topmost at its position
-is_on_top = await element._is_element_on_top()
+is_on_top = await element.is_on_top()
+
+# Check if element is interactable
+is_interactable = await element.is_interactable()
 ```
 
 这些验证对于实现可靠的自动化至关重要，可在尝试操作前确保元素可被交互。

pydoll/browser/tab.py

@@ -46,7 +46,7 @@
     PageLoadTimeout,
     WaitElementTimeout,
 )
-from pydoll.protocol.base import EmptyResponse
+from pydoll.protocol.base import EmptyResponse, Response
 from pydoll.protocol.browser.events import (
     BrowserEvent,
     DownloadProgressEvent,
@@ -282,7 +282,7 @@ async def enable_fetch_events(
         Note:
             Intercepted requests must be explicitly continued or timeout.
         """
-        response: EmptyResponse = await self._execute_command(
+        response: Response[EmptyResponse] = await self._execute_command(
             FetchCommands.enable(
                 handle_auth_requests=handle_auth,
                 resource_type=resource_type,
@@ -424,6 +424,10 @@ async def get_frame(self, frame: WebElement) -> IFrame:
 
         return Tab(self._browser, self._connection_port, iframe_target['targetId'])
 
+    async def bring_to_front(self):
+        """Brings the page to front."""
+        return await self._execute_command(PageCommands.bring_to_front())
+
     async def get_cookies(self) -> list[Cookie]:
         """Get all cookies accessible from current page."""
         response: GetCookiesResponse = await self._execute_command(
@@ -545,6 +549,7 @@ async def take_screenshot(
         self,
         path: Optional[str] = None,
         quality: int = 100,
+        beyond_viewport: bool = False,
         as_base64: bool = False,
     ) -> Optional[str]:
         """
@@ -553,6 +558,8 @@ async def take_screenshot(
         Args:
             path: File path for screenshot (extension determines format).
             quality: Image quality 0-100 (default 100).
+            beyond_viewport: The page will be scrolled to the bottom and the screenshot will
+                include the entire page
             as_base64: Return as base64 string instead of saving file.
 
         Returns:
@@ -573,6 +580,7 @@ async def take_screenshot(
             PageCommands.capture_screenshot(
                 format=ScreenshotFormat.get_value(output_extension),
                 quality=quality,
+                capture_beyond_viewport=beyond_viewport,
             )
         )
         screenshot_data = response['result']['data']

pydoll/elements/web_element.py

@@ -130,12 +130,12 @@ async def get_bounds_using_js(self) -> dict[str, int]:
 
         Returns coordinates relative to viewport (alternative to bounds property).
         """
-        response = await self._execute_script(Scripts.BOUNDS, return_by_value=True)
+        response = await self.execute_script(Scripts.BOUNDS, return_by_value=True)
         return json.loads(response['result']['result']['value'])
 
     async def get_parent_element(self) -> 'WebElement':
         """Element's parent element."""
-        result = await self._execute_script(Scripts.GET_PARENT_NODE)
+        result = await self.execute_script(Scripts.GET_PARENT_NODE)
         if not self._has_object_id_key(result):
             raise ElementNotFound(f'Parent element not found for element: {self}')
 
@@ -195,14 +195,12 @@ async def wait_until(
             WaitElementTimeout: If the condition is not met within ``timeout``.
         """
         checks_map = [
-            (is_visible, self._is_element_visible),
-            (is_interactable, self._is_element_interactable),
+            (is_visible, self.is_visible),
+            (is_interactable, self.is_interactable),
         ]
         checks = [func for flag, func in checks_map if flag]
         if not checks:
-            raise ValueError(
-                'At least one of is_visible or is_interactable must be True'
-            )
+            raise ValueError('At least one of is_visible or is_interactable must be True')
 
         condition_parts = []
         if is_visible:
@@ -219,9 +217,7 @@ async def wait_until(
                 return
 
             if timeout and loop.time() - start_time > timeout:
-                raise WaitElementTimeout(
-                    f'Timed out waiting for element to become {condition_msg}'
-                )
+                raise WaitElementTimeout(f'Timed out waiting for element to become {condition_msg}')
 
             await asyncio.sleep(0.5)
 
@@ -242,10 +238,10 @@ async def click_using_js(self):
 
         await self.scroll_into_view()
 
-        if not await self._is_element_visible():
+        if not await self.is_visible():
             raise ElementNotVisible()
 
-        result = await self._execute_script(Scripts.CLICK, return_by_value=True)
+        result = await self.execute_script(Scripts.CLICK, return_by_value=True)
         clicked = result['result']['result']['value']
         if not clicked:
             raise ElementNotInteractable()
@@ -274,7 +270,7 @@ async def click(
         if self._is_option_tag():
             return await self._click_option_tag()
 
-        if not await self._is_element_visible():
+        if not await self.is_visible():
             raise ElementNotVisible()
 
         await self.scroll_into_view()
@@ -410,24 +406,22 @@ async def _click_option_tag(self):
             )
         )
 
-    async def _is_element_visible(self):
+    async def is_visible(self):
         """Check if element is visible using comprehensive JavaScript visibility test."""
-        result = await self._execute_script(Scripts.ELEMENT_VISIBLE, return_by_value=True)
+        result = await self.execute_script(Scripts.ELEMENT_VISIBLE, return_by_value=True)
         return result['result']['result']['value']
 
-    async def _is_element_on_top(self):
+    async def is_on_top(self):
         """Check if element is topmost at its center point (not covered by overlays)."""
-        result = await self._execute_script(Scripts.ELEMENT_ON_TOP, return_by_value=True)
+        result = await self.execute_script(Scripts.ELEMENT_ON_TOP, return_by_value=True)
         return result['result']['result']['value']
 
-    async def _is_element_interactable(self):
+    async def is_interactable(self):
         """Check if element is interactable based on visibility and position."""
-        result = await self._execute_script(
-            Scripts.ELEMENT_INTERACTIVE, return_by_value=True
-        )
+        result = await self.execute_script(Scripts.ELEMENT_INTERACTIVE, return_by_value=True)
         return result['result']['result']['value']
 
-    async def _execute_script(self, script: str, return_by_value: bool = False):
+    async def execute_script(self, script: str, return_by_value: bool = False):
         """
         Execute JavaScript in element context.
 

pydoll/protocol/browser/methods.py

@@ -203,26 +203,26 @@ class GetWindowForTargetResult(TypedDict):
 
 # Command types
 AddPrivacySandboxCoordinatorKeyConfigCommand = Command[
-    AddPrivacySandboxCoordinatorKeyConfigParams, EmptyResponse
+    AddPrivacySandboxCoordinatorKeyConfigParams, Response[EmptyResponse]
 ]
 AddPrivacySandboxEnrollmentOverrideCommand = Command[
-    AddPrivacySandboxEnrollmentOverrideParams, EmptyResponse
+    AddPrivacySandboxEnrollmentOverrideParams, Response[EmptyResponse]
 ]
-CancelDownloadCommand = Command[CancelDownloadParams, EmptyResponse]
-CloseCommand = Command[EmptyParams, EmptyResponse]
-CrashCommand = Command[EmptyParams, EmptyResponse]
-CrashGpuProcessCommand = Command[EmptyParams, EmptyResponse]
-ExecuteBrowserCommandCommand = Command[ExecuteBrowserCommandParams, EmptyResponse]
+CancelDownloadCommand = Command[CancelDownloadParams, Response[EmptyResponse]]
+CloseCommand = Command[EmptyParams, Response[EmptyResponse]]
+CrashCommand = Command[EmptyParams, Response[EmptyResponse]]
+CrashGpuProcessCommand = Command[EmptyParams, Response[EmptyResponse]]
+ExecuteBrowserCommandCommand = Command[ExecuteBrowserCommandParams, Response[EmptyResponse]]
 GetBrowserCommandLineCommand = Command[EmptyParams, GetBrowserCommandLineResponse]
 GetHistogramCommand = Command[GetHistogramParams, GetHistogramResponse]
 GetHistogramsCommand = Command[GetHistogramsParams, GetHistogramsResponse]
 GetVersionCommand = Command[EmptyParams, GetVersionResponse]
 GetWindowBoundsCommand = Command[GetWindowBoundsParams, GetWindowBoundsResponse]
 GetWindowForTargetCommand = Command[GetWindowForTargetParams, GetWindowForTargetResponse]
-GrantPermissionsCommand = Command[GrantPermissionsParams, EmptyResponse]
-ResetPermissionsCommand = Command[ResetPermissionsParams, EmptyResponse]
-SetContentsSizeCommand = Command[SetContentsSizeParams, EmptyResponse]
-SetDockTileCommand = Command[SetDockTileParams, EmptyResponse]
-SetDownloadBehaviorCommand = Command[SetDownloadBehaviorParams, EmptyResponse]
-SetPermissionCommand = Command[SetPermissionParams, EmptyResponse]
-SetWindowBoundsCommand = Command[SetWindowBoundsParams, EmptyResponse]
+GrantPermissionsCommand = Command[GrantPermissionsParams, Response[EmptyResponse]]
+ResetPermissionsCommand = Command[ResetPermissionsParams, Response[EmptyResponse]]
+SetContentsSizeCommand = Command[SetContentsSizeParams, Response[EmptyResponse]]
+SetDockTileCommand = Command[SetDockTileParams, Response[EmptyResponse]]
+SetDownloadBehaviorCommand = Command[SetDownloadBehaviorParams, Response[EmptyResponse]]
+SetPermissionCommand = Command[SetPermissionParams, Response[EmptyResponse]]
+SetWindowBoundsCommand = Command[SetWindowBoundsParams, Response[EmptyResponse]]