Merge branch 'main' of https://github.com/autoscrape-labs/pydoll

Author: thalissonvs

CHANGELOG.md

@@ -1,3 +1,27 @@
+## 2.8.0 (2025-08-28)
+
+### Feat
+
+- adding get_siblings_elements method
+- adding get_children_elements method
+- refactor Tab class to support optional WebSocket address handling
+- add WebSocket connection support for existing browser instances
+- add optional WebSocket address support in connection handler
+
+### Fix
+
+- add get siblings and get childen methods a raise_exc option
+- improving children and parent retrive docstring and creating a private generic method for then
+- using new execute_script public method
+- solving conflicts
+- rename pages fixtures files and adding a error test
+
+### Refactor
+
+- refactor Tab class to improve initialization and error handling
+- refactor Browser class to manage opened tabs and WebSocket setup
+- add new exception classes for connection and WebSocket errors
+
 ## 2.7.0 (2025-08-22)
 
 ### Feat

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright © 2025 <copyright holders>
+Copyright © 2025 AutoscrapeLabs
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 

README.md

@@ -47,6 +47,45 @@ We believe that powerful automation shouldn't require you to become an expert in
 
 ## What's New
 
+### Remote connections via WebSocket — control any Chrome from anywhere!
+
+You asked for it, we delivered. You can now connect to an already running browser remotely via its WebSocket address and use the full Pydoll API immediately.
+
+```python
+from pydoll.browser.chromium import Chrome
+
+chrome = Chrome()
+tab = await chrome.connect('ws://YOUR_HOST:9222/devtools/browser/XXXX')
+
+# Full power unlocked: navigation, element automation, requests, events…
+await tab.go_to('https://example.com')
+title = await tab.execute_script('return document.title')
+print(title)
+```
+
+This makes it effortless to run Pydoll against remote/CI browsers, containers, or shared debugging targets — no local launch required. Just point to the WS endpoint and automate.
+
+### Navigate the DOM like a pro: get_children_elements() and get_siblings_elements()
+
+Two delightful helpers to traverse complex layouts with intention:
+
+```python
+# Grab direct children of a container
+container = await tab.find(id='cards')
+cards = await container.get_children_elements(max_depth=1)
+
+# Want to go deeper? This will return children of children (and so on)
+elements = await container.get_children_elements(max_depth=2) 
+
+# Walk horizontal lists without re-querying the DOM
+active = await tab.find(class_name='item-active')
+siblings = await active.get_siblings_elements()
+
+print(len(cards), len(siblings))
+```
+
+Use them to cut boilerplate, express intent, and keep your scraping/automation logic clean and readable — especially in dynamic grids, lists and menus.
+
 ### WebElement: state waiting and new public APIs
 
 - New `wait_until(...)` on `WebElement` to await element states with minimal code:

README_zh.md

@@ -49,6 +49,45 @@ Pydoll 采用全新设计理念，从零构建，直接对接 Chrome DevTools Pr
 
 ## 最新功能
 
+### 通过 WebSocket 进行远程连接 —— 随时随地控制浏览器！
+
+现在你可以使用浏览器的 WebSocket 地址直接连接到已运行的实例，并立即使用完整的 Pydoll API：
+
+```python
+from pydoll.browser.chromium import Chrome
+
+chrome = Chrome()
+tab = await chrome.connect('ws://YOUR_HOST:9222/devtools/browser/XXXX')
+
+# 直接开干：导航、元素自动化、请求、事件…
+await tab.go_to('https://example.com')
+title = await tab.execute_script('return document.title')
+print(title)
+```
+
+这让你可以轻松对接远程/CI 浏览器、容器或共享调试目标——无需本地启动，只需指向 WS 端点即可自动化。
+
+### 像专业人士一样漫游 DOM：get_children_elements() 与 get_siblings_elements()
+
+两个让复杂布局遍历更优雅的小助手：
+
+```python
+# 获取容器的直接子元素
+container = await tab.find(id='cards')
+cards = await container.get_children_elements(max_depth=1)
+
+# 想更深入？这将返回子元素的子元素（以此类推）
+elements = await container.get_children_elements(max_depth=2) 
+
+# 在横向列表中无痛遍历兄弟元素
+active = await tab.find(class_name='item--active')
+siblings = await active.get_siblings_elements()
+
+print(len(cards), len(siblings))
+```
+
+用更少样板代码表达更多意图，特别适合动态网格、列表与菜单的场景，让抓取/自动化逻辑更清晰、更可读。
+
 ### WebElement：状态等待与新的公共 API
 
 - 新增 `wait_until(...)` 用于等待元素状态，使用更简单：

cz.yaml

@@ -2,4 +2,4 @@
 commitizen:
   name: cz_conventional_commits
   tag_format: $version
-  version: 2.7.0
+  version: 2.8.0

public/docs/features.md

@@ -57,6 +57,53 @@ Capture visual content from web pages:
 - **High-Quality PDF Export**: Generate PDF documents from web pages
 - **Custom Formatting**: Coming soon!
 
+## Remote Connections and Hybrid Automation
+
+### Connect to a running browser via WebSocket
+
+Control an already running browser remotely by pointing Pydoll to its DevTools WebSocket address.
+
+```python
+import asyncio
+from pydoll.browser.chromium import Chrome
+
+async def main():
+    chrome = Chrome()
+    tab = await chrome.connect('ws://YOUR_HOST:9222/devtools/browser/XXXX')
+
+    await tab.go_to('https://example.com')
+    title = await tab.execute_script('return document.title')
+    print(title)
+
+asyncio.run(main())
+```
+
+Perfect for CI, containers, remote hosts, or shared debugging targets—no local launch required. Just provide the WS endpoint and automate.
+
+### Bring your own CDP: wrap existing sessions with Pydoll objects
+
+If you already have your own CDP integration, you can still leverage Pydoll’s high-level API by wiring it to an existing DevTools session. As long as you know an element’s `objectId`, you can create a `WebElement` directly:
+
+```python
+from pydoll.connection import ConnectionHandler
+from pydoll.elements.web_element import WebElement
+
+# Your DevTools WebSocket endpoint and an element objectId you resolved via CDP
+ws = 'ws://YOUR_HOST:9222/devtools/page/ABCDEF...'
+object_id = 'REMOTE_ELEMENT_OBJECT_ID'
+
+connection_handler = ConnectionHandler(ws_address=ws)
+element = WebElement(object_id=object_id, connection_handler=connection_handler)
+
+# Use the full WebElement API immediately
+visible = await element.is_visible()
+await element.wait_until(is_interactable=True, timeout=10)
+await element.click()
+text = await element.text
+```
+
+This hybrid approach lets you blend your low-level CDP tooling (for discovery, instrumentation, or custom flows) with Pydoll’s ergonomic element API.
+
 ## Intuitive Element Finding
 
 Pydoll v2.0+ introduces a revolutionary approach to finding elements that's both more intuitive and more powerful than traditional selector-based methods.
@@ -137,6 +184,44 @@ async def query_examples():
 asyncio.run(query_examples())
 ```
 
+### DOM Traversal Helpers: get_children_elements() and get_siblings_elements()
+
+These helpers let you traverse the DOM tree from a known anchor, preserving scope and intent.
+
+- get_children_elements(max_depth: int = 1, tag_filter: list[str] | None = None, raise_exc: bool = False) -> list[WebElement]
+  - Returns descendants up to max_depth using pre-order traversal (direct children first, then their descendants)
+  - max_depth=1 returns only direct children; 2 includes grandchildren, and so on
+  - tag_filter restricts results to specific tags (use lowercase names, e.g. ['a', 'li'])
+  - raise_exc=True raises ElementNotFound if the underlying script fails to resolve
+
+- get_siblings_elements(tag_filter: list[str] | None = None, raise_exc: bool = False) -> list[WebElement]
+  - Returns elements sharing the same parent, excluding the current element
+  - tag_filter narrows by tag; order follows the parent’s child order
+
+```python
+# Direct children in document order
+container = await tab.find(id='cards')
+children = await container.get_children_elements(max_depth=1)
+
+# Include grandchildren
+descendants = await container.get_children_elements(max_depth=2)
+
+# Filter by tag
+links = await container.get_children_elements(max_depth=4, tag_filter=['a'])
+
+# Horizontal traversal
+active = await tab.find(class_name='item-active')
+siblings = await active.get_siblings_elements()
+link_siblings = await active.get_siblings_elements(tag_filter=['a'])
+```
+
+Performance and correctness notes:
+
+- DOM is a tree: breadth expands quickly with depth. Prefer small max_depth values and apply tag_filter to minimize work.
+- Ordering: children follow document order; siblings follow the parent’s order for stable iteration.
+- iFrames: each iframe has its own tree. Use `tab.get_frame(iframe_element)` to traverse inside the frame, then call these helpers there.
+- Large documents: deep traversals can touch many nodes. Combine shallow traversal with targeted `find()`/`query()` on subtree anchors for best performance.
+
 ## Native Cloudflare Captcha Bypass
 
 !!! warning "Important Information About Captcha Bypass"

public/docs/zh/features.md

@@ -59,6 +59,53 @@ Pydoll支持操作任何Chromium核心的浏览器:
 - **高质量 PDF 导出**：从网页生成 PDF 文档
 - **自定义格式**：即将推出！
 
+## 远程连接与混合自动化
+
+### 通过 WebSocket 连接已运行的浏览器
+
+只需提供 DevTools 的 WebSocket 地址，即可远程控制已经在运行的浏览器实例：
+
+```python
+import asyncio
+from pydoll.browser.chromium import Chrome
+
+async def main():
+    chrome = Chrome()
+    tab = await chrome.connect('ws://YOUR_HOST:9222/devtools/browser/XXXX')
+
+    await tab.go_to('https://example.com')
+    title = await tab.execute_script('return document.title')
+    print(title)
+
+asyncio.run(main())
+```
+
+非常适合 CI、容器、远程主机或共享调试目标——无需本地启动，只需指向 WS 端点即可自动化。
+
+### 自带 CDP：用 Pydoll 封装已有会话
+
+如果你已经有自己的 CDP 集成，也可以将其与 Pydoll 的高级 API 结合使用。只要你知道元素的 `objectId`，就能直接构造 `WebElement`：
+
+```python
+from pydoll.connection import ConnectionHandler
+from pydoll.elements.web_element import WebElement
+
+# 你的 DevTools WebSocket 地址，以及通过 CDP 获取到的元素 objectId
+ws = 'ws://YOUR_HOST:9222/devtools/page/ABCDEF...'
+object_id = 'REMOTE_ELEMENT_OBJECT_ID'
+
+connection_handler = ConnectionHandler(ws_address=ws)
+element = WebElement(object_id=object_id, connection_handler=connection_handler)
+
+# 立刻使用完整的 WebElement API
+visible = await element.is_visible()
+await element.wait_until(is_interactable=True, timeout=10)
+await element.click()
+text = await element.text
+```
+
+这种混合模式让你可以将底层的 CDP 能力（用于发现、注入或自定义流程）与 Pydoll 更易用的元素 API 顺畅结合。
+
 ## 直观的元素查找
 
 Pydoll v2.0+ 引入了一种革命性的元素查找方法，比传统的基于选择器的方法更直观、更强大。
@@ -140,6 +187,44 @@ async def query_examples():
 asyncio.run(query_examples())
 ```
 
+### DOM 遍历助手：get_children_elements() 与 get_siblings_elements()
+
+从已知锚点按树形结构遍历 DOM，更加明确且安全：
+
+- get_children_elements(max_depth: int = 1, tag_filter: list[str] | None = None, raise_exc: bool = False) -> list[WebElement]
+  - 使用先序遍历返回后代元素（先直接子元素，再其后代），深度不超过 max_depth
+  - max_depth=1 仅返回直接子元素；2 包含孙辈元素，以此类推
+  - tag_filter 用于按标签名过滤（小写，如 ['a', 'li']）
+  - 当 raise_exc=True 且脚本解析失败时会抛出 ElementNotFound
+
+- get_siblings_elements(tag_filter: list[str] | None = None, raise_exc: bool = False) -> list[WebElement]
+  - 返回与当前元素同一父节点下的兄弟元素（不包含当前元素）
+  - tag_filter 可按标签名过滤；返回顺序与父节点的子元素顺序一致
+
+```python
+# 文档顺序的直接子元素
+container = await tab.find(id='cards')
+children = await container.get_children_elements(max_depth=1)
+
+# 包含孙辈
+descendants = await container.get_children_elements(max_depth=2)
+
+# 按标签过滤
+links = await container.get_children_elements(max_depth=4, tag_filter=['a'])
+
+# 横向遍历
+active = await tab.find(class_name='item-active')
+siblings = await active.get_siblings_elements()
+link_siblings = await active.get_siblings_elements(tag_filter=['a'])
+```
+
+性能与正确性提示：
+
+- DOM 是树结构：深度增加会迅速扩展宽度。优先使用较小的 max_depth，并结合 tag_filter 限制范围。
+- 顺序：子元素遵循文档顺序；兄弟元素遵循父节点的子元素顺序，便于稳定迭代。
+- iFrame：每个 iframe 是独立的 DOM 树。使用 `tab.get_frame(iframe_element)` 进入后，再在该 frame 内调用这些助手。
+- 大型文档：深层遍历可能访问大量节点。建议将浅层遍历与基于锚点的精确 `find()`/`query()` 结合，以获得更佳性能。
+
 ## 原生 Cloudflare 验证码绕过
 
 !!! 警告“关于验证码绕过的重要信息”