python, although some scripts use Syntax Formats that expect pytest (a Python unit-testing framework included with SeleniumBase that can discover, collect, and run tests automatically).
-âšī¸ Scripts can be called via python, although some Syntax Formats expect pytest (a Python unit-testing framework included with SeleniumBase that can discover, collect, and run tests automatically).
+--------
-đ Here's my_first_test.py, which tests login, shopping, and checkout:
+đ Here's raw_google.py, which performs a Google search:
-```bash -pytest my_first_test.py +```python +from seleniumbase import SB + +with SB(test=True, ad_block=True, locale_code="en") as sb: + sb.open("https://google.com/ncr") + sb.type('[title="Search"]', "SeleniumBase GitHub page\n") + sb.click('[href*="github.com/seleniumbase/SeleniumBase"]') + sb.save_screenshot_to_logs() # (See ./latest_logs folder) + print(sb.get_page_title()) ``` -
+> `python raw_google.py`
-> ``pytest`` uses ``--chrome`` by default unless set differently.
+
+
+--------
+
+đ Here's test_get_swag.py, which tests a fake shopping site:
+ +```python +from seleniumbase import BaseCase +BaseCase.main(__name__, __file__) # Call pytest + +class MyTestClass(BaseCase): + def test_swag_labs(self): + self.open("https://www.saucedemo.com") + self.type("#user-name", "standard_user") + self.type("#password", "secret_sauce\n") + self.assert_element("div.inventory_list") + self.click('button[name*="backpack"]') + self.click("#shopping_cart_container a") + self.assert_text("Backpack", "div.cart_item") + self.click("button#checkout") + self.type("input#first-name", "SeleniumBase") + self.type("input#last-name", "Automation") + self.type("input#postal-code", "77123") + self.click("input#continue") + self.click("button#finish") + self.assert_text("Thank you for your order!") +``` + +> `pytest test_get_swag.py` + +
+
+> (The default browser is ``--chrome`` if not set.)
--------
@@ -168,7 +207,7 @@ With raw Selenium, that requires more code:đ Learn about different ways of writing tests:
-đđ Here's test_simple_login.py, which uses BaseCase class inheritance, and runs with pytest or pynose. (Use self.driver to access Selenium's raw driver.)
đđ Here's test_simple_login.py, which uses BaseCase class inheritance, and runs with pytest or pynose. (Use self.driver to access Selenium's raw driver.)
đđ Here's a test from sb_fixture_tests.py, which uses the sb pytest fixture. Runs with pytest. (Use sb.driver to access Selenium's raw driver.)
đđ Here's raw_login_sb.py, which uses the SB Context Manager. Runs with pure python. (Use sb.driver to access Selenium's raw driver.)
đđ Here's raw_login_sb.py, which uses the SB Context Manager. Runs with pure python. (Use sb.driver to access Selenium's raw driver.)
đđ Here's raw_login_context.py, which uses the DriverContext Manager. Runs with pure python. (The driver is an improved version of Selenium's raw driver, with more methods.)
đđ Here's raw_login_driver.py, which uses the Driver Manager. Runs with pure python. (The driver is an improved version of Selenium's raw driver, with more methods.)
đđ Here's raw_login_driver.py, which uses the Driver Manager. Runs with pure python. (The driver is an improved version of Selenium's raw driver, with more methods.)
đđ Here's login_app.feature, which uses behave-BDD Gherkin syntax. Runs with behave. (Learn about the SeleniumBase behave-BDD integration)
-Here's the code for my_first_test.py:
+Here's the full code for my_first_test.py:
```python from seleniumbase import BaseCase diff --git a/examples/ReadMe.md b/examples/ReadMe.md index 0f3102fc0c2..a8381af1805 100644 --- a/examples/ReadMe.md +++ b/examples/ReadMe.md @@ -6,7 +6,7 @@ * SeleniumBase tests are run with pytest. * Chrome is the default browser if not specified. -* Tests are structured using [23 unique syntax formats](https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/syntax_formats.md). +* Tests are structured using [25 unique syntax formats](https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/syntax_formats.md). * Logs from test failures are saved to ``./latest_logs/``. * Tests can be run with [multiple command-line options](https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/customizing_test_runs.md). * Examples can be found in: **[SeleniumBase/examples/](https://github.com/seleniumbase/SeleniumBase/tree/master/examples)**. diff --git a/examples/cdp_mode/ReadMe.md b/examples/cdp_mode/ReadMe.md index 5815ba409b5..ddef88e9100 100644 --- a/examples/cdp_mode/ReadMe.md +++ b/examples/cdp_mode/ReadMe.md @@ -2,7 +2,7 @@ ## [
](https://github.com/seleniumbase/SeleniumBase/) CDP Mode đ
-đ SeleniumBase CDP Mode (Chrome Devtools Protocol Mode) is a special mode inside of SeleniumBase UC Mode that lets bots appear human while controlling the browser with the CDP-Driver. Although regular UC Mode can't perform WebDriver actions while the driver is disconnected from the browser, the CDP-Driver can still perform actions while maintaining its cover.
+đ SeleniumBase CDP Mode (Chrome Devtools Protocol Mode) is a special mode inside of SeleniumBase UC Mode that lets bots appear human while controlling the browser with the CDP-Driver. Although regular UC Mode can't perform WebDriver actions while the driver is disconnected from the browser, the CDP-Driver can.
--------
@@ -13,7 +13,7 @@
đ¤ UC Mode avoids bot-detection by first disconnecting WebDriver from the browser at strategic times, calling special PyAutoGUI methods to bypass CAPTCHAs (as needed), and finally reconnecting the driver afterwards so that WebDriver actions can be performed again. Although this approach works for bypassing simple CAPTCHAs, more flexibility is needed for bypassing bot-detection on websites with advanced protection. (That's where CDP Mode comes in.)
-đ CDP Mode is based on python-cdp, trio-cdp, and nodriver. trio-cdp is an early implementation of python-cdp, and nodriver is a modern implementation of python-cdp. (Refactored Python-CDP code is imported from MyCDP.)
+đ CDP Mode is based on python-cdp, trio-cdp, and nodriver. trio-cdp is an early implementation of python-cdp, and nodriver is a modern implementation of python-cdp. (Refactored Python-CDP code is imported from MyCDP.)
đ CDP Mode includes multiple updates to the above, such as:
@@ -35,17 +35,52 @@
That disconnects WebDriver from Chrome (which prevents detection), and gives you access to `sb.cdp` methods (which don't trigger anti-bot checks).
+Simple example: ([SeleniumBase/examples/cdp_mode/raw_planetmc.py](https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/raw_planetmc.py))
+
+```python
+from seleniumbase import SB
+
+with SB(uc=True, test=True) as sb:
+ url = "www.planetminecraft.com/account/sign_in/"
+ sb.activate_cdp_mode(url)
+ sb.sleep(2)
+ sb.cdp.gui_click_element("#turnstile-widget div")
+ sb.sleep(2)
+```
+
+
+
+(If the CAPTCHA wasn't initially bypassed, then the click gets the job done.)
+
+Note that `PyAutoGUI` is an optional dependency. If calling a method that uses it when not already installed, then `SeleniumBase` will install it at run-time, which pauses the script briefly.
+
+For standard Cloudflare pages, use `sb.uc_gui_click_captcha()` if Turnstiles aren't initially bypassed. Example: ([SeleniumBase/examples/cdp_mode/raw_gitlab.py](https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/raw_gitlab.py))
+
+```python
+from seleniumbase import SB
+
+with SB(uc=True, test=True, locale_code="en") as sb:
+ url = "https://gitlab.com/users/sign_in"
+ sb.activate_cdp_mode(url)
+ sb.uc_gui_click_captcha()
+ sb.sleep(2)
+```
+
+
+
+--------
+
### đ Here are a few common `sb.cdp` methods:
-* `sb.cdp.click(selector)`
+* `sb.cdp.click(selector)` (Uses the CDP API to click)
* `sb.cdp.click_if_visible(selector)`
-* `sb.cdp.gui_click_element(selector)`
+* `sb.cdp.gui_click_element(selector)` (Uses `PyAutoGUI`)
* `sb.cdp.type(selector, text)`
-* `sb.cdp.press_keys(selector, text)`
+* `sb.cdp.press_keys(selector, text)` (Human-speed `type`)
* `sb.cdp.select_all(selector)`
* `sb.cdp.get_text(selector)`
-When `type()` is too fast, use the slower `press_keys()` to avoid detection. You can also use `sb.sleep(seconds)` to slow things down. Methods that start with `sb.cdp.gui` use `PyAutoGUI` for interaction.
+Methods that start with `sb.cdp.gui` use `PyAutoGUI` for interaction.
To use WebDriver methods again, call:
@@ -57,15 +92,17 @@ To disconnect again, call:
* **`sb.disconnect()`**
-While disconnected, if you accidentally call a WebDriver method, then SeleniumBase will attempt to use the CDP Mode version of that method (if available). For example, if you accidentally call `sb.click(selector)` instead of `sb.cdp.click(selector)`, then your WebDriver call will automatically be redirected to the CDP Mode version. Not all WebDriver methods have a matching CDP Mode method. In that scenario, calling a WebDriver method while disconnected could raise an error, or make WebDriver automatically reconnect first.
+While disconnected, if you accidentally call a WebDriver method, then SeleniumBase will attempt to use the CDP Mode version of that method (if available). For example, if you accidentally call `sb.click(selector)` instead of `sb.cdp.click(selector)`, then your WebDriver call will automatically be redirected to the CDP Mode version. Not all WebDriver methods have a matching CDP Mode method. In that scenario, calling a WebDriver method while disconnected could raise an error, or make WebDriver automatically reconnect first.
To find out if WebDriver is connected or disconnected, call:
* **`sb.is_connected()`**
+Note: When CDP Mode is initialized from UC Mode, the WebDriver is disconnected from the browser. (The stealthy CDP-Driver takes over.)
+
--------
-### đ CDP Mode Examples ([SeleniumBase/examples/cdp_mode](https://github.com/seleniumbase/SeleniumBase/tree/master/examples/cdp_mode))
+### đ CDP Mode examples ([SeleniumBase/examples/cdp_mode](https://github.com/seleniumbase/SeleniumBase/tree/master/examples/cdp_mode))
diff --git a/examples/cdp_mode/raw_planetmc.py b/examples/cdp_mode/raw_planetmc.py
index 79647f779e2..d80ef551178 100644
--- a/examples/cdp_mode/raw_planetmc.py
+++ b/examples/cdp_mode/raw_planetmc.py
@@ -1,6 +1,6 @@
from seleniumbase import SB
-with SB(uc=True, test=True, locale_code="en") as sb:
+with SB(uc=True, test=True) as sb:
url = "www.planetminecraft.com/account/sign_in/"
sb.activate_cdp_mode(url)
sb.sleep(2)
diff --git a/examples/cdp_mode/raw_tiktok.py b/examples/cdp_mode/raw_tiktok.py
index b685f9171a3..391a0e2a43f 100644
--- a/examples/cdp_mode/raw_tiktok.py
+++ b/examples/cdp_mode/raw_tiktok.py
@@ -1,15 +1,12 @@
from seleniumbase import SB
-with SB(
- uc=True, test=True, locale_code="en", incognito=True, ad_block=True
-) as sb:
+with SB(uc=True, test=True, incognito=True, ad_block=True) as sb:
url = "https://www.tiktok.com/@startrek?lang=en"
sb.activate_cdp_mode(url)
sb.sleep(2.5)
- sb.cdp.click('button:contains("Refresh")')
+ sb.cdp.click_if_visible('button:contains("Refresh")')
sb.sleep(1.5)
print(sb.cdp.get_text('h2[data-e2e="user-bio"]'))
- for i in range(55):
+ for i in range(50):
sb.cdp.scroll_down(12)
- sb.sleep(0.05)
sb.sleep(1)
diff --git a/examples/presenter/uc_presentation_3.py b/examples/presenter/uc_presentation_3.py
index ac9f6b3e610..3f6d6ca2536 100644
--- a/examples/presenter/uc_presentation_3.py
+++ b/examples/presenter/uc_presentation_3.py
@@ -456,7 +456,7 @@ def test_presentation_3(self):
"
+ The 23 Syntax Formats / Design Patterns The 25 Syntax Formats / Design Patternsđ SeleniumBase supports multiple ways of structuring tests:
@@ -32,6 +32,8 @@
@@ -994,6 +1011,87 @@ The ``Driver()`` manager format can be used as a drop-in replacement for virtual
When using the ``Driver()`` format, you may need to activate a Virtual Display on your own if you want to run headed tests in a headless Linux environment. (See https://github.com/mdmintz/sbVirtualDisplay for details.) One such example of this is using an authenticated proxy, which is configured via a Chrome extension that is generated at runtime. (Note that regular headless mode in Chrome doesn't support extensions.)
+
+
22. The driver manager (via context manager)
+
+This format provides a pure CDP way of using SeleniumBase (without Selenium or a test runner). The async/await API is used. Here's an example:
+
+```python
+import asyncio
+import time
+from seleniumbase.undetected import cdp_driver
+
+
+async def main():
+ driver = await cdp_driver.cdp_util.start_async()
+ page = await driver.get("about:blank")
+ await page.set_locale("en")
+ await page.get("https://www.priceline.com/")
+ time.sleep(3)
+ print(await page.evaluate("document.title"))
+ element = await page.select('[data-testid*="endLocation"]')
+ await element.click_async()
+ time.sleep(1)
+ await element.send_keys_async("Boston")
+ time.sleep(2)
+
+if __name__ == "__main__":
+ loop = asyncio.new_event_loop()
+ loop.run_until_complete(main())
+```
+
+(See examples/cdp_mode/raw_async.py for the test.)
+
+
+ 24. CDP driver (async/await API. No Selenium)
+
+This format provides a pure CDP way of using SeleniumBase (without Selenium or a test runner). The expanded SB-CDP sync API is used. Here's an example:
+
+```python
+import asyncio
+from seleniumbase.core import sb_cdp
+from seleniumbase.undetected import cdp_driver
+
+
+def main():
+ url0 = "about:blank" # Set Locale code from here first
+ url1 = "https://www.priceline.com/" # (The "real" URL)
+ loop = asyncio.new_event_loop()
+ driver = cdp_driver.cdp_util.start_sync()
+ page = loop.run_until_complete(driver.get(url0))
+ sb = sb_cdp.CDPMethods(loop, page, driver)
+ sb.set_locale("en") # This test expects English locale
+ sb.open(url1)
+ sb.sleep(2.5)
+ sb.internalize_links() # Don't open links in a new tab
+ sb.click("#link_header_nav_experiences")
+ sb.sleep(3.5)
+ sb.remove_elements("msm-cookie-banner")
+ sb.sleep(1.5)
+ location = "Amsterdam"
+ where_to = 'div[data-automation*="experiences"] input'
+ button = 'button[data-automation*="experiences-search"]'
+ sb.gui_click_element(where_to)
+ sb.press_keys(where_to, location)
+ sb.sleep(1)
+ sb.gui_click_element(button)
+ sb.sleep(3)
+ print(sb.get_title())
+ print("************")
+ for i in range(8):
+ sb.scroll_down(50)
+ sb.sleep(0.2)
+ cards = sb.select_all('h2[data-automation*="product-list-card"]')
+ for card in cards:
+ print("* %s" % card.text)
+
+
+if __name__ == "__main__":
+ main()
+```
+
+(See examples/cdp_mode/raw_cdp.py for the test.)
+
--------
25. CDP driver (SB-CDP sync API. No Selenium)
diff --git a/requirements.txt b/requirements.txt
index 8f3f8fed029..a52ab7e25da 100755
--- a/requirements.txt
+++ b/requirements.txt
@@ -4,7 +4,7 @@ setuptools~=70.2;python_version<"3.10"
setuptools>=75.6.0;python_version>="3.10"
wheel>=0.45.1
attrs>=24.2.0
-certifi>=2024.8.30
+certifi>=2024.12.14
exceptiongroup>=1.2.2
websockets~=13.1;python_version<"3.9"
websockets>=14.1;python_version>="3.9"
diff --git a/seleniumbase/__version__.py b/seleniumbase/__version__.py
index dc9b0326b88..46f03196f5e 100755
--- a/seleniumbase/__version__.py
+++ b/seleniumbase/__version__.py
@@ -1,2 +1,2 @@
# seleniumbase package
-__version__ = "4.33.10"
+__version__ = "4.33.11"
diff --git a/seleniumbase/core/browser_launcher.py b/seleniumbase/core/browser_launcher.py
index c2d9ae5b140..eb50a898daf 100644
--- a/seleniumbase/core/browser_launcher.py
+++ b/seleniumbase/core/browser_launcher.py
@@ -2407,8 +2407,6 @@ def _set_firefox_options(
options.set_preference("dom.webnotifications.enabled", False)
options.set_preference("dom.disable_beforeunload", True)
options.set_preference("browser.contentblocking.database.enabled", True)
- options.set_preference("extensions.allowPrivateBrowsingByDefault", True)
- options.set_preference("extensions.PrivateBrowsing.notification", False)
options.set_preference("extensions.systemAddon.update.enabled", False)
options.set_preference("extensions.update.autoUpdateDefault", False)
options.set_preference("extensions.update.enabled", False)
diff --git a/seleniumbase/core/sb_cdp.py b/seleniumbase/core/sb_cdp.py
index 6d196672446..7cde2ee2f39 100644
--- a/seleniumbase/core/sb_cdp.py
+++ b/seleniumbase/core/sb_cdp.py
@@ -519,7 +519,20 @@ def __get_attribute(self, element, attribute):
try:
return element.get_js_attributes()[attribute]
except Exception:
- return None
+ if not attribute:
+ raise
+ try:
+ attribute_str = element.get_js_attributes()
+ locate = ' %s="' % attribute
+ if locate in attribute_str.outerHTML:
+ outer_html = attribute_str.outerHTML
+ attr_start = outer_html.find(locate) + len(locate)
+ attr_end = outer_html.find('"', attr_start)
+ value = outer_html[attr_start:attr_end]
+ return value
+ except Exception:
+ pass
+ return None
def __get_x_scroll_offset(self):
x_scroll_offset = self.loop.run_until_complete(
@@ -620,11 +633,12 @@ def click_active_element(self):
def click_if_visible(self, selector):
if self.is_element_visible(selector):
- element = self.find_element(selector)
- element.scroll_into_view()
- element.click()
- self.__slow_mode_pause_if_set()
- self.loop.run_until_complete(self.page.wait())
+ with suppress(Exception):
+ element = self.find_element(selector, timeout=0)
+ element.scroll_into_view()
+ element.click()
+ self.__slow_mode_pause_if_set()
+ self.loop.run_until_complete(self.page.wait())
def click_visible_elements(self, selector, limit=0):
"""Finds all matching page elements and clicks visible ones in order.
@@ -1094,7 +1108,14 @@ def get_element_attributes(self, selector):
)
def get_element_attribute(self, selector, attribute):
- return self.get_element_attributes(selector)[attribute]
+ attributes = self.get_element_attributes(selector)
+ with suppress(Exception):
+ return attributes[attribute]
+ locate = ' %s="' % attribute
+ value = self.get_attribute(selector, attribute)
+ if not value and locate not in attributes:
+ raise KeyError(attribute)
+ return value
def get_attribute(self, selector, attribute):
return self.find_element(selector).get_attribute(attribute)
diff --git a/seleniumbase/extensions/ReadMe.md b/seleniumbase/extensions/ReadMe.md
index 3b528c91dca..3b93bc7cfe9 100644
--- a/seleniumbase/extensions/ReadMe.md
+++ b/seleniumbase/extensions/ReadMe.md
@@ -9,4 +9,5 @@
* ad_block.zip => This extension blocks certain types of iframe ads from loading.
* disable_csp.zip => This extension disables a website's Content-Security-Policy.
* recorder.zip => Save browser actions to sessionStorage with good CSS selectors.
-* sbase_ext.zip => An extension that does nothing, but helps people learn things.
+* sbase_ext.zip => A Chromium extension that does nothing. (Used for testing purposes)
+* firefox_addon.xpi => A Firefox add-on that does nothing. (Used for testing purposes)
diff --git a/seleniumbase/extensions/firefox_addon.xpi b/seleniumbase/extensions/firefox_addon.xpi
new file mode 100644
index 00000000000..dca8e2e1231
Binary files /dev/null and b/seleniumbase/extensions/firefox_addon.xpi differ
diff --git a/seleniumbase/fixtures/base_case.py b/seleniumbase/fixtures/base_case.py
index bbb610ce212..d93fa380625 100644
--- a/seleniumbase/fixtures/base_case.py
+++ b/seleniumbase/fixtures/base_case.py
@@ -3416,6 +3416,14 @@ def safe_execute_script(self, script, *args, **kwargs):
self.activate_jquery()
return self.driver.execute_script(script, *args, **kwargs)
+ def get_element_at_x_y(self, x, y):
+ """Return element at current window's x,y coordinates."""
+ self.__check_scope()
+ self._check_browser()
+ return self.execute_script(
+ "return document.elementFromPoint(%s, %s);" % (x, y)
+ )
+
def get_gui_element_rect(self, selector, by="css selector"):
"""Very similar to element.rect, but the x, y coordinates are
relative to the entire screen, rather than the browser window.
diff --git a/seleniumbase/undetected/cdp_driver/_contradict.py b/seleniumbase/undetected/cdp_driver/_contradict.py
index e087e4cf07e..ab6cf681c9a 100644
--- a/seleniumbase/undetected/cdp_driver/_contradict.py
+++ b/seleniumbase/undetected/cdp_driver/_contradict.py
@@ -31,12 +31,12 @@ class ContraDict(dict):
def __init__(self, *args, **kwargs):
super().__init__()
- silent = kwargs.pop("silent", False)
+ # silent = kwargs.pop("silent", False)
_ = dict(*args, **kwargs)
super().__setattr__("__dict__", self)
for k, v in _.items():
- _check_key(k, self, False, silent)
+ _check_key(k, self, False, True)
super().__setitem__(k, _wrap(self.__class__, v))
def __setitem__(self, key, value):
@@ -90,7 +90,7 @@ def _wrap(cls, v):
def _check_key(
- key: str, mapping: _Mapping, boolean: bool = False, silent=False
+ key: str, mapping: _Mapping, boolean: bool = False, silent=True
):
"""Checks `key` and warns if needed.
:param key:
diff --git a/seleniumbase/undetected/cdp_driver/browser.py b/seleniumbase/undetected/cdp_driver/browser.py
index cadb8fa73bf..83ed2c133a9 100644
--- a/seleniumbase/undetected/cdp_driver/browser.py
+++ b/seleniumbase/undetected/cdp_driver/browser.py
@@ -10,6 +10,7 @@
import pickle
import re
import shutil
+import time
import urllib.parse
import urllib.request
import warnings
@@ -30,8 +31,6 @@ def get_registered_instances():
def deconstruct_browser():
- import time
-
for _ in __registered__instances__:
if not _.stopped:
_.stop()
@@ -117,8 +116,13 @@ async def create(
port=port,
**kwargs,
)
- instance = cls(config)
- await instance.start()
+ try:
+ instance = cls(config)
+ await instance.start()
+ except Exception:
+ time.sleep(0.15)
+ instance = cls(config)
+ await instance.start()
return instance
def __init__(self, config: Config, **kwargs):
@@ -379,8 +383,6 @@ async def start(self=None) -> Browser:
--------------------------------
Failed to connect to the browser
--------------------------------
- Possibly because you are running as "root".
- If so, you may need to use no_sandbox=True.
"""
)
)
diff --git a/setup.py b/setup.py
index 9ae77f4b8fb..8a50ea512c7 100755
--- a/setup.py
+++ b/setup.py
@@ -153,7 +153,7 @@
'setuptools>=75.6.0;python_version>="3.10"',
'wheel>=0.45.1',
'attrs>=24.2.0',
- "certifi>=2024.8.30",
+ "certifi>=2024.12.14",
"exceptiongroup>=1.2.2",
'websockets~=13.1;python_version<"3.9"',
'websockets>=14.1;python_version>="3.9"',
