Merge pull request #681 from seleniumbase/support-the-contains-selector

Add support for the ":contains()" selector

Author: mdmintz

help_docs/happy_customers.md

@@ -9,6 +9,7 @@
 * [MIT](https://web.mit.edu/)
 * [Harvard Medical School](https://hms.harvard.edu/)
 * [Mississippi State University](https://www.msstate.edu/)
+* [StreamSets](https://streamsets.com/)
 * [Akamai](https://www.akamai.com/)
 * [VMware](https://www.vmware.com/)
 * [Baidu](https://www.baidu.com/)
@@ -22,7 +23,7 @@
 
 <h3>Case Study: (<i>HubSpot</i>)</h3>
 
-In addition to using SeleniumBase for testing the UI of their content management system, HubSpot used SeleniumBase to automate the migration of website pages from one content management system to another, which saved them over one million US dollars and a significant amount of time.
+In addition to using SeleniumBase for testing the UI of their content management system, HubSpot used SeleniumBase to automate the migration of website pages from one content management system to another, which saved them over one million USD and a significant amount of time.
 
 Learn how HubSpot uses SeleniumBase for website testing by reading: [Automated Testing with Selenium](https://dev.hubspot.com/blog/bid/88880/Automated-Integration-Testing-with-Selenium-at-HubSpot#hs_cos_wrapper_name)
 

help_docs/method_summary.md

@@ -309,6 +309,8 @@ self.inspect_html()
 
 self.get_google_auth_password(totp_key=None)
 
+self.convert_css_to_xpath(css)
+
 self.convert_xpath_to_css(xpath)
 
 self.convert_to_css_selector(selector, by)

integrations/github/ReadMe.md

@@ -2,3 +2,4 @@
 
 > **Table of Contents / Navigation:**
 > - [**Actions/Workflows**](https://github.com/seleniumbase/SeleniumBase/blob/master/integrations/github/workflows/ReadMe.md)
+> - [**Extras/Action-Integrations**](https://github.com/seleniumbase/SeleniumBase/blob/master/integrations/github/workflows/extras.md)

integrations/github/workflows/ReadMe.md

@@ -51,35 +51,3 @@
 ### Congratulations! You now know how to create and run browser tests with GitHub Actions/Workflows!
 
 ### **Study [SeleniumBase](https://github.com/seleniumbase/SeleniumBase) to learn more!**
-
-----------
-----------
-
-### **Integrations for GitHub Actions:**
-
-### Slack Notifications - [rtCamp/action-slack-notify](https://github.com/rtCamp/action-slack-notify) can be used to send notifications to Slack.
-
-**Usage:**
-* Create a slack integration webhook if you don't have one already.
-* Create a ``SLACK_WEBHOOK`` secret on your repository with the webhook token value.
-* For this particular action, ``SLACK_CHANNEL`` is an optional environment variable that defaults to the webhook token channel if not specified.
-* The following example shows how to put a link to your workflow as the ``SLACK_MESSAGE`` (Lets you see artifacts pushed up, such as from the SeleniumBase Presenter feature!):
-```
-    - name: Slack notification
-      uses: rtCamp/action-slack-notify@master
-      env:
-        SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
-        SLACK_CHANNEL: general
-        SLACK_ICON_EMOJI: rocket
-        SLACK_USERNAME: SeleniumBase
-        SLACK_MESSAGE: 'Actions workflow completed successful! :tada:  https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}'
-```
-
-### Uploading Artifacts:
-* Here's an example using [upload-artifact@v2](https://github.com/actions/upload-artifact) to push up a SeleniumBase-generated presentation as an artifact. (You can use this together with the Slack notification action to view the presentation directly from GitHub)
-```
-    - uses: actions/upload-artifact@v2
-      with:
-        name: Click to download the presentation
-        path: saved_presentations/my_presentation.html
-```

integrations/github/workflows/extras.md

@@ -0,0 +1,28 @@
+### <img src="https://seleniumbase.io/img/sb_icon.png" title="SeleniumBase" width="30" /> Integrations for GitHub Actions:
+
+### Uploading Artifacts:
+* Here's an example using [upload-artifact@v2](https://github.com/actions/upload-artifact) to push up a SeleniumBase-generated artifact.
+```
+    - uses: actions/upload-artifact@v2
+      with:
+        name: Click to download the presentation
+        path: saved_presentations/my_presentation.html
+```
+
+### Slack Notifications - [rtCamp/action-slack-notify](https://github.com/rtCamp/action-slack-notify) can be used to send notifications to Slack.
+
+**Usage:**
+* Create a slack integration webhook if you don't have one already.
+* Create a ``SLACK_WEBHOOK`` secret on your repository with the webhook token value.
+* For this particular action, ``SLACK_CHANNEL`` is an optional environment variable that defaults to the webhook token channel if not specified.
+* The following example shows how to put a link to your workflow as the ``SLACK_MESSAGE`` (Lets you see artifacts pushed up, such as from the SeleniumBase Presenter feature!):
+```
+    - name: Slack notification
+      uses: rtCamp/action-slack-notify@master
+      env:
+        SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
+        SLACK_CHANNEL: general
+        SLACK_ICON_EMOJI: rocket
+        SLACK_USERNAME: SeleniumBase
+        SLACK_MESSAGE: 'Actions workflow completed successful! :tada:  https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}'
+```

requirements.txt

@@ -1,7 +1,7 @@
 pip>=20.2.2
 packaging>=20.4
 setuptools>=44.1.1;python_version<"3.5"
-setuptools>=50.1.0;python_version>="3.5"
+setuptools>=50.2.0;python_version>="3.5"
 setuptools-scm>=4.1.2
 wheel>=0.35.1
 six==1.15.0
@@ -17,6 +17,7 @@ selenium==3.141.0
 msedge-selenium-tools==3.141.2
 more-itertools==5.0.0;python_version<"3.5"
 more-itertools==8.5.0;python_version>="3.5"
+cssselect==1.1.0
 pluggy==0.13.1
 attrs>=20.1.0
 py==1.8.1;python_version<"3.5"
@@ -44,7 +45,7 @@ pyopenssl==19.1.0
 pygments==2.5.2;python_version<"3.5"
 pygments==2.6.1;python_version>="3.5"
 traitlets==4.3.3;python_version<"3.7"
-traitlets==5.0.2;python_version>="3.7"
+traitlets==5.0.3;python_version>="3.7"
 prompt-toolkit==1.0.18;python_version<"3.6"
 prompt-toolkit==3.0.7;python_version>="3.6"
 ipython==5.10.0;python_version<"3.5"

seleniumbase/fixtures/base_case.py

@@ -45,6 +45,7 @@ def test_anything(self):
 from seleniumbase.core import log_helper
 from seleniumbase.core import tour_helper
 from seleniumbase.fixtures import constants
+from seleniumbase.fixtures import css_to_xpath
 from seleniumbase.fixtures import js_utils
 from seleniumbase.fixtures import page_actions
 from seleniumbase.fixtures import page_utils
@@ -1429,8 +1430,8 @@ def __select_option(self, dropdown_selector, option,
             timeout = settings.SMALL_TIMEOUT
         if self.timeout_multiplier and timeout == settings.SMALL_TIMEOUT:
             timeout = self.__get_new_timeout(timeout)
-        if page_utils.is_xpath_selector(dropdown_selector):
-            dropdown_by = By.XPATH
+        dropdown_selector, dropdown_by = self.__recalculate_selector(
+            dropdown_selector, dropdown_by)
         self.wait_for_ready_state_complete()
         element = self.wait_for_element_present(
             dropdown_selector, by=dropdown_by, timeout=timeout)
@@ -2984,6 +2985,9 @@ def get_google_auth_password(self, totp_key=None):
         totp = pyotp.TOTP(totp_key)
         return str(totp.now())
 
+    def convert_css_to_xpath(self, css):
+        return css_to_xpath.convert_css_to_xpath(css)
+
     def convert_xpath_to_css(self, xpath):
         return xpath_to_css.convert_xpath_to_css(xpath)
 
@@ -5902,6 +5906,9 @@ def __recalculate_selector(self, selector, by):
             name = page_utils.get_name_from_selector(selector)
             selector = '[name="%s"]' % name
             by = By.CSS_SELECTOR
+        if ":contains(" in selector and by == By.CSS_SELECTOR:
+            selector = self.convert_css_to_xpath(selector)
+            by = By.XPATH
         return (selector, by)
 
     def __looks_like_a_page_url(self, url):

seleniumbase/fixtures/css_to_xpath.py

@@ -0,0 +1,15 @@
+"""
+Convert CSS selectors into XPath selectors
+"""
+
+from cssselect import GenericTranslator
+
+
+def convert_css_to_xpath(css):
+    """ Convert CSS Selectors to XPath Selectors.
+        Example:
+            convert_css_to_xpath('button:contains("Next")')
+            Output => "//button[contains(., 'Next')]"
+    """
+    xpath = GenericTranslator().css_to_xpath(css, prefix='//')
+    return xpath

seleniumbase/fixtures/xpath_to_css.py

@@ -76,6 +76,7 @@ def _filter_xpath_grouping(xpath):
 
 def _get_raw_css_from_xpath(xpath):
     css = ""
+    attr = ""
     position = 0
 
     while position < len(xpath):
@@ -108,7 +109,9 @@ def _get_raw_css_from_xpath(xpath):
                 attr = '[%s*="%s"]' % (match['cattr'].replace("@", ""),
                                        match['cvalue'])
             elif match['cattr'] == "text()":
-                attr = ":contains(%s)" % match['cvalue']
+                attr = ':contains("%s")' % match['cvalue']
+            elif match['cattr'] == ".":
+                attr = ':contains("%s")' % match['cvalue']
         else:
             attr = ""
 
@@ -128,6 +131,20 @@ def _get_raw_css_from_xpath(xpath):
 def convert_xpath_to_css(xpath):
     if xpath[0] != '"' and xpath[-1] != '"' and xpath.count('"') % 2 == 0:
         xpath = _handle_brackets_in_strings(xpath)
+    xpath = xpath.replace("descendant-or-self::*/", "descORself/")
+    xpath = xpath.replace(" = '", "='")
+    if " and contains(@" in xpath and xpath.count(" and contains(@") == 1:
+        spot1 = xpath.find(" and contains(@")
+        spot1 = spot1 + len(" and contains(@")
+        spot2 = xpath.find(",", spot1)
+        attr = xpath[spot1:spot2]
+        swap = " and contains(@%s, " % attr
+        if swap in xpath:
+            swap_spot = xpath.find(swap)
+            close_paren = xpath.find(']', swap_spot) - 1
+            if close_paren > 1:
+                xpath = xpath[:close_paren] + xpath[close_paren+1:]
+                xpath = xpath.replace(swap, "_STAR_=")
 
     if xpath.startswith('('):
         xpath = _filter_xpath_grouping(xpath)
@@ -149,4 +166,16 @@ def convert_xpath_to_css(xpath):
     css = css.replace('_STR_L_bracket_', '\\[')
     css = css.replace('_STR_R_bracket_', '\\]')
 
+    # Handle a lot of edge cases with conversion
+    css = css.replace(" > descORself > ", ' ')
+    css = css.replace(" descORself > ", ' ')
+    css = css.replace("/descORself/*", ' ')
+    css = css.replace("/descORself/", ' ')
+    css = css.replace("descORself/", ' ')
+    css = css.replace("_STAR_=", "*=")
+    css = css.replace("]/", "] ")
+    css = css.replace("] *[", "] > [")
+    css = css.replace("\'", '"')
+    css = css.replace("[@", '[')
+
     return css

setup.py

@@ -54,7 +54,7 @@
 
 setup(
     name='seleniumbase',
-    version='1.49.1',
+    version='1.49.2',
     description='Web Automation and Test Framework - https://seleniumbase.io',
     long_description=long_description,
     long_description_content_type='text/markdown',
@@ -93,7 +93,7 @@
         'pip>=20.2.2',
         'packaging>=20.4',
         'setuptools>=44.1.1;python_version<"3.5"',
-        'setuptools>=50.1.0;python_version>="3.5"',
+        'setuptools>=50.2.0;python_version>="3.5"',
         'setuptools-scm',
         'wheel>=0.35.1',
         'six',
@@ -109,6 +109,7 @@
         'msedge-selenium-tools==3.141.2',
         'more-itertools==5.0.0;python_version<"3.5"',
         'more-itertools==8.5.0;python_version>="3.5"',
+        'cssselect==1.1.0',
         'pluggy==0.13.1',
         'attrs>=20.1.0',
         'py==1.8.1;python_version<"3.5"',
@@ -136,7 +137,7 @@
         'pygments==2.5.2;python_version<"3.5"',
         'pygments==2.6.1;python_version>="3.5"',
         'traitlets==4.3.3;python_version<"3.7"',
-        'traitlets==5.0.2;python_version>="3.7"',
+        'traitlets==5.0.3;python_version>="3.7"',
         'ipython==5.10.0;python_version<"3.5"',
         'prompt-toolkit==1.0.18;python_version<"3.6"',
         'prompt-toolkit==3.0.7;python_version>="3.6"',