-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Add slimmed-down Selenium project code #596
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 10 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
56c2360
Add Selenium project code
martin-martin eb7db86
Merge branch 'master' into python-selenium
bzaczynski 648edc9
Apply Technical Review suggestions
martin-martin 32b9670
Merge branch 'python-selenium' of https://github.com/realpython/mater…
martin-martin 79a06fc
Merge branch 'master' into python-selenium
martin-martin a0fd072
Restructures project to work with /discover page
martin-martin ae7cdf5
Re-apply old TR feedback
martin-martin 07280d0
Fix linter error
martin-martin a0a3391
Fix black formatting
martin-martin d17eb80
Merge branch 'master' into python-selenium
martin-martin 21d505c
Apply TR feedback
martin-martin b6b18e8
Fix double-exit loop
martin-martin 37a10cf
Refix ruff to black
martin-martin ee77eb2
Merge branch 'master' into python-selenium
martin-martin 446dec2
Update project code, add training files
martin-martin 67f6d93
Format black
martin-martin 4605c10
Merge branch 'master' into python-selenium
martin-martin 79b9b8a
Rename modules, improve CLI output, rename entry point
martin-martin ef6919a
blacken
martin-martin 3f35024
Merge branch 'python-selenium' of https://github.com/realpython/mater…
martin-martin bf5326a
Add implicit wait
martin-martin 4d0304f
Merge branch 'master' into python-selenium
bzaczynski 7f22ab0
Add package info
martin-martin d80539b
Merge branch 'python-selenium' of https://github.com/realpython/mater…
martin-martin 87d63aa
Language edit
brendaweles d813234
Merge branch 'master' into python-selenium
brendaweles File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
# Modern Web Automation With Python and Selenium | ||
|
||
This repository contains the module `bandcamp`, which is the sample app built in the Real Python tutorial [Modern Web Automation With Python and Selenium](https://realpython.com/modern-web-automation-with-python-and-selenium/). | ||
|
||
## Installation and Setup | ||
|
||
Create and activate a [Python virtual environment](https://realpython.com/python-virtual-environments-a-primer/). | ||
|
||
Then, install the requirements: | ||
|
||
```sh | ||
(venv) $ python -m pip install -r requirements.txt | ||
``` | ||
|
||
The only direct dependency for this project is [Selenium](https://selenium-python.readthedocs.io/). You should use a Python version of at least 3.10, which is necessary to support [structural pattern matching](https://realpython.com/structural-pattern-matching/). | ||
|
||
You'll need a [Firefox Selenium driver](https://selenium-python.readthedocs.io/installation.html#drivers) called `geckodriver` to run the project as-is. Make sure to [download and install](https://github.com/mozilla/geckodriver/releases) it before running the project. | ||
|
||
## Run the Bandcamp Discover Player | ||
|
||
To run the music player, install the package, then use the entry point command from your command-line: | ||
|
||
```sh | ||
(venv) $ python -m pip install . | ||
(venv) $ bandcamp-player | ||
``` | ||
|
||
You'll see a text-based user interface that allows you to interact with the music player: | ||
bzaczynski marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
``` | ||
Type: play [<track_number>] | pause | tracks | more | exit | ||
> | ||
``` | ||
|
||
Type one of the available commands to interact with Bandcamp's Discover section through your headless browser. Listen to songs with `play`, list available tracks with `tracks`, and load more songs using `more`. You can exit the music player by typing `exit`. | ||
|
||
## About the Authors | ||
|
||
Martin Breuss - Email: [email protected] | ||
Bartosz Zaczyński - Email: [email protected] | ||
|
||
## License | ||
|
||
Distributed under the MIT license. See `LICENSE` for more information. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
[build-system] | ||
requires = ["setuptools", "wheel"] | ||
build-backend = "setuptools.build_meta" | ||
|
||
[project] | ||
name = "bandcamp_player" | ||
version = "0.1.0" | ||
requires-python = ">=3.10" | ||
description = "A web player for Bandcamp using Selenium" | ||
authors = [ | ||
{ name = "Martin Breuss", email = "[email protected]" }, | ||
{ name = "Bartosz Zaczyński", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"selenium", | ||
] | ||
[project.scripts] | ||
bandcamp-player = "bandcamp.__main__:main" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
attrs==25.2.0 | ||
certifi==2025.1.31 | ||
h11==0.14.0 | ||
idna==3.10 | ||
outcome==1.3.0.post0 | ||
pysocks==1.7.1 | ||
selenium==4.29.0 | ||
sniffio==1.3.1 | ||
sortedcontainers==2.4.0 | ||
trio==0.29.0 | ||
trio-websocket==0.12.2 | ||
typing-extensions==4.12.2 | ||
urllib3==2.3.0 | ||
websocket-client==1.8.0 | ||
wsproto==1.2.0 |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
from bandcamp.app.tui import interact | ||
|
||
|
||
def main(): | ||
"""Provide the main entry point for the app.""" | ||
interact() | ||
|
||
|
||
main() |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
from selenium.webdriver import Firefox | ||
from selenium.webdriver.firefox.options import Options | ||
|
||
from bandcamp.web.element import TrackElement | ||
from bandcamp.web.page import DiscoverPage | ||
|
||
BANDCAMP_DISCOVER_URL = "https://bandcamp.com/discover/" | ||
|
||
|
||
class Player: | ||
"""Play tracks from Bandcamp's Discover page.""" | ||
|
||
def __init__(self) -> None: | ||
self._driver = self._set_up_driver() | ||
self.page = DiscoverPage(self._driver) | ||
self.tracklist = self.page.discover_tracklist | ||
self._current_track = TrackElement( | ||
self.tracklist.available_tracks[0], self._driver | ||
) | ||
|
||
def __enter__(self): | ||
return self | ||
|
||
def __exit__(self, exc_type, exc_value, exc_tb): | ||
"""Close the headless browser.""" | ||
self._driver.quit() | ||
|
||
def play(self, track_number=None): | ||
"""Play the first track, or one of the available numbered tracks.""" | ||
if track_number: | ||
self._current_track = TrackElement( | ||
self.tracklist.available_tracks[track_number - 1], | ||
self._driver, | ||
) | ||
self._current_track.play() | ||
|
||
def pause(self): | ||
"""Pause the current track.""" | ||
self._current_track.pause() | ||
|
||
def _set_up_driver(self): | ||
"""Create a headless browser pointing to Bandcamp.""" | ||
options = Options() | ||
options.add_argument("--headless") | ||
browser = Firefox(options=options) | ||
browser.get(BANDCAMP_DISCOVER_URL) | ||
return browser |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
from bandcamp.app.player import Player | ||
|
||
COLUMN_WIDTH = CW = 30 | ||
MAX_TRACKS = 100 # Allows to load more tracks once. | ||
|
||
|
||
def interact(): | ||
"""Control the player through user interactions.""" | ||
with Player() as player: | ||
while True: | ||
print( | ||
"\nType: play [<track number>] | pause | tracks | more | exit" | ||
) | ||
match input("> ").strip().lower().split(): | ||
case ["play"]: | ||
play(player) | ||
case ["play", track]: | ||
try: | ||
track_number = int(track) | ||
play(player, track_number) | ||
except ValueError: | ||
print("Please provide a valid track number.") | ||
case ["pause"]: | ||
pause(player) | ||
case ["tracks"]: | ||
display_tracks(player) | ||
case ["more"] if len( | ||
player.tracklist.available_tracks | ||
) >= MAX_TRACKS: | ||
print( | ||
"Can't load more tracks. Pick one from the track list." | ||
) | ||
case ["more"]: | ||
player.tracklist.load_more() | ||
display_tracks(player) | ||
case ["exit"]: | ||
print("Exiting the player...") | ||
break | ||
case _: | ||
print("Unknown command. Try again.") | ||
|
||
|
||
def play(player, track_number=None): | ||
"""Play a track and shows info about the track.""" | ||
player.play(track_number) | ||
print(player._current_track._get_track_info()) | ||
bzaczynski marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
def pause(player): | ||
"""Pause the current track.""" | ||
player.pause() | ||
|
||
|
||
def display_tracks(player): | ||
"""Display information about the currently playable tracks.""" | ||
header = f"{'#':<5} {'Album':<{CW}} {'Artist':<{CW}} {'Genre':<{CW}}" | ||
print(header) | ||
print("-" * 100) | ||
for track_number, track in enumerate( | ||
player.tracklist.available_tracks, start=1 | ||
): | ||
if track.text: | ||
album, artist, *genre = track.text.split("\n") | ||
album = _truncate(album, CW) | ||
artist = _truncate(artist, CW) | ||
genre = _truncate(genre[0], CW) if genre else "" | ||
print( | ||
f"{track_number:<5} {album:<{CW}} {artist:<{CW}} {genre:<{CW}}" | ||
) | ||
|
||
|
||
def _truncate(text, width): | ||
"""Truncate track information.""" | ||
return text[: width - 3] + "..." if len(text) > width else text |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
from dataclasses import dataclass | ||
|
||
from selenium.webdriver.remote.webdriver import WebDriver | ||
from selenium.webdriver.remote.webelement import WebElement | ||
from selenium.webdriver.support.wait import WebDriverWait | ||
|
||
MAX_WAIT_SECONDS = 10.0 | ||
DEFAULT_WINDOW_SIZE = (1920, 3000) # Shows 60 tracks | ||
|
||
|
||
@dataclass | ||
class Track: | ||
album: str | ||
artist: str | ||
genre: str | ||
url: str | ||
|
||
|
||
class WebPage: | ||
def __init__(self, driver: WebDriver) -> None: | ||
self._driver = driver | ||
self._driver.set_window_size(*DEFAULT_WINDOW_SIZE) | ||
self._wait = WebDriverWait(driver, MAX_WAIT_SECONDS) | ||
|
||
|
||
class WebComponent(WebPage): | ||
def __init__(self, parent: WebElement, driver: WebDriver) -> None: | ||
super().__init__(driver) | ||
self._parent = parent |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
from selenium.common.exceptions import NoSuchElementException | ||
from selenium.webdriver.remote.webdriver import WebDriver | ||
from selenium.webdriver.remote.webelement import WebElement | ||
from selenium.webdriver.support import expected_conditions as EC | ||
|
||
from bandcamp.web.base import Track, WebComponent | ||
from bandcamp.web.locators import DiscoverPageLocator, TrackLocator | ||
|
||
|
||
class TrackElement(WebComponent, TrackLocator): | ||
"""Model a playable track in Bandcamp's Discover section.""" | ||
|
||
def play(self) -> None: | ||
"""Play the track.""" | ||
if not self.is_playing: | ||
self._get_play_button().click() | ||
|
||
def pause(self) -> None: | ||
"""Pause the track.""" | ||
if self.is_playing: | ||
self._get_play_button().click() | ||
|
||
@property | ||
def is_playing(self) -> bool: | ||
return "Pause" in self._get_play_button().get_attribute("aria-label") | ||
|
||
def _get_track_info(self) -> Track: | ||
"""Create a representation of the track's relevant information.""" | ||
full_url = self._parent.find_element(*self.URL).get_attribute("href") | ||
# Cut off the referrer query parameter | ||
clean_url = full_url.split("?")[0] if full_url else "" | ||
# Some tracks don't have a genre | ||
try: | ||
genre = self._parent.find_element(*self.GENRE).text | ||
except NoSuchElementException: | ||
genre = "" | ||
return Track( | ||
album=self._parent.find_element(*self.ALBUM).text, | ||
artist=self._parent.find_element(*self.ARTIST).text, | ||
genre=genre, | ||
url=clean_url, | ||
) | ||
|
||
def _get_play_button(self): | ||
return self._parent.find_element(*self.PLAY_BUTTON) | ||
|
||
|
||
class DiscoverTrackList(WebComponent, DiscoverPageLocator, TrackLocator): | ||
bzaczynski marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"""Model the track list in Bandcamp's Discover section.""" | ||
|
||
def __init__(self, parent: WebElement, driver: WebDriver = None) -> None: | ||
super().__init__(parent, driver) | ||
self.available_tracks = self._get_available_tracks() | ||
|
||
def load_more(self) -> None: | ||
"""Load additional tracks in the Discover section.""" | ||
view_more_button = self._driver.find_element(*self.PAGINATION_BUTTON) | ||
view_more_button.click() | ||
# The button is disabled until all new tracks are loaded. | ||
self._wait.until(EC.element_to_be_clickable(self.PAGINATION_BUTTON)) | ||
self.available_tracks = self._get_available_tracks() | ||
|
||
def _get_available_tracks(self) -> list: | ||
"""Find all currently available tracks in the Discover section.""" | ||
self._wait.until( | ||
lambda driver: any( | ||
e.is_displayed() and e.text.strip() | ||
for e in driver.find_elements(*self.ITEM) | ||
), | ||
message="Timeout waiting for track text to load", | ||
) | ||
bzaczynski marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
all_items = self._driver.find_elements(*self.ITEM) | ||
all_tracks = [] | ||
for item in all_items: | ||
if item.find_element(*self.PLAY_BUTTON): | ||
all_tracks.append(item) | ||
|
||
# Filter tracks that are displayed and have text. | ||
return [ | ||
track | ||
for track in all_tracks | ||
if track.is_displayed() and track.text.strip() | ||
] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
from selenium.webdriver.common.by import By | ||
|
||
|
||
class DiscoverPageLocator: | ||
DISCOVER_RESULTS = (By.CLASS_NAME, "results-grid") | ||
ITEM = (By.CLASS_NAME, "results-grid-item") | ||
PAGINATION_BUTTON = (By.ID, "view-more") | ||
COOKIE_ACCEPT_NECESSARY = ( | ||
By.CSS_SELECTOR, | ||
"#cookie-control-dialog button.g-button.outline", | ||
) | ||
|
||
|
||
class TrackLocator: | ||
PLAY_BUTTON = (By.CSS_SELECTOR, "button.play-pause-button") | ||
URL = (By.CSS_SELECTOR, "div.meta p a") | ||
ALBUM = (By.CSS_SELECTOR, "div.meta p a strong") | ||
GENRE = (By.CSS_SELECTOR, "div.meta p.genre") | ||
ARTIST = (By.CSS_SELECTOR, "div.meta p a span") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
from selenium.webdriver.remote.webdriver import WebDriver | ||
|
||
from bandcamp.web.base import WebPage | ||
from bandcamp.web.element import DiscoverTrackList | ||
from bandcamp.web.locators import DiscoverPageLocator | ||
|
||
|
||
class DiscoverPage(WebPage, DiscoverPageLocator): | ||
"""Model the relevant parts of the Bandcamp Discover page.""" | ||
|
||
def __init__(self, driver: WebDriver) -> None: | ||
super().__init__(driver) | ||
self._accept_cookie_consent() | ||
self.discover_tracklist = DiscoverTrackList( | ||
self._driver.find_element(*self.DISCOVER_RESULTS), self._driver | ||
) | ||
|
||
def _accept_cookie_consent(self) -> None: | ||
"""Accept the necessary cookie consent.""" | ||
self._driver.find_element(*self.COOKIE_ACCEPT_NECESSARY).click() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.