Provide a ScrapingClient that doesn't need API access #5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pR0Ps
force-pushed
the
feature/standalone-scraping
branch
from
b2f0204 to
923a1c3
Compare
pR0Ps
force-pushed
the
feature/standalone-scraping
branch
from
923a1c3 to
13737f7
Compare
pR0Ps
force-pushed
the
feature/standalone-scraping
branch
2 times, most recently
from
9a82176 to
d8ed33a
Compare
Also adds the ability to list activities using web scraping instead of the API. The activities are returned as `ScrapedActivity` objects that are mostly compatible with the normal `Activity` objects that are returned by the list activities function that uses the API.
This should be done by the library consumer if it's needed
It's not going to be perfect, but the idea is that for the most basic of cases it should be a pretty close replacement. The goal is to keep the amount of work to support both API and scraping-based clients to a minimum. To support this, the WebClient now uses delegation instead of inheritance to add scraper-based functionality. This enables the `ScrapingClient` class to use the same function names without automatically overriding the `stravalib.Client` functions when used through the `WebClient` class.
The default used to be to just download the JSON blob. It was changed to request the GPX format instead since this is a more standardized format for an activity.
Now accepts (but ignores) parameters that the `stravalib` version accepts
- Make pagination actually work (forgot to increment page number) - Handle stopping based on the `before` param - Properly handle workout types
Replaces `get_bike_components`
BeautifulSoup v4.9.0 changed how `.text` works for `<script>` tags (ie. not at all), breaking parsing. See https://bazaar.launchpad.net/~leonardr/beautifulsoup/bs4/revision/564
pR0Ps
force-pushed
the
feature/standalone-scraping
branch
from
d8ed33a to
3685464
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also adds the ability to list activities using web scraping instead of the API. The activities are returned as
ScrapedActivityobjects that are mostly compatible with the normalActivityobjects that are returned by the list activities function that uses the API.Fixes #4
NOTE:
stravalibmoving to Pydantic for its models is going to break a LOT of this. Will need some work.