Skip to content

Commit 0b9ed30

Browse files
committed
feat: kick off the page
1 parent 1b14a63 commit 0b9ed30

File tree

1 file changed

+13
-0
lines changed

1 file changed

+13
-0
lines changed

sources/academy/webscraping/scraping_basics_python/13_platform.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,19 @@ sidebar_position: 13
66
slug: /scraping-basics-python/platform
77
---
88

9+
import Exercises from './_exercises.mdx';
10+
11+
**In this lesson, we'll deploy our application to a scraping platform that automatically runs it daily. We'll also use the platform's API to retrieve and work with the results.**
12+
13+
---
14+
15+
Before starting with a scraping platform, let's point out several caveats in our current solution:
16+
17+
- **User-operated:** We have to run the scraper ourselves. If we're interested in price trends, we'd have to remember to run the program every day. If we want to be notified about a big discount, having a program we need to run manually isn't much of an improvement over manually opening the web page in our browser every day.
18+
- **No monitoring:** If we have a spare server or a RapsberryPi under table, we could use [cron](https://en.wikipedia.org/wiki/Cron) to schedule the program, but even then we'd have little visibility into whether it finished successfully, what errors or warnings occur, how long it runs, or what resources it consumes.
19+
- **Manual data management:** To keep track of prices in time, we'd have to figure out a way how to organize the exported data. If we wanted to process the data, we might find out that different data analysis tools require different formats.
20+
- **Prone to anti-scraping:** If the target website detects we're scraping their data, they can rate-limit or even block us. We could take a laptop to a nearby coffee place and run the program while connected to their public wi-fi, but eventually they'll probably block that one too, which puts you in a serious hazard of angrying your barista.
21+
922
:::danger Work in progress
1023

1124
This course is incomplete. As we work on adding new lessons, we would love to hear your feedback. You can comment right here under each page or [file a GitHub Issue](https://github.com/apify/apify-docs/issues) to discuss a problem.

0 commit comments

Comments
 (0)