You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/academy/anti_scraping.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -73,7 +73,7 @@ Solely based on the way how the bots operate. It comperes data-rich pages visits
73
73
74
74
By definition, this is not an anti-scraping method, but it can heavily affect the reliability of a scraper. If your target website drastically changes its CSS selectors, and your scraper is heavily reliant on selectors, it could break. In principle, websites using this method change their HTML structure or CSS selectors randomly and frequently, making the parsing of the data harder, and requiring more maintenance of the bot.
75
75
76
-
One of the best ways of avoiding the possible breaking of your scraper due to website structure changes is to limit your reliance on data from HTML elements as much as possible (see [API Scraping]({{@link api_scraping.md}}) and [JavaScript objects within HTML]({{@linktutorials/js_in_html.md}}))
76
+
One of the best ways of avoiding the possible breaking of your scraper due to website structure changes is to limit your reliance on data from HTML elements as much as possible (see [API Scraping]({{@link api_scraping.md}}) and [JavaScript objects within HTML]({{@link js_in_html.md}}))
Copy file name to clipboardExpand all lines: content/academy/dealing_with_dynamic_pages.md
+6-5Lines changed: 6 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,9 +1,10 @@
1
1
---
2
2
title: Dealing with dynamic pages
3
3
description: Learn about dynamic pages and dynamic content. How can we find out if a page is dynamic? How do we programmatically scrape dynamic content?
4
-
menuWeight: 1
4
+
menuWeight: 13
5
+
category: tutorials
5
6
paths:
6
-
- tutorials/dealing-with-dynamic-pages
7
+
- dealing-with-dynamic-pages
7
8
---
8
9
9
10
# [](#dealing-with-dynamic-pages) Dealing with dynamic pages
@@ -16,7 +17,7 @@ In this lesson, we'll be discussing dynamic content and how to scrape it while u
16
17
17
18
From our adored and beloved [Fakestore](https://demo-webstore.apify.org/), we have been tasked to scrape each product's title, price, and image from the [new arrivals](https://demo-webstore.apify.org/search/new-arrivals) page. Easy enough! We did something very similar in the previous modules.
18
19
19
-

20
+

20
21
21
22
First, create a file called **dynamic.js** and copy-paste the following boiler plate code into it:
After running it, you might say, "Great! It works!" **But wait...** What are those results being logged to console?
80
81
81
-

82
+

82
83
83
84
Every single image seems to have the same exact "URL," but they are most definitely not the image URLs we are looking for. This is strange, because in the browser, we were getting URLs that looked like this:
Well... Not quite. It seems that the only images which we got the full links to were the ones that were being displayed within the view of the browser. This means that the images are lazy-loaded. **Lazy-loading** is a common technique used across the web to improve performance. Lazy-loaded items allow the user to load content incrementally, as they perform some action. In most cases, including our current one, this action is scrolling.
0 commit comments