Skip to content

Commit 69429d7

Browse files
committed
rotating_proxies
1 parent 9f4ff88 commit 69429d7

File tree

2 files changed

+6
-6
lines changed

2 files changed

+6
-6
lines changed

content/academy/expert_scraping_with_apify/bypassing_anti_scraping.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: VII - Bypassing anti-scraping methods
3-
description: Learn about bypassing anti-bot methods with proxies and proxy/session rotation. Use the Apify SDK to abstract away the overheads that come with these concepts.
3+
description: Learn about bypassing anti-bot methods with proxies and proxy/session rotation. Use Crawlee and the Apify SDK to abstract away the overheads that come with these concepts.
44
menuWeight: 6.7
55
paths:
66
- expert-scraping-with-apify/bypassing-anti-scraping
@@ -20,7 +20,7 @@ You might have already noticed that we've been using the **RESIDENTIAL** proxy g
2020
- Give the [proxy documentation](https://docs.apify.com/proxy#our-proxies) a solid readover (feel free to skip most of the examples).
2121
- Read about our [proxy trial](https://help.apify.com/en/articles/2108625-free-trial-of-apify-proxy).
2222
- Check out [this](https://help.apify.com/en/articles/1961361-several-tips-on-how-to-bypass-website-anti-scraping-protections) article about bypassing anti-scraping measures.
23-
- Gain a solid understanding of the [SessionPool](https://sdk.apify.com/docs/api/session-pool).
23+
- Gain a solid understanding of the [SessionPool](https://crawlee.dev/api/core/class/SessionPool).
2424
- Look at a few actors on the [Apify store](https://apify.com/store). How are they utilizing proxies?
2525

2626
## [](#quiz) Knowledge check 📝

content/academy/expert_scraping_with_apify/solutions/rotating_proxies.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ paths:
1111
If you take a look at our current code for the Amazon scraping actor, you might notice this snippet:
1212

1313
```JavaScript
14-
const proxyConfiguration = await Apify.createProxyConfiguration({
14+
const proxyConfiguration = await Actor.createProxyConfiguration({
1515
groups: ['RESIDENTIAL'],
1616
});
1717
```
@@ -27,7 +27,7 @@ In order to rotate sessions, we must utilize the [**SessionPool**](https://crawl
2727
Let's go ahead and add a **sessionPoolOptions** key to our crawler's configuration so that we can modify the default settings:
2828

2929
```JavaScript
30-
const crawler = new Apify.CheerioCrawler({
30+
const crawler = new CheerioCrawler({
3131
requestList,
3232
requestQueue,
3333
proxyConfiguration,
@@ -50,7 +50,7 @@ const crawler = new Apify.CheerioCrawler({
5050
Now, we'll use the **maxUsageCount** key to force each session to be thrown away after 5 uses, and **maxErrorScore** to trash a session once it receives an error.
5151

5252
```JavaScript
53-
const crawler = new Apify.CheerioCrawler({
53+
const crawler = new CheerioCrawler({
5454
requestList,
5555
requestQueue,
5656
proxyConfiguration,
@@ -73,7 +73,7 @@ And that's it! We've successfully configured the session pool to match the task'
7373
The final requirement was to only use proxies from the US. Back in our **ProxyConfiguration**, we just need to add the **countryCode** key and set it to **US**:
7474

7575
```JavaScript
76-
const proxyConfiguration = await Apify.createProxyConfiguration({
76+
const proxyConfiguration = await Actor.createProxyConfiguration({
7777
groups: ['RESIDENTIAL'],
7878
countryCode: 'US',
7979
});

0 commit comments

Comments
 (0)