Skip to content

Commit 5a6828a

Browse files
committed
fixes
1 parent 5837889 commit 5a6828a

File tree

2 files changed

+15
-3
lines changed

2 files changed

+15
-3
lines changed

README.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1431,7 +1431,14 @@ In such situations, the only solution is to switch to a different proxy provider
14311431

14321432
Some good proxy providers we personally use are:
14331433

1434-
- For Rotating Datacenter Proxies: **BrightData Datacenter Proxies**, which cost around $0.6 per GB on a pay-as-you-go basis. No KYC is required.
1434+
- For Rotating Datacenter Proxies:
1435+
1436+
- **[requests-ip-rotator](https://pypi.org/project/requests-ip-rotator/):**
1437+
Routes your API requests through AWS API Gateway, leveraging AWS’s large IP pool to automatically rotate IPs. The cost is negligible even for large-scale scraping (millions of pages). **Highly recommended** — this should be your default choice for datacenter proxy rotation.
1438+
1439+
- **BrightData Datacenter Proxies:**
1440+
Paid alternative costing around **$0.6 per GB** (pay-as-you-go). Offers a smaller proxy pool compared to AWS.
1441+
14351442
- For Rotating Residential Proxies: **IPRoyal Royal Residential Proxies**, which cost around $7 per GB on a pay-as-you-go basis. No KYC is required.
14361443

14371444
As always, nothing good in life comes free. Proxies are expensive, and will take up almost all of your scraping costs.

github-docs/change-ip.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,17 @@ Here's how you can enable and disable airplane mode to change your IP:
2626

2727
For devices connected to a Wi-Fi network, turning off the router and then turning it back on will change the IP. It is a slow method, and the router will take 2-3 minutes to re-establish connection.
2828

29-
**#3 Use Proxies**
29+
**#3 Use AWS Api Gateway**
30+
If your goal is to use datacenter IPs for sending requests, use the [requests-ip-rotator](https://pypi.org/project/requests-ip-rotator/) library. It routes your API requests through AWS API Gateway, leveraging AWS’s large IP pool to automatically rotate IPs.
31+
32+
Commercial datacenter proxies are expensive and cost around **$0.6/GB**, but this approach provides same benefit at a much much lower cost. Very Much Recommended.
33+
34+
**#4 Use Proxies**
3035
Paid services like BrightData, Oxylab, and IPRoyal provide a pool of residential IPes. However, they can be quite expensive, with prices averaging around $15 per GB for residential IPs.
3136

3237
If you opt for this method and are scraping large amounts of data, consider disabling the loading of CSS and images to reduce costs.
3338

34-
**#4 Use Tor Network**
39+
**#5 Use Tor Network**
3540
The Tor network is a decentralized network that anonymizes your internet traffic by routing it through multiple volunteer-operated relays.
3641

3742
If you have access to a Windows device, you can use the [AnonSurf Application](https://github.com/ultrafunkamsterdam/AnonSurf) by ultrafunkamsterdam.

0 commit comments

Comments
 (0)