Skip to content

Commit 28b38e4

Browse files
authored
Merge pull request #67 from scrapy-plugins/dont-proxy-docs
Dont proxy docs
2 parents a484690 + f28d4c3 commit 28b38e4

File tree

1 file changed

+12
-2
lines changed

1 file changed

+12
-2
lines changed

docs/index.rst

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,19 @@ How to use it
5555

5656

5757
With the middleware, the usage of crawlera is automatic, every request will go through crawlera without nothing to worry about.
58+
If you want to *disable* crawlera on a specific Request, you can do so by updating `meta` with `dont_proxy=True`::
5859

59-
Remember that you are now making request to Crawlera, and the Crawlera service will be the one actually making the requests to the different sites.
60+
61+
scrapy.Request(
62+
'http://example.com',
63+
meta={
64+
'dont_proxy': True,
65+
...
66+
},
67+
)
68+
69+
70+
Remember that you are now making requests to Crawlera, and the Crawlera service will be the one actually making the requests to the different sites.
6071

6172
If you need to specify special `Crawlera Headers <https://doc.scrapinghub.com/crawlera.html#request-headers>`_, just apply them as normal `Scrapy Headers <https://doc.scrapy.org/en/latest/topics/request-response.html#scrapy.http.Request.headers>`_.
6273

@@ -78,7 +89,6 @@ requests with `DEFAULT_REQUEST_HEADERS <http://doc.scrapy.org/en/1.0/topics/sett
7889
but keep sending ``X-Crawlera-*`` headers in your requests, those will be removed from the
7990
request headers.
8091

81-
8292
This Middleware also adds some configurable Scrapy Settings, check :ref:`the complete list here <settings>`.
8393

8494
All the rest

0 commit comments

Comments
 (0)