-
Notifications
You must be signed in to change notification settings - Fork 96
Open
Labels
Description
In Scrapy 2.10 RetryMiddleware
was updated (PR scrapy/scrapy#5929) to deprecate EXCEPTIONS_TO_RETRY
tuple. There is an error (issue scrapy/scrapy#6049) that makes RetryUserAgentMiddleware
instances to throw AttributeError
upon accessing EXCEPTIONS_TO_RETRY
.
To resolve this problem and add support for new RETRY_EXCEPTIONS
setting, __init__
must be updated in the following way:
def __init__(self, crawler: Crawler):
RetryMiddleware.__init__(self, crawler.settings)
RandomUserAgentBase.__init__(self, crawler)
if hasattr(self, 'exceptions_to_retry'):
self.EXCEPTIONS_TO_RETRY = self.exceptions_to_retry
Such solution maintains support for previous Scrapy versions.
alecxe, dapicester, amd-yan, honzajavorek, alberand and 2 more