-
Notifications
You must be signed in to change notification settings - Fork 96
Open
Labels
Description
I did a small tweak so we can have custom folder for local cache DB.
`
class RandomUserAgentMiddleware(object):
def init(self, crawler):
super(RandomUserAgentMiddleware, self).init()
loc = crawler.settings.get('LOCATION')
if loc is None:
self.ua = UserAgent()
else:
self.ua = UserAgent(path=crawler.settings.get('LOCATION'))
self.per_proxy = crawler.settings.get('RANDOM_UA_PER_PROXY', False)
self.proxy2ua = {}`
In settings file, u can have an option like:
LOCATION = '/home/user/fake_useragent%s.json' % fake_useragent.VERSION