Change User Agent?

[mattFlemingTamr]

Hi,

I was curious how to change the user agent in conjunction with using this scraper.

-Matt

[Andreilys]

Hey Matt, you can change the user agent when you're initializing CustomRequest, the parameter is list_user_agent.

c = CustomRequest()

c = CustomRequest(list_user_agent=user_agent, list_proxies=list_proxies)

[ericfourrier]

Thanks for your help @Andreilys, @mattFlemingTamr you need to do the following (there is actually a typo in the docs) :

c=CustomRequest(list_user_agent=user_agent, rotate_ua=True, list_proxies=None)

Nevertheless i think it won't be enough to bypass LinkedIn ban even if you pass a GoogleBot user agent or anything similar.

[Andreilys]

Yes seems like the ban is very hard to get around, have you been able to implement anything on your end @ericfourrier ? Couldn't get this to work no matter what I tried (Even Selenium failed). I'm only looking to scrape 50-60 profiles, so not much

[ericfourrier]

I have tried some stuff too the only thing that worked (1 year ago) was buying a pool of residential ips but now it seems even residential ips are quickly banned.