Are Musk,s new limitations impacting this module?

[banajana]

These lines of code worked fine just a few weeks ago and now I'm getting TypeError: object of type 'NoneType' has no len()

This is the code I'm using:
from twitter_scraper_selenium import get_profile_details

twitter_username = "tim_cook"
filename = "twitter_dummy_ceo"
get_profile_details(twitter_username=twitter_username, filename=filename)

[shaikhsajid1111]

Yes, this module is supposed to scrape public data only. Now, we can't access pages without logging in which leads to a blocker

[ansisme]

so does that mean we cant scrape data at all? without logging in ?

[shaikhsajid1111]

Yeah, without logging in it is not possible currently

[feusagittaire]

Yeah, without logging in it is not possible currently

I how could we implement the logging in? Could you show how in code?

[ansisme]

Yeah, without logging in it is not possible currently

I have logged in, still not working, what now?

[banajana]

So scraping tweets now works, although I'm getting random 10, not the latest 10. I'm using this code:
from twitter_scraper_selenium import scrape_profile

scrape_profile(twitter_username="tim_cook",output_format="csv",browser="firefox",tweets_count=10,filename="tim_cook_tweetsV3",directory="C:/Users/MCOBPHD19/OneDrive/Documents/Dissertation")

But scraping profile details doesn't. Using:
from twitter_scraper_selenium import get_profile_details

twitter_username = "tim_cook"
filename = "timCookProfile"
get_profile_details(twitter_username=twitter_username, filename=filename)

The error I get is this. Is this still the authentication issue? If so, why can I now scrape tweets?
TypeError Traceback (most recent call last)
C:\Users\MCOBPH~1\AppData\Local\Temp/ipykernel_3168/1681914123.py in
4 twitter_username = "tim_cook"
5 filename = "timCookProfile"
----> 6 get_profile_details(twitter_username=twitter_username, filename=filename)

~\anaconda3\lib\site-packages\twitter_scraper_selenium\profile_details.py in get_profile_details(twitter_username, proxy, filename, directory)
56 # if filename was not provided then print the JSON to console
57 return json.dumps(data)
---> 58 elif filename != '' and len(data) > 0:
59 # if filename was provided, save it to that file
60 mode = 'w'

TypeError: object of type 'NoneType' has no len()

[eloragh]

So scraping tweets now works, although I'm getting random 10, not the latest 10. I'm using this code: from twitter_scraper_selenium import scrape_profile

scrape_profile(twitter_username="tim_cook",output_format="csv",browser="firefox",tweets_count=10,filename="tim_cook_tweetsV3",directory="C:/Users/MCOBPHD19/OneDrive/Documents/Dissertation")

But scraping profile details doesn't. Using: from twitter_scraper_selenium import get_profile_details

twitter_username = "tim_cook" filename = "timCookProfile" get_profile_details(twitter_username=twitter_username, filename=filename)

The error I get is this. Is this still the authentication issue? If so, why can I now scrape tweets? TypeError Traceback (most recent call last) C:\Users\MCOBPH~1\AppData\Local\Temp/ipykernel_3168/1681914123.py in 4 twitter_username = "tim_cook" 5 filename = "timCookProfile" ----> 6 get_profile_details(twitter_username=twitter_username, filename=filename)

~\anaconda3\lib\site-packages\twitter_scraper_selenium\profile_details.py in get_profile_details(twitter_username, proxy, filename, directory) 56 # if filename was not provided then print the JSON to console 57 return json.dumps(data) ---> 58 elif filename != '' and len(data) > 0: 59 # if filename was provided, save it to that file 60 mode = 'w'

TypeError: object of type 'NoneType' has no len()

I'm struggling with this same issue. Is there any workaround?

[shaikhsajid1111]

There are still problems for users scraping without logging in. The tweets are not present the same as it is present for logged-in users on Twitter