This update is all about threads⚡! Three main large changes:
- Adds twitter threads tracking via --depth 2
*Tweets within a thread now have their tweets who they are replying to saved as parent_id.- Screenshots are taken for each thread.
- Adds multi-threading support via --multi-threading N
- Massive internal architectural changes.
Changes
- Added
--depthand-dfor archiving threads. - Added
--multi-threadingand-tto specify number of threads to use. - Added
parent_idtotweets.jsonwhich tracks direct tweet responses. - Added
threading.pywhich manages thread workers and queues. - Added
pages.pywhich abstracts away a page on Twitter. Useful for multi-threading.- Added
TwitterPageused for representing a page in Twitter. - Added
TwitterBiowhich is used for fetching and obtaining bio pages. - Added
TwitterThreadwhich represents a tweet thread page.
- Added
- Added
TweetExtractorwhich attempts to fetch all available Tweets on a page. - Added
Scrollerwhich abstracts away scrolling metrics. - Added thread expansion functions:
hit_more_replies()andget_recommend_tweets_height(). - Changed random messages from ChromeDriver, makes logs cleaner.
- Fixed potential for some posts to be skipped on ad removal.
- Fixed logger not outputting any info.
- Fixed logger not printing debug.