Add User-Agent rotation system (#198)#199
Conversation
Changes: 1. Added get_random_user_agent() function in core/helpers.py - 18 modern User-Agents (Windows, macOS, iPhone, Android) - Covers Chrome, Firefox, Safari, Edge, Samsung Internet - Updated 2026-02 for current versions 2. Updated substack.py to use get_random_user_agent() - Demonstrates the new helper function The remaining modules can be updated similarly: - user_scan/creator/hashnode.py - user_scan/creator/medium.py - user_scan/creator/producthunt.py - user_scan/creator/twitch.py - user_scan/gaming/chess_com.py - user_scan/gaming/monkeytype.py - user_scan/gaming/battlenet.py - user_scan/social/youtube.py - user_scan/social/x.py Issue: kaifcodec#198
|
@ssiweifnag Appreciate you efforts but I'm going to close this PR for now. While the Re-fork the latest |
|
@ssiweifnag One final note: your Git identity is currently set to Since that's a local email, GitHub can't link these commits to your account, so you won't show up in the Contributors list. If you want credit for the work, make sure to set your global |
完成了 issue kaifcodec#198 要求的 User-Agent rotation 系統。 變更內容: - 在所有 75 個 modules 中替換 hardcoded User-Agent - 使用 get_random_user_agent() 函數進行隨機輪換 - 覆蓋 email_scan 和 user_scan 所有模組 相比 PR kaifcodec#199: - PR kaifcodec#199: 僅添加了 get_random_user_agent() 函數 - 本 PR: 替換了所有 75 個模組的 User-Agent 讓 scanner 更難被指紋識別,提升隱私性。 Co-authored-by: Szu-Wei Fang <panboo@gmail.com>
Summary
This PR implements a User-Agent rotation system to reduce fingerprinting when scanning, as requested in issue #198.
Changes
1. Added
get_random_user_agent()inuser_scanner/core/helpers.pyA new function that returns a random User-Agent string from a curated list of 18 modern user agents:
2. Updated
substack.pyas DemoThe
substack.pymodule now usesget_random_user_agent()instead of hardcoded User-Agent.Remaining Modules
The following modules still use hardcoded User-Agents and can be updated similarly:
user_scan/creator/hashnode.pyuser_scan/creator/medium.pyuser_scan/creator/producthunt.pyuser_scan/creator/twitch.pyuser_scan/gaming/chess_com.pyuser_scan/gaming/monkeytype.pyuser_scan/gaming/battlenet.pyuser_scan/social/youtube.pyuser_scan/social/x.pyIssue Reference
#198