Skip to content

Commit ec83b52

Browse files
vzuchervzucher
authored
Merge pull request #8 from vzucher/cli-feature
Cli feature
2 parents 9f66214 + f1e2250 commit ec83b52

File tree

10 files changed

+1388
-0
lines changed

10 files changed

+1388
-0
lines changed

‎pyproject.toml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,12 @@ dependencies = [
1818
"pydantic>=2.0.0",
1919
"pydantic-settings>=2.0.0",
2020
"aiolimiter>=1.1.0",
21+
"click>=8.1.0",
2122
]
2223

24+
[project.scripts]
25+
brightdata = "brightdata.cli.main:main"
26+
2327
[project.optional-dependencies]
2428
dev = [
2529
"pytest>=7.4.0",

‎requirements.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,5 @@ python-dotenv>=1.0.0
44
tldextract>=5.0.0
55
pydantic>=2.0.0
66
pydantic-settings>=2.0.0
7+
click>=8.1.0
78

‎src/brightdata/cli/README.md‎

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
# Bright Data CLI
2+
3+
Command-line interface for Bright Data Python SDK.
4+
5+
## Installation
6+
7+
The CLI is automatically installed with the SDK:
8+
9+
```bash
10+
pip install brightdata-sdk
11+
```
12+
13+
## Usage
14+
15+
### Authentication
16+
17+
All commands require an API key. You can provide it in three ways:
18+
19+
1. **Command-line flag** (highest priority):
20+
```bash
21+
brightdata scrape amazon products --api-key YOUR_API_KEY https://amazon.com/dp/...
22+
```
23+
24+
2. **Environment variable**:
25+
```bash
26+
export BRIGHTDATA_API_TOKEN=YOUR_API_KEY
27+
brightdata scrape amazon products https://amazon.com/dp/...
28+
```
29+
30+
3. **Interactive prompt** (if neither is provided):
31+
```bash
32+
brightdata scrape amazon products https://amazon.com/dp/...
33+
# Will prompt: Enter your Bright Data API key:
34+
```
35+
36+
### Scrape Commands (URL-based extraction)
37+
38+
#### Generic Scraper
39+
```bash
40+
brightdata scrape generic <URL> [--country CODE] [--response-format FORMAT]
41+
```
42+
43+
#### Amazon
44+
```bash
45+
# Products
46+
brightdata scrape amazon products <URL> [--timeout SECONDS]
47+
48+
# Reviews
49+
brightdata scrape amazon reviews <URL> [--past-days DAYS] [--keyword KEYWORD] [--num-reviews NUM] [--timeout SECONDS]
50+
51+
# Sellers
52+
brightdata scrape amazon sellers <URL> [--timeout SECONDS]
53+
```
54+
55+
#### LinkedIn
56+
```bash
57+
# Profiles
58+
brightdata scrape linkedin profiles <URL> [--timeout SECONDS]
59+
60+
# Posts
61+
brightdata scrape linkedin posts <URL> [--timeout SECONDS]
62+
63+
# Jobs
64+
brightdata scrape linkedin jobs <URL> [--timeout SECONDS]
65+
66+
# Companies
67+
brightdata scrape linkedin companies <URL> [--timeout SECONDS]
68+
```
69+
70+
#### Facebook
71+
```bash
72+
# Posts by profile
73+
brightdata scrape facebook posts-by-profile <URL> [--num-posts NUM] [--start-date DATE] [--end-date DATE] [--timeout SECONDS]
74+
75+
# Posts by group
76+
brightdata scrape facebook posts-by-group <URL> [--num-posts NUM] [--start-date DATE] [--end-date DATE] [--timeout SECONDS]
77+
78+
# Posts by URL
79+
brightdata scrape facebook posts-by-url <URL> [--timeout SECONDS]
80+
81+
# Comments
82+
brightdata scrape facebook comments <URL> [--num-comments NUM] [--start-date DATE] [--end-date DATE] [--timeout SECONDS]
83+
84+
# Reels
85+
brightdata scrape facebook reels <URL> [--num-posts NUM] [--start-date DATE] [--end-date DATE] [--timeout SECONDS]
86+
```
87+
88+
#### Instagram
89+
```bash
90+
# Profiles
91+
brightdata scrape instagram profiles <URL> [--timeout SECONDS]
92+
93+
# Posts
94+
brightdata scrape instagram posts <URL> [--timeout SECONDS]
95+
96+
# Comments
97+
brightdata scrape instagram comments <URL> [--timeout SECONDS]
98+
99+
# Reels
100+
brightdata scrape instagram reels <URL> [--timeout SECONDS]
101+
```
102+
103+
#### ChatGPT
104+
```bash
105+
brightdata scrape chatgpt prompt <PROMPT> [--country CODE] [--web-search] [--additional-prompt PROMPT] [--timeout SECONDS]
106+
```
107+
108+
### Search Commands (Parameter-based discovery)
109+
110+
#### SERP Services
111+
```bash
112+
# Google
113+
brightdata search google <QUERY> [--location LOCATION] [--language CODE] [--device TYPE] [--num-results NUM]
114+
115+
# Bing
116+
brightdata search bing <QUERY> [--location LOCATION] [--language CODE] [--num-results NUM]
117+
118+
# Yandex
119+
brightdata search yandex <QUERY> [--location LOCATION] [--language CODE] [--num-results NUM]
120+
```
121+
122+
#### LinkedIn Search
123+
```bash
124+
# Posts
125+
brightdata search linkedin posts <PROFILE_URL> [--start-date DATE] [--end-date DATE] [--timeout SECONDS]
126+
127+
# Profiles
128+
brightdata search linkedin profiles <FIRST_NAME> [--last-name LAST_NAME] [--timeout SECONDS]
129+
130+
# Jobs
131+
brightdata search linkedin jobs [--url URL] [--keyword KEYWORD] [--location LOCATION] [--country CODE] [--remote] [--timeout SECONDS]
132+
```
133+
134+
#### ChatGPT Search
135+
```bash
136+
brightdata search chatgpt prompt <PROMPT> [--country CODE] [--web-search] [--secondary-prompt PROMPT] [--timeout SECONDS]
137+
```
138+
139+
#### Instagram Search
140+
```bash
141+
# Posts
142+
brightdata search instagram posts <URL> [--num-posts NUM] [--start-date DATE] [--end-date DATE] [--post-type TYPE] [--timeout SECONDS]
143+
144+
# Reels
145+
brightdata search instagram reels <URL> [--num-posts NUM] [--start-date DATE] [--end-date DATE] [--timeout SECONDS]
146+
```
147+
148+
### Output Options
149+
150+
All commands support output formatting:
151+
152+
```bash
153+
# JSON format (default)
154+
brightdata scrape amazon products <URL> --output-format json
155+
156+
# Pretty format (human-readable)
157+
brightdata scrape amazon products <URL> --output-format pretty
158+
159+
# Minimal format (just the data)
160+
brightdata scrape amazon products <URL> --output-format minimal
161+
162+
# Save to file
163+
brightdata scrape amazon products <URL> --output-file results.json
164+
```
165+
166+
### Examples
167+
168+
```bash
169+
# Scrape Amazon product
170+
brightdata scrape amazon products https://amazon.com/dp/B0123456 --api-key YOUR_KEY
171+
172+
# Search Google
173+
brightdata search google "python tutorial" --location "United States" --num-results 20
174+
175+
# Scrape LinkedIn profile
176+
brightdata scrape linkedin profiles https://linkedin.com/in/johndoe
177+
178+
# Search LinkedIn jobs
179+
brightdata search linkedin jobs --keyword "python developer" --location "New York" --remote
180+
181+
# Scrape Instagram profile
182+
brightdata scrape instagram profiles https://instagram.com/username
183+
184+
# Send ChatGPT prompt
185+
brightdata scrape chatgpt prompt "Explain async programming" --web-search --country us
186+
```
187+
188+
## Help
189+
190+
Get help for any command:
191+
192+
```bash
193+
brightdata --help
194+
brightdata scrape --help
195+
brightdata scrape amazon --help
196+
brightdata search --help
197+
```
198+

‎src/brightdata/cli/__init__.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
"""
2+
Bright Data CLI - Command-line interface for Bright Data SDK.
3+
4+
Provides easy access to all search and scrape tools through a unified CLI.
5+
"""
6+
7+
from .main import cli
8+
9+
__all__ = ["cli"]
10+

‎src/brightdata/cli/banner.py‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
"""
2+
ANSI art banner for Bright Data Python SDK CLI.
3+
"""
4+
5+
import sys
6+
import os
7+
8+
9+
def _supports_color() -> bool:
10+
"""Check if terminal supports ANSI colors."""
11+
# Check if we're in a terminal
12+
if not hasattr(sys.stdout, 'isatty') or not sys.stdout.isatty():
13+
return False
14+
15+
# Windows 10+ supports ANSI colors
16+
if sys.platform == "win32":
17+
# Check if Windows version supports ANSI
18+
try:
19+
import ctypes
20+
kernel32 = ctypes.windll.kernel32
21+
# Enable ANSI escape sequences on Windows
22+
kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)
23+
return True
24+
except:
25+
return False
26+
27+
# Check for common environment variables
28+
if os.getenv("TERM") in ("xterm", "xterm-256color", "screen", "screen-256color"):
29+
return True
30+
31+
return False
32+
33+
34+
def get_banner() -> str:
35+
"""
36+
Get ANSI art banner for Bright Data Python SDK.
37+
38+
Returns:
39+
Formatted banner string with colors
40+
"""
41+
banner = """
42+
43+
\033[1;33m██████╗ ██████╗ ██╗ ██████╗ ██╗ ██╗████████╗\033[0m
44+
\033[1;33m██╔══██╗██╔══██╗██║██╔════╝ ██║ ██║╚══██╔══╝\033[0m
45+
\033[1;33m██████╔╝██████╔╝██║██║ ███╗███████║ ██║ \033[0m
46+
\033[1;33m██╔══██╗██╔══██╗██║██║ ██║██╔══██║ ██║ \033[0m
47+
\033[1;33m██████╔╝██║ ██║██║╚██████╔╝██║ ██║ ██║ \033[0m
48+
\033[1;33m╚═════╝ ╚═╝ ╚═╝╚═╝ ╚═════╝ ╚═╝ ╚═╝ ╚═╝ \033[0m
49+
50+
\033[1;35m██████╗ █████╗ ████████╗ █████╗ \033[0m
51+
\033[1;35m██╔══██╗██╔══██╗╚══██╔══╝██╔══██╗\033[0m
52+
\033[1;35m██║ ██║███████║ ██║ ███████║\033[0m
53+
\033[1;35m██║ ██║██╔══██║ ██║ ██╔══██║\033[0m
54+
\033[1;35m██████╔╝██║ ██║ ██║ ██║ ██║\033[0m
55+
\033[1;35m╚═════╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝\033[0m
56+
57+
\033[1;32m██████╗ ██╗ ██╗████████╗██╗ ██╗ ██████╗ ███╗ ██╗\033[0m
58+
\033[1;32m██╔══██╗╚██╗ ██╔╝╚══██╔══╝██║ ██║██╔═══██╗████╗ ██║\033[0m
59+
\033[1;32m██████╔╝ ╚████╔╝ ██║ ███████║██║ ██║██╔██╗ ██║\033[0m
60+
\033[1;32m██╔═══╝ ╚██╔╝ ██║ ██╔══██║██║ ██║██║╚██╗██║\033[0m
61+
\033[1;32m██║ ██║ ██║ ██║ ██║╚██████╔╝██║ ╚████║\033[0m
62+
\033[1;32m╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝\033[0m
63+
64+
\033[1;37m███████╗██████╗ ██╗ ██╗\033[0m
65+
\033[1;37m██╔════╝██╔══██╗██║ ██╔╝\033[0m
66+
\033[1;37m███████╗██║ ██║█████╔╝ \033[0m
67+
\033[1;37m╚════██║██║ ██║██╔═██╗ \033[0m
68+
\033[1;37m███████║██████╔╝██║ ██╗\033[0m
69+
\033[1;37m╚══════╝╚═════╝ ╚═╝ ╚═╝\033[0m
70+
71+
\033[1;93m🐍\033[0m
72+
73+
"""
74+
return banner
75+
76+
77+
def print_banner() -> None:
78+
"""Print the banner to stdout with proper encoding and color support."""
79+
# Enable color support on Windows
80+
supports_color = _supports_color()
81+
82+
banner = get_banner()
83+
84+
# If no color support, strip ANSI codes
85+
if not supports_color:
86+
import re
87+
# Remove ANSI escape sequences
88+
ansi_escape = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])')
89+
banner = ansi_escape.sub('', banner)
90+
91+
# Ensure UTF-8 encoding for Windows compatibility
92+
try:
93+
if hasattr(sys.stdout, 'buffer') and sys.stdout.encoding != 'utf-8':
94+
sys.stdout.buffer.write(banner.encode('utf-8'))
95+
sys.stdout.buffer.write(b'\n')
96+
else:
97+
print(banner)
98+
except (AttributeError, UnicodeEncodeError):
99+
# Fallback: print without special characters
100+
print(banner.encode('ascii', 'ignore').decode('ascii'))

‎src/brightdata/cli/commands/__init__.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
"""
2+
CLI command groups for scrape and search operations.
3+
"""
4+
5+
from .scrape import scrape_group
6+
from .search import search_group
7+
8+
__all__ = ["scrape_group", "search_group"]
9+

0 commit comments

Comments
 (0)