Skip to content

Commit 18248ee

Browse files
thalissonvsthalissonvs
committed
docs: update README to highlight new browser preferences and HTTP requests
1 parent f4c6bc9 commit 18248ee

File tree

7 files changed

+937
-80
lines changed

7 files changed

+937
-80
lines changed

‎README.md‎

Lines changed: 123 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,10 @@
1818

1919
<p align="center">
2020
📖 <a href="https://autoscrape-labs.github.io/pydoll/">Documentation</a> •
21-
🚀 <a href="#getting-started">Getting Started</a> •
22-
⚡ <a href="#advanced-features">Advanced Features</a> •
23-
🤝 <a href="#contributing">Contributing</a> •
24-
💖 <a href="#support-my-work">Support My Work</a>
21+
🚀 <a href="#-getting-started">Getting Started</a> •
22+
⚡ <a href="#-advanced-features">Advanced Features</a> •
23+
🤝 <a href="#-contributing">Contributing</a> •
24+
💖 <a href="#-support-my-work">Support My Work</a>
2525
</p>
2626

2727
Imagine the following scenario: you need to automate tasks in your browser. Maybe it's testing a web application, collecting data from a site, or even automating repetitive processes. Normally this involves using external drivers, complex configurations, and many compatibility issues.
@@ -45,43 +45,144 @@ We believe that powerful automation shouldn't require you to become an expert in
4545
- **Humanized Interactions**: Mimic real user behavior
4646
- **Simplicity**: With Pydoll, you install and you're ready to automate.
4747

48-
## What's New in 2.4.0
48+
## What's New
4949

50-
### Advanced browser preferences support (thanks to [@LucasAlvws](https://github.com/LucasAlvws))
51-
You can now customize Chromium browser preferences through the `browser_preferences` dict in ChromiumOptions.<br><br>
52-
Set things like download directory, language, notification blocking, PDF handling, and more.
53-
Helper properties like `set_default_download_directory`, `set_accept_languages`, and `prompt_for_download` were added for convenience.
54-
Preferences are merged automatically, no need to redefine everything.<br><br>
55-
Here's an example:
50+
### Browser-context HTTP requests - game changer for hybrid automation!
51+
Ever wished you could make HTTP requests that automatically inherit all your browser's session state? **Now you can!**<br>
52+
The `tab.request` property gives you a beautiful `requests`-like interface that executes HTTP calls directly in the browser's JavaScript context. This means every request automatically gets cookies, authentication headers, CORS policies, and session state, just as if the browser made the request itself.
5653

54+
**Perfect for Hybrid Automation:**
55+
```python
56+
# Navigate to a site and login normally with PyDoll
57+
await tab.go_to('https://example.com/login')
58+
await (await tab.find(id='username')).type_text('[email protected]')
59+
await (await tab.find(id='password')).type_text('password')
60+
await (await tab.find(id='login-btn')).click()
61+
62+
# Now make API calls that inherit the logged-in session!
63+
response = await tab.request.get('https://example.com/api/user/profile')
64+
user_data = response.json()
65+
66+
# POST data while staying authenticated
67+
response = await tab.request.post(
68+
'https://example.com/api/settings',
69+
json={'theme': 'dark', 'notifications': True}
70+
)
71+
72+
# Access response content in different formats
73+
raw_data = response.content
74+
text_data = response.text
75+
json_data = response.json()
76+
77+
# Check cookies that were set
78+
for cookie in response.cookies:
79+
print(f"Cookie: {cookie['name']} = {cookie['value']}")
80+
81+
# Add custom headers to your requests
82+
headers = [
83+
{'name': 'X-Custom-Header', 'value': 'my-value'},
84+
{'name': 'X-API-Version', 'value': '2.0'}
85+
]
86+
87+
await tab.request.get('https://api.example.com/data', headers=headers)
88+
89+
```
90+
91+
**Why this is great:**
92+
- **No more session juggling** - Requests inherit browser cookies automatically
93+
- **CORS just works** - Requests respect browser security policies
94+
- **Perfect for modern SPAs** - Seamlessly mix UI automation with API calls
95+
- **Authentication made easy** - Login once via UI, then hammer APIs
96+
- **Hybrid workflows** - Use the best tool for each step (UI or API)
97+
98+
This opens up incredible possibilities for automation scenarios where you need both browser interaction AND API efficiency!
99+
100+
### Total browser control with custom preferences! (thanks to [@LucasAlvws](https://github.com/LucasAlvws))
101+
Want to completely customize how Chrome behaves? **Now you can control EVERYTHING!**<br>
102+
The new `browser_preferences` system gives you access to hundreds of internal Chrome settings that were previously impossible to change programmatically. We're talking about deep browser customization that goes way beyond command-line flags!
103+
104+
**The possibilities are endless:**
57105
```python
58106
options = ChromiumOptions()
59-
options.browser_preferences = { # you can set the entire dict
107+
108+
# Create the perfect automation environment
109+
options.browser_preferences = {
60110
'download': {
61111
'default_directory': '/tmp/downloads',
62-
'prompt_for_download': False
63-
},
64-
'intl': {
65-
'accept_languages': 'en-US,en,pt-BR'
112+
'prompt_for_download': False,
113+
'directory_upgrade': True,
114+
'extensions_to_open': '' # Don't auto-open any downloads
66115
},
67116
'profile': {
68117
'default_content_setting_values': {
69-
'notifications': 2 # Block notifications
70-
}
118+
'notifications': 2, # Block all notifications
119+
'geolocation': 2, # Block location requests
120+
'media_stream_camera': 2, # Block camera access
121+
'media_stream_mic': 2, # Block microphone access
122+
'popups': 1 # Allow popups (useful for automation)
123+
},
124+
'password_manager_enabled': False, # Disable password prompts
125+
'exit_type': 'Normal' # Always exit cleanly
126+
},
127+
'intl': {
128+
'accept_languages': 'en-US,en',
129+
'charset_default': 'UTF-8'
130+
},
131+
'browser': {
132+
'check_default_browser': False, # Don't ask about default browser
133+
'show_update_promotion_infobar': False
71134
}
72135
}
73136

74-
options.set_default_download_directory('/tmp/downloads') # or just the individual properties
75-
options.set_accept_languages('en-US,en,pt-BR')
137+
# Or use the convenient helper methods
138+
options.set_default_download_directory('/tmp/downloads')
139+
options.set_accept_languages('en-US,en,pt-BR')
76140
options.prompt_for_download = False
77141
```
78-
See [docs/features.md](docs/features.md#custom-browser-preferences) for more details.
142+
143+
**Real-world power examples:**
144+
- **Silent downloads** - No prompts, no dialogs, just automated downloads
145+
- **Block ALL distractions** - Notifications, popups, camera requests, you name it
146+
- **Perfect for CI/CD** - Disable update checks, default browser prompts, crash reporting
147+
- **Multi-region testing** - Change languages, timezones, and locale settings instantly
148+
- **Security hardening** - Lock down permissions and disable unnecessary features
149+
- **Advanced fingerprinting control** - Modify browser install dates, engagement history, and behavioral patterns
150+
151+
**Fingerprint customization for stealth automation:**
152+
```python
153+
import time
154+
155+
# Simulate a browser that's been around for months
156+
fake_engagement_time = int(time.time()) - (7 * 24 * 60 * 60) # 7 days ago
157+
158+
options.browser_preferences = {
159+
'settings': {
160+
'touchpad': {
161+
'natural_scroll': True,
162+
}
163+
},
164+
'profile': {
165+
'last_engagement_time': fake_engagement_time,
166+
'exit_type': 'Normal',
167+
'exited_cleanly': True
168+
},
169+
'newtab_page_location_override': 'https://www.google.com',
170+
'session': {
171+
'restore_on_startup': 1, # Restore last session
172+
'startup_urls': ['https://www.google.com']
173+
}
174+
}
175+
```
176+
177+
This level of control was previously only available to Chrome extension developers - now it's in your automation toolkit!
178+
179+
Check the [documentation](https://autoscrape-labs.github.io/pydoll/features/custom-browser-preferences/) for more details.
79180

80181
### New `get_parent_element()` method
81182
Retrieve the parent of any WebElement, making it easier to navigate the DOM structure:
82183
```python
83184
element = await tab.find(id='button')
84-
parent = await element.get_parent_parent()
185+
parent = await element.get_parent_element()
85186
```
86187
### New start_timeout option (thanks to [@j0j1j2](https://github.com/j0j1j2))
87188
Added to ChromiumOptions to control how long the browser can take to start. Useful on slower machines or CI environments.

0 commit comments

Comments
 (0)