Add files via upload

Author: doudol

docs/api.md

@@ -0,0 +1,133 @@
+# API Reference
+
+Complete reference for the EasyScrape API.
+
+## Core Functions
+
+### `scrape()`
+
+```python
+es.scrape(url: str, **options) -> ScrapeResult
+```
+
+Fetch a URL and return a result object.
+
+**Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `url` | `str` | required | The URL to fetch |
+| `method` | `str` | `"GET"` | HTTP method |
+| `headers` | `dict` | `None` | Custom headers |
+| `timeout` | `float` | `30.0` | Request timeout in seconds |
+| `retries` | `int` | `3` | Number of retry attempts |
+| `follow_redirects` | `bool` | `True` | Follow HTTP redirects |
+
+**Returns:** `ScrapeResult` object
+
+**Example:**
+
+```python
+result = es.scrape(
+    "https://example.com",
+    timeout=10,
+    headers={"Accept-Language": "en-GB"}
+)
+```
+
+---
+
+### `async_scrape()`
+
+```python
+await es.async_scrape(url: str, **options) -> ScrapeResult
+```
+
+Async version of `scrape()`. Same parameters.
+
+**Example:**
+
+```python
+import asyncio
+import easyscrape as es
+
+async def main():
+    result = await es.async_scrape("https://example.com")
+    print(result.title())
+
+asyncio.run(main())
+```
+
+---
+
+## ScrapeResult
+
+The result object returned by `scrape()` and `async_scrape()`.
+
+### Properties
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `status_code` | `int` | HTTP status code |
+| `text` | `str` | Response body as text |
+| `content` | `bytes` | Response body as bytes |
+| `headers` | `dict` | Response headers |
+| `url` | `str` | Final URL (after redirects) |
+
+### Methods
+
+#### `css(selector: str) -> str | None`
+
+Extract the text of the first matching element.
+
+```python
+title = result.css("h1")
+```
+
+#### `css_all(selector: str) -> list[str]`
+
+Extract text from all matching elements.
+
+```python
+items = result.css_all("li.item")
+```
+
+#### `json() -> dict | list`
+
+Parse response as JSON.
+
+```python
+data = result.json()
+```
+
+#### `title() -> str | None`
+
+Get the page title.
+
+#### `main_text() -> str`
+
+Extract main content, stripped of navigation and boilerplate.
+
+#### `safe_links() -> list[str]`
+
+Get all links, filtered to remove unsafe protocols.
+
+---
+
+## Configuration
+
+### `Config`
+
+```python
+from easyscrape import Config
+
+config = Config(
+    timeout=60,
+    retries=5,
+    user_agent="MyBot/1.0"
+)
+
+result = es.scrape("https://example.com", config=config)
+```
+
+See [Configuration Guide](configuration.md) for details.

docs/async.md

@@ -0,0 +1,105 @@
+# Async Scraping
+
+Scrape multiple URLs concurrently for maximum speed.
+
+## Basic Async
+
+```python
+import asyncio
+import easyscrape as es
+
+async def main():
+    result = await es.async_scrape("https://example.com")
+    print(result.title())
+
+asyncio.run(main())
+```
+
+## Concurrent Requests
+
+Scrape multiple URLs in parallel:
+
+```python
+import asyncio
+import easyscrape as es
+
+async def scrape_all(urls: list[str]):
+    tasks = [es.async_scrape(url) for url in urls]
+    results = await asyncio.gather(*tasks)
+    return results
+
+urls = [
+    "https://example.com/page1",
+    "https://example.com/page2",
+    "https://example.com/page3",
+]
+
+results = asyncio.run(scrape_all(urls))
+
+for result in results:
+    print(f"{result.url}: {result.title()}")
+```
+
+## Rate-Limited Concurrency
+
+Control the number of simultaneous requests:
+
+```python
+import asyncio
+import easyscrape as es
+
+async def scrape_with_limit(urls: list[str], max_concurrent: int = 5):
+    semaphore = asyncio.Semaphore(max_concurrent)
+
+    async def limited_scrape(url: str):
+        async with semaphore:
+            return await es.async_scrape(url)
+
+    tasks = [limited_scrape(url) for url in urls]
+    return await asyncio.gather(*tasks)
+```
+
+## Error Handling
+
+Handle failures gracefully:
+
+```python
+import asyncio
+import easyscrape as es
+
+async def safe_scrape(url: str):
+    try:
+        return await es.async_scrape(url)
+    except es.ScrapeError as e:
+        print(f"Failed: {url} - {e}")
+        return None
+
+async def main():
+    urls = ["https://example.com", "https://invalid.example"]
+    tasks = [safe_scrape(url) for url in urls]
+    results = await asyncio.gather(*tasks)
+    
+    successful = [r for r in results if r is not None]
+    print(f"Succeeded: {len(successful)}/{len(urls)}")
+```
+
+## With Async Config
+
+```python
+from easyscrape import Config
+
+config = Config(timeout=10, retries=2)
+
+async def main():
+    result = await es.async_scrape(
+        "https://example.com",
+        config=config
+    )
+```
+
+## Best Practices
+
+1. **Limit concurrency** - Don't overwhelm servers. Use semaphores.
+2. **Handle errors** - Network requests fail. Plan for it.
+3. **Respect robots.txt** - Check before bulk scraping.
+4. **Add delays** - Use `rate_limit` in config for politeness.

docs/browser.md

@@ -0,0 +1,108 @@
+# Browser Mode
+
+Handle JavaScript-rendered pages with browser automation.
+
+## When to Use
+
+Use browser mode when:
+
+- Content is loaded via JavaScript
+- Page requires interaction (clicks, scrolls)
+- Site blocks non-browser requests
+- You need screenshots
+
+## Basic Usage
+
+```python
+import easyscrape as es
+
+# Enable browser mode
+result = es.scrape(
+    "https://example.com",
+    browser=True
+)
+
+# Works just like regular scraping
+title = result.css("h1")
+```
+
+## Wait for Content
+
+Wait for elements to appear:
+
+```python
+result = es.scrape(
+    "https://example.com",
+    browser=True,
+    wait_for="div.content-loaded"
+)
+```
+
+## JavaScript Execution
+
+Run JavaScript on the page:
+
+```python
+result = es.scrape(
+    "https://example.com",
+    browser=True,
+    js_script="window.scrollTo(0, document.body.scrollHeight)"
+)
+```
+
+## Screenshots
+
+Capture page screenshots:
+
+```python
+result = es.scrape(
+    "https://example.com",
+    browser=True,
+    screenshot="page.png"
+)
+```
+
+## Browser Options
+
+```python
+result = es.scrape(
+    "https://example.com",
+    browser=True,
+    headless=True,      # Run without visible window (default)
+    timeout=60,         # Page load timeout
+    wait_for="h1",      # CSS selector to wait for
+    js_script=None,     # JavaScript to execute
+    screenshot=None,    # Path to save screenshot
+)
+```
+
+## Async Browser
+
+```python
+import asyncio
+import easyscrape as es
+
+async def main():
+    result = await es.async_scrape(
+        "https://example.com",
+        browser=True
+    )
+    print(result.title())
+
+asyncio.run(main())
+```
+
+## Performance Tips
+
+1. **Reuse sessions** - Browser startup is slow. Batch requests.
+2. **Disable images** - Faster loads when you only need text.
+3. **Use headless** - Always use headless mode in production.
+4. **Set timeouts** - Prevent hangs on slow pages.
+
+## Limitations
+
+- Slower than HTTP requests (browser overhead)
+- Higher memory usage
+- Requires browser dependencies
+
+For most sites, regular HTTP scraping is sufficient. Use browser mode only when needed.

docs/changelog.md

@@ -0,0 +1,25 @@
+# Changelog
+
+All notable changes to EasyScrape.
+
+## [0.1.0] - 2024
+
+### Added
+
+- Initial release
+- Core `scrape()` function with automatic retries
+- CSS selector extraction with `css()` and `css_all()`
+- Async support with `async_scrape()`
+- Browser mode for JavaScript-rendered pages
+- Built-in helpers: `title()`, `main_text()`, `safe_links()`
+- Configuration system with `Config` class
+- Rate limiting support
+- Proxy support
+- Full type hints (PEP 561 compliant)
+
+### Documentation
+
+- Quick start guide
+- Complete tutorial
+- API reference
+- Cookbook with real-world recipes