feat: overhaul documentation and add deployment workflow for GitHub Pages

- Revamped the README.md to provide a clearer quick start guide and detailed usage instructions.
- Introduced a new GitHub Actions workflow for deploying documentation to GitHub Pages.
- Added comprehensive documentation covering installation, configuration, command-line options, and troubleshooting.
- Included a .gitignore file for the docs directory to prevent tracking of build artifacts and dependencies.
- Established a structured content organization for the documentation using Astro and Starlight.

Author: 3dnsfw

.github/workflows/docs-deploy.yml

@@ -0,0 +1,41 @@
+name: Deploy Docs to GitHub Pages
+
+on:
+  push:
+    branches: [master]
+    paths:
+      - 'docs/**'
+      - '.github/workflows/docs-deploy.yml'
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  build-and-deploy:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: docs
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v6
+
+      - name: Setup Bun
+        uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+
+      - name: Install dependencies
+        run: bun install --frozen-lockfile
+
+      - name: Build docs
+        run: bun run build
+
+      - name: Deploy to GitHub Pages
+        uses: peaceiris/actions-gh-pages@v4
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./docs/dist
+          publish_branch: gh-pages

.github/workflows/release.yml

@@ -10,6 +10,7 @@ on:
       - 'README.md'
       - '.github/dependabot.yml'
       - '.gitignore'
+      - 'docs/**'
 
 permissions:
   contents: write
@@ -217,3 +218,65 @@ jobs:
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
+      - name: Verify release assets exist
+        if: steps.release_info.outputs.is_prerelease == 'false'
+        id: verify_assets
+        run: |
+          TAG_NAME="${{ steps.release_info.outputs.tag_name }}"
+          echo "Verifying assets for release ${TAG_NAME}..."
+          
+          EXPECTED_ASSETS=(
+            "kemono-scraper-linux-x64"
+            "kemono-scraper-linux-arm64"
+            "kemono-scraper-windows-x64.exe"
+            "kemono-scraper-darwin-x64"
+            "kemono-scraper-darwin-arm64"
+          )
+          
+          ACTUAL_ASSETS=$(gh release view "${TAG_NAME}" --json assets --jq '.assets[].name')
+          
+          MISSING=()
+          for asset in "${EXPECTED_ASSETS[@]}"; do
+            if ! echo "${ACTUAL_ASSETS}" | grep -q "^${asset}$"; then
+              MISSING+=("${asset}")
+            fi
+          done
+          
+          if [[ ${#MISSING[@]} -gt 0 ]]; then
+            echo "::warning::Missing assets: ${MISSING[*]}"
+            echo "assets_verified=false" >> $GITHUB_OUTPUT
+          else
+            echo "All expected assets found."
+            echo "assets_verified=true" >> $GITHUB_OUTPUT
+          fi
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update docs with latest stable release tag
+        if: steps.release_info.outputs.is_prerelease == 'false' && steps.verify_assets.outputs.assets_verified == 'true'
+        run: |
+          TAG_NAME="${{ steps.release_info.outputs.tag_name }}"
+          DOCS_FILE="docs/src/content/docs/getting-started/installation.md"
+          
+          echo "Updating docs with stable release tag: ${TAG_NAME}"
+          
+          # Replace __LATEST_TAG__ placeholder with actual tag
+          sed -i "s/__LATEST_TAG__/${TAG_NAME}/g" "${DOCS_FILE}"
+          
+          # Check if any changes were made
+          if git diff --quiet "${DOCS_FILE}"; then
+            echo "No changes to docs (placeholder may already be updated or not present)."
+            exit 0
+          fi
+          
+          # Configure git
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+          
+          # Commit and push
+          git add "${DOCS_FILE}"
+          git commit -m "docs: update download links to ${TAG_NAME}"
+          git push origin HEAD:master
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+

README.md

@@ -13,202 +13,30 @@ Download all media from Kemono and Coomer. Scrapes posts from a given service an
 - **Standalone executables** — no runtime required
 - **Media compression** — convert images to JPEG XL and videos to AV1
 
-## Quick Start
-
-### Option 1: Download Pre-built Executable (Recommended)
-
-Download the latest release for your platform from [GitHub Releases](../../releases):
-
-| Platform | File |
-|----------|------|
-| Linux x64 | `kemono-scraper-linux-x64` |
-| Linux ARM64 | `kemono-scraper-linux-arm64` |
-| Windows x64 | `kemono-scraper-windows-x64.exe` |
-| macOS Intel | `kemono-scraper-darwin-x64` |
-| macOS Apple Silicon | `kemono-scraper-darwin-arm64` |
-
-```bash
-# Linux/macOS - make executable
-chmod +x kemono-scraper-*
-
-# Run
-./kemono-scraper-linux-x64 -s onlyfans -u belledelphine
-```
-
-### Option 2: Run from Source
-
-Requires [Bun](https://bun.sh) installed.
-
-```bash
-# Clone and install
-git clone https://github.com/3dnsfw/kemono-scraper.git
-cd kemono-scraper
-bun install
-
-# Run
-bun start -s patreon -u 30037948
-```
-
-## Usage
-
-### Command-Line Arguments
-
-| Argument | Alias | Description | Required |
-|----------|-------|-------------|----------|
-| `--config` | `-c` | Path to YAML config file | No* |
-| `--service` | `-s` | Service to scrape from | No* |
-| `--userId` | `-u` | User ID to scrape | No* |
-| `--host` | `-h` | Base host (default: `kemono.cr`) | No |
-| `--outputDir` | `-o` | Output directory (default: `downloads-%username%`) | No |
-| `--maxPosts` | `-m` | Max posts to fetch (default: 5000, 0 = unlimited) | No |
-
-\* Either `--config` OR both `--service` and `--userId` must be provided.
-
-**Supported services:** `patreon`, `fanbox`, `discord`, `fantia`, `afdian`, `boosty`, `gumroad`, `subscribestar`, `dlsite`, `onlyfans`, `fansly`, `candfans`
-
-**Supported hosts:** `kemono.cr`, `coomer.st`, `kemono.su`, `coomer.su`
-
-### Single Creator Mode
-
-```bash
-# Scrape a Patreon creator from Kemono
-bun start -s patreon -u 30037948
-
-# Scrape from Coomer
-bun start -s onlyfans -u belledelphine --host coomer.st
-
-# Custom output directory
-bun start -s fansly -u someuser -o ./my-downloads
-
-# Using the standalone executable
-./kemono-scraper-linux-x64 -s onlyfans -u belledelphine --host coomer.st
-```
-
-### Config File Mode (Multiple Creators)
-
-Create a YAML config file to scrape multiple creators in one run:
-
-```bash
-# Copy the example config
-cp config.example.yaml config.yaml
-
-# Edit with your creators
-nano config.yaml
+## Documentation
 
-# Run with config file
-bun start --config config.yaml
+📖 **[View the full documentation](https://3dnsfw.github.io/kemono-scraper/)**
 
-# Or with standalone executable
-./kemono-scraper-linux-x64 --config config.yaml
-```
-
-**Example `config.yaml`:**
-
-```yaml
-# Global defaults
-host: kemono.cr
-outputDir: downloads-%username%
-maxPosts: 5000
-proxyRotation: round_robin
-proxies:
-  # - type: http        # Options: http, https, socks5
-  #   host: proxy.example.com
-  #   port: 8080
-  #   username: user    # Optional
-  #   password: pass    # Optional
-  # - type: socks5
-  #   host: socks.example.com
-  #   port: 1080
-
-# Creators to scrape
-creators:
-  - service: patreon
-    userId: "30037948"
-
-  - service: fanbox
-    userId: "3316400"
-
-  - service: onlyfans
-    userId: "belledelphine"
-    host: coomer.st  # Override for this creator
-
-  - service: fantia
-    userId: "83679"
-    outputDir: fantia/%username%  # Custom output dir
-    maxPosts: 100  # Limit posts for this creator
-```
-
-See `config.example.yaml` for a full example with all options.
-
-### Proxy configuration
-
-- Add proxies under `proxies` in your config. Supported `type` values: `http`, `https`, `socks5`.
-- Rotation is round-robin; unhealthy proxies are cooled down for a short period on connection/auth errors.
-- Leave `proxies` empty to disable proxying; the scraper falls back to direct connections automatically.
-- Enable verbose proxy logs with `DEBUG_PROXY=1 bun start --config config.yaml`.
-
-## Building Executables
-
-Build standalone executables that run without Bun installed:
-
-```bash
-# Build all platforms
-bun run build
-
-# Build specific platform
-bun run build:linux      # Linux x64
-bun run build:linux-arm  # Linux ARM64
-bun run build:windows    # Windows x64
-bun run build:macos      # macOS x64
-bun run build:macos-arm  # macOS ARM64
-```
-
-Executables are output to the `dist/` directory.
-
-## Compression
-
-After downloading, compress media files to save disk space:
-
-- **Images**: JPG/JPEG → JPEG XL (typically 30-50% smaller)
-- **Videos**: MP4 → AV1 (typically 30-50% smaller)
-
-The scraper automatically detects compressed files on subsequent runs.
-
-### Requirements
-
-- **libjxl** for JPEG XL (`cjxl` command)
-  - Linux: `paru -S libjxl` or `apt install libjxl-tools`
-  - Windows: `winget install --id=libjxl.libjxl -e`
-- **ffmpeg** with SVT-AV1 support
-  - Linux: `paru -S ffmpeg` or `apt install ffmpeg`
-  - Windows: `winget install ffmpeg`
+The documentation covers:
+- Installation (pre-built executables or from source)
+- Quick start guide
+- Configuration file for multiple creators
+- All command-line options
+- Proxy setup
+- FAQ & troubleshooting
 
-### Running Compression
+## Quick Start
 
 ```bash
-bun run compress
-```
-
-### Configuration
-
-Set environment variables to customize compression:
+# Download the executable for your platform from GitHub Releases
+# Then run:
+./kemono-scraper -s patreon -u 30037948
 
-```bash
-JPEG_XL_QUALITY=95 AV1_CRF=28 bun run compress
+# Or for OnlyFans/Fansly (via Coomer):
+./kemono-scraper -s onlyfans -u username --host coomer.st
 ```
 
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `JPEG_XL_QUALITY` | 90 | JPEG XL quality (1-100, higher = better) |
-| `JPEG_XL_EFFORT` | 7 | Encoding effort (1-9, higher = slower but smaller) |
-| `AV1_CRF` | 30 | AV1 quality (lower = better, 18-35 typical) |
-| `AV1_PRESET` | 6 | SVT-AV1 preset (0-13, lower = slower but better) |
-
-On Windows PowerShell:
-
-```powershell
-.\compress.ps1 -JpegXlQuality 95 -Av1Crf 28
-```
+See the [documentation](https://3dnsfw.github.io/kemono-scraper/) for detailed instructions.
 
 ## License
 

docs/.gitignore

@@ -0,0 +1,20 @@
+# Dependencies
+node_modules/
+
+# Build output
+dist/
+
+# Astro
+.astro/
+
+# Environment
+.env
+.env.*
+
+# Editor
+.vscode/
+.idea/
+
+# OS
+.DS_Store
+Thumbs.db