Skip to content

Latest commit

Β 

History

History
715 lines (546 loc) Β· 29.5 KB

File metadata and controls

715 lines (546 loc) Β· 29.5 KB

Changelog

All notable changes to FetchtiumV2.


[2.0.1] - 2026-02-24 β€” Docs Sync for Current Snapshot

πŸ“ Documentation Updates

  • Updated README and wiki pages to match current routes in src/app/api/**/route.ts.
  • Documented YouTube watch-url fast-path behavior in GET /api/v1/download and optional watch-url mode in GET /api/v1/merge.
  • Clarified that split-stream merge (videoUrl + audioUrl or videoH + audioH) remains supported and unchanged.

⚠️ Accuracy and Scope Notes

  • Removed/marked unavailable endpoint docs for routes not present in this snapshot (for example, /api/v1/events and /api/extract).
  • Added caveats that local Python wrapper mode and Docker/Railway full-profile claims depend on the presence of the Python api/ module, which is not included in this snapshot.
  • Kept deployment and platform coverage statements conditional on active profile and Python backend availability.

πŸ”’ Tooling Notes

  • Documented dependency security remediation context (including minimatch override for eslint-config-next).
  • Updated lint/tooling notes to reflect current stack alignment (eslint@9 + eslint-config-next@16) without claiming warning-free lint output.

[2.0.0] - 2026-02-23 β€” YouTube Multi-Codec + UI Polish

πŸš€ Highlights

  • YouTube Multi-Codec Support - Returns ALL codecs per resolution (H.264, VP9, AV1) for user choice
  • Progressive Format Priority - 360p H.264 with audio now shown FIRST with "READY" status (no merge needed)
  • 18 Video Sources - Returns 18 YouTube formats (vs 8 before) with proper codec grouping
  • DASH Warning Popups - Shows warning when downloading DASH formats requiring merge
  • New UI Banner - Added "Try our new version of social media downloader: DownAria" link below header

🎨 UI/UX Improvements

  • Logo Integration - Added icon.png logo in header using Next.js Image component
  • Auto-Extract on Paste - Paste button now automatically extracts when valid URL is in clipboard
  • Button Labels - Added text labels "paste" and "settings" to action buttons
  • Font Update - All text now uses JetBrains Mono font
  • Removed Backlinks - Cleaned up header area, removed backlinks section
  • Offline Message - Footer shows "When DownAria is offline, FetchtiumV2 is ready to wait" message
  • Section Rename - Changed "Supported Platforms" to "More media supported"

πŸ”§ Technical Changes

  • Format Processing - process_youtube_formats() now separates progressive vs DASH, multi-codec per resolution
  • HLS Detection - Added /manifest/, index.m3u8, hls protocol checks
  • YouTube URL Handling - Added noplaylist=True and URL canonicalization for playlist/radio URLs
  • Codec Grouping - FormatList.tsx groups formats by codec: Progressive, H.264, VP9, AV1, Audio
  • Thumbnail Authorization - Fixed stream endpoint for YouTube thumbnails (i.ytimg.com, yt3.ggpht.com)
  • Download Speed - Added KB/s, MB/s indicator in loading state
  • Merge Optimization - Merge endpoint now uses audio-copy flag and reconnect flags

πŸ› Fixes

  • Python Bytecode Cache - Documented cache clearing with taskkill /F /IM python.exe to avoid stale results
  • CookieModal Animation - Fixed close behavior to animation-driven close lifecycle
  • DownAria Link Update - Changed from downaria.site to downaria.vercel.app

πŸ“¦ Files Changed

~ package.json                    # Version 2.0.0
~ src/app/layout.tsx             # JetBrains Mono font
~ src/app/page.tsx               # Logo, banner, button labels, section rename
~ src/components/ExtractForm.tsx # Auto-extract on paste, text labels
~ src/components/FormatList.tsx  # Codec grouping, warning popups
~ api/services/formats.py        # Multi-codec, progressive priority
~ api/routes/extract.py           # URL canonicalization

[1.8.0] - 2026-02-23 β€” FetchtiumV2 Modular Restructure

πŸš€ Highlights

  • Introduced modular Python API architecture with app factory and blueprints.
  • Added extractor service layer modules (formats, resolver, ytdlp, gallery_dl, transforms).
  • Added TypeScript extractor architecture split (native, wrappers, shared types, Python client).
  • Added new Next.js route POST /api/extract while preserving /api/v1/extract compatibility.
  • Standardized Python runtime port to 5000 across scripts, env, and orchestration.
  • Fixed CookieModal close behavior to animation-driven close lifecycle.

πŸ§ͺ Testing

  • Added Python tests for config, security, errors, formats, routes, wrappers, resolver, and smoke checks.
  • Added TypeScript unit tests with Vitest for Python client, wrappers, router detection, and health route.

πŸ“¦ Ops and Docs

  • Added docker-compose.yml for local dual-service setup.
  • Simplified start.sh dual-process orchestration (Python background + Next foreground).
  • Updated docs and added architecture/migration documentation.

[1.7.0] - 2026-02-23 β€” Dynamic Deployment Profile

πŸš€ What's New

  • Dynamic Deployment Profile - Single codebase works on both Vercel and Docker/Linux
    • vercel profile: Native extractors only (Facebook, Instagram, TikTok, Twitter, Pixiv)
    • full profile: Native + Python extractors (YouTube, BiliBili, SoundCloud, etc.)
  • Auto Environment Detection - Automatically detects Vercel vs Docker/Railway
    • Priority: EXTRACTOR_PROFILE env var β†’ Vercel auto-detect β†’ Default to full
  • Platform Availability Error - New PLATFORM_UNAVAILABLE_ON_DEPLOYMENT error code
    • Returns helpful message when Python platforms requested on Vercel deployment
  • Public Access Model - Removed API key requirement, open access for all users

πŸ”§ Technical Changes

  • EXTRACTOR_PROFILE env var - Controls extractor capability at runtime
  • getExtractorProfile() - Resolves profile from environment with fallback chain
  • isPythonEnabled() - Helper to check Python extractor availability
  • Direct HTTP calls - Removed Next.js rewrites, Python service called directly
  • PYTHON_API_URL support - Configurable Python endpoint for non-Vercel deployments

πŸ“¦ Files Changed

+ .gitattributes                    # LF enforcement for shell scripts
~ vercel.json                       # Added framework, regions, EXTRACTOR_PROFILE
~ next.config.ts                    # Removed rewrites, updated CORS headers
~ src/lib/config/index.ts           # Added getExtractorProfile(), isPythonEnabled()
~ src/types/config.ts               # Added extractorProfile, removed apiKeys
~ src/app/api/v1/extract/route.ts   # Profile-aware routing, platform blocking
~ src/lib/utils/error.utils.ts      # Added PLATFORM_UNAVAILABLE_ON_DEPLOYMENT
~ Dockerfile                        # Added EXTRACTOR_PROFILE=full, use requirements.txt
~ start.sh                          # Added EXTRACTOR_PROFILE env var
~ src/app/page.tsx                  # Logo from public/icon.png, Lucide icons
~ src/app/changelog/page.tsx        # Logo from public/icon.png, Lucide icons
- StatusBadge component             # Removed cold/warm state from UI

[1.6.0] - 2026-01-13 β€” Go Backend Extractors Reorganization

πŸ”§ Refactoring (Go Backend)

  • Extractors Reorganization - Restructured internal/extractors/ into logical subfolders:

    • core/ - Shared types and interfaces (types.go)
    • registry/ - Platform detection and extractor registry (registry.go, patterns.go)
    • cookies/ - Cookie parsing and management (cookies.go)
    • native/ - TypeScript-equivalent extractors (Facebook, Instagram, TikTok, Twitter, Pixiv)
    • aria-extended/ - yt-dlp/gallery-dl wrapper extractors (YouTube, BiliBili, SoundCloud, etc.)
    • tests/ - Property-based and integration tests
  • HTTP Package Simplification - Merged and cleaned up pkg/http/:

    • client.go - HTTP client with connection pooling
    • pool.go - Connection pool management
    • helpers.go - Request/response utilities
  • Type Fixes - Fixed ExtractorFactory type mismatch between registry and extractors

πŸ“¦ Files Changed (Go)

.planning/fetchtium_go/internal/extractors/
β”œβ”€β”€ core/types.go           # Shared types (ExtractResult, Source, etc.)
β”œβ”€β”€ registry/
β”‚   β”œβ”€β”€ registry.go         # Extractor registration & lookup
β”‚   └── patterns.go         # URL pattern matching
β”œβ”€β”€ cookies/cookies.go      # Cookie parsing (Netscape, JSON, raw)
β”œβ”€β”€ native/                 # Native extractors
β”‚   β”œβ”€β”€ facebook/
β”‚   β”œβ”€β”€ instagram/
β”‚   β”œβ”€β”€ tiktok/
β”‚   β”œβ”€β”€ twitter/
β”‚   └── pixiv/
β”œβ”€β”€ aria-extended/          # yt-dlp/gallery-dl wrappers
β”‚   β”œβ”€β”€ wrapper.go
β”‚   β”œβ”€β”€ youtube/
β”‚   β”œβ”€β”€ bilibili/
β”‚   └── ...
└── tests/                  # Test files

.planning/fetchtium_go/pkg/http/
β”œβ”€β”€ client.go               # Simplified HTTP client
β”œβ”€β”€ pool.go                 # Connection pool
└── helpers.go              # Utilities

[1.5.1] - 2026-01-12 β€” Pixiv Native Support + Hotfixes

πŸš€ What's New

  • Pixiv Native Extractor - Direct API integration without gallery-dl dependency

    • Single and multi-page artwork support
    • Original quality images with file sizes
    • Automatic Referer header handling via proxy
  • All Resolutions Support - No more 360p/480p/720p/1080p limit

    • Returns ALL available video qualities (144p to 4K+)
    • Deduplication by exact height (one format per resolution)
  • Changelog Page - New /changelog page that reads directly from CHANGELOG.md

    • Auto-updates when changelog is modified
    • Styled markdown rendering
  • Format Badge - Shows original format with codec in FormatList

    • Combined format: [HLS-H.264], [MP4-VP9], [DASH-AV1], [Opus]
    • Mime type shown below for output format

πŸ”§ Technical Changes

  • src/lib/extractors/pixiv/ - New native TypeScript extractor using Pixiv's /ajax/illust/{id} API
  • Removed HTTP/2 from TypeScript - Simplified to undici only (Python keeps httpx HTTP/2)
  • Proxy support for Pixiv - Stream/download/thumbnail routes add Referer: https://www.pixiv.net/
  • needsProxy flag - Pixiv sources marked for automatic proxy routing

πŸ› Fixes

  • PlayerModal audio leak - Fixed audio continuing after modal close (proper cleanup with load())
  • Merge loading message - Changed to "Merging video and audio for playback... please wait"
  • FormatList Open button - Now uses proxy for needsProxy sources (Pixiv images)
  • ResultCard thumbnails - Added Pixiv to proxy thumbnail platforms

πŸ“¦ Files Changed

+ src/lib/extractors/pixiv/index.ts    # New Pixiv native extractor
+ src/app/changelog/page.tsx           # Changelog page
+ src/app/api/changelog/route.ts       # Changelog API endpoint
- src/lib/core/network/http2-client.ts # Removed (Python has own httpx)
~ src/lib/core/network/client.ts       # Simplified to undici only
~ src/lib/extractors/index.ts          # Register PixivExtractor
~ src/lib/extractors/python-platforms.ts # Remove Pixiv from Python
~ src/app/api/v1/stream/route.ts       # Add Pixiv Referer
~ src/app/api/v1/download/route.ts     # Add Pixiv Referer
~ src/app/api/v1/thumbnail/route.ts    # Add pximg.net + Referer
~ src/components/PlayerModal.tsx       # Fix audio cleanup
~ src/components/FormatList.tsx        # Format badge + proxy for needsProxy
~ src/components/ResultCard.tsx        # Pixiv thumbnail proxy
~ src/app/page.tsx                     # Add changelog link in header
~ src/middleware.ts                    # Add /api/changelog to public routes
~ api/extract.py                       # Remove Pixiv, all resolutions support

[1.5.0] - 2026-01-11 β€” HTTP/2 Support + Rule34Video + Stream Improvements

πŸš€ What's New

  • HTTP/2 Support (TypeScript) - Auto-detect and use HTTP/2 for HTTPS requests when available (Vercel Functions, Railway)
  • HTTP/2 Support (Python) - Shared httpx client with HTTP/2 and 5-minute connection pooling
  • Rule34Video Support - Multi-format extraction (360p-4K) via yt-dlp with URL resolution
  • Eporner Multi-Format - Returns all available formats (240p-1080p 60fps) with file sizes
  • URL Resolution - Wrapper URLs (Rule34Video, Eporner) resolved to real CDN URLs in parallel
  • Stream Expired URL Detection - Returns "URL_EXPIRED" error instead of following rickroll redirects

πŸ”§ Technical Changes

  • http2-client.ts - New HTTP/2 client with session pooling (5-min keepalive), auto-cleanup
  • client.ts - Smart detection: HTTP/2 for HTTPS when available, fallback to undici HTTP/1.1
  • api/extract.py - Shared HTTP_CLIENT with httpx HTTP/2, 5-min keepalive, 10 max connections
  • resolve_media_url() - Resolves wrapper URLs to CDN, removes download=true params
  • resolve_media_urls_parallel() - Parallel URL resolution with ThreadPoolExecutor (5 workers)
  • transform_eporner_result() - Checks quality field first (Rule34Video uses this), adds "p" suffix
  • stream route - Detects rickroll/placeholder redirects and returns 410 Gone with URL_EXPIRED code

πŸ› Fixes

  • Rule34Video Quality - Fixed quality showing "0", "1", "2" instead of "360p", "720p", "1080p"
  • Rule34Video Playback - Fixed "Open in new tab" redirecting to rickroll (URL resolution)
  • Rule34Video Download - Fixed browser downloading instead of playing (removed download=true param)
  • Eporner Quality - Fixed quality string formatting for all formats

πŸ“¦ New Files

src/lib/core/network/http2-client.ts  # HTTP/2 client with session pooling

πŸ“‹ Dependencies

# Python (requirements.txt)
httpx[http2]  # HTTP/2 support for Python extractor

πŸ”§ Environment Detection

// Auto-detected at startup (zero runtime overhead)
- Vercel Functions β†’ HTTP/2 βœ“
- Railway Node.js β†’ HTTP/2 βœ“
- Vercel Edge β†’ undici HTTP/1.1 (no http2 module)
- Local dev β†’ HTTP/2 βœ“

[1.4.0] - 2026-01-11 β€” Twitter, Instagram Stories, Pinterest Video, YouTube HLS

πŸš€ What's New

  • Twitter/X Cookie Support - Fixed Netscape cookie format parsing with netscapeToCookieHeader() converter
  • Twitter TweetWithVisibilityResults - Handle GraphQL wrapper for tweets with visibility restrictions
  • Instagram Stories - Full support via Internal API, fetches all story items as carousel
  • Instagram Reels URL - Support /reels/ (plural) in addition to /reel/
  • Pinterest Video - Extract MP4 sources from video pins (V_720P, V_HLSV4 fallback)
  • YouTube HLS Proxy - True streaming playback via HLS.js instead of full download
  • YouTube HLS Download - FFmpeg routes through proxy for proper segment fetching

πŸ”§ Technical Changes

  • New Endpoint - /api/v1/hls-proxy proxies HLS manifest and segments on-demand
  • Thumbnail Endpoint - Changed from hash-based (?h=) to direct URL (?url=) for easier API usage
  • CookieModal - Added netscapeToCookieHeader() to convert Netscape format to HTTP Cookie header
  • Twitter extract.ts - Unwrap TweetWithVisibilityResults to get actual tweet from result.tweet
  • Instagram scanner.ts - Added fetchStory(), fetchStoryByUsername() functions
  • Instagram extract.ts - Added parseStoryResponse(), handle reels[userId] response structure
  • Instagram patterns - Added XDTGraphSidecar, XDTGraphImage, XDTGraphVideo typename checks
  • api/extract.py - transform_pinterest_result() handles video pins with contentType: 'video'
  • PlayerModal - Split needsHlsStream into needsHlsTranscode (Opus) and needsHlsProxy (YouTube)
  • hls-stream route - Detects YouTube HLS and routes through proxy for FFmpeg download
  • middleware.ts - Added /api/v1/hls-proxy to public routes and streaming endpoint checks

πŸ› Fixes

  • Twitter Cookies - Fixed cookie parsing not working with Netscape export format
  • Twitter Visibility - Fixed extraction failing on tweets wrapped in TweetWithVisibilityResults
  • Instagram Stories - Fixed "reels_media[0] undefined" error - API returns reels[userId] not reels_media
  • Instagram Carousel - Fixed typename check missing XDT-prefixed types from Internal API
  • YouTube HLS Play - Fixed video downloading entire stream before playing (now true streaming)
  • YouTube HLS Download - Fixed 403 errors by adding Sec-Fetch-* headers to proxy requests

πŸ“¦ New Files

src/app/api/v1/hls-proxy/route.ts  # HLS proxy for CORS-restricted streams

[1.3.0] - 2026-01-11 β€” Extract.py Cleanup + API Docs + UI Polish

πŸš€ What's New

  • YouTube Loading Indicator - Shows "Processing stream... please wait" during FFmpeg transcoding
  • BiliBili Loading Indicator - Shows "Merging video and audio... please wait" during merge
  • Thumbnail Proxy - BiliBili and YouTube thumbnails now display correctly via stream proxy

🎨 UI Improvements

  • Header - Added "API Docs" link button
  • Footer Redesign - Clean 2-column grid layout with Links (left) and Platforms (right)
  • Platform Badges - Color-coded: Native (green), Python (blue), NSFW (red)
  • Deploy Links - Railway marked "Full Support", Vercel marked "Limited"

πŸ”§ Extract.py Cleanup (Major Refactor)

  • Centralized Configuration - Single DEFAULT_USER_AGENT and DEFAULT_HEADERS constants
  • Platform Registry - New PLATFORM_CONFIG dict with extractor type, patterns, NSFW flag, custom headers
  • Derived Lists - YTDLP_PLATFORMS, GALLERY_DL_PLATFORMS, NSFW_PLATFORMS auto-generated from config
  • Short URL Resolution - Unified resolve_short_url() for pin.it, b23.tv, redd.it
  • Format Processing - New process_video_formats() and process_audio_formats() functions
  • Header Merge - merge_headers() helper for platform-specific header overrides
  • File Organization - Clear sections: Imports, Constants, Config, Security, Detection, Extractors, Processing, Transformers, Routes

πŸ› Fixes

  • Thumbnail Display - Fixed BiliBili/YouTube thumbnails not showing (removed hash-based approach, use stream proxy)
  • YouTube Playback - Fixed immediate loading without indicator
  • Middleware Cleanup - Removed all console.log debug statements

πŸ”§ Technical Changes

  • ResultCard - Uses /api/v1/stream?url=... for thumbnail proxy instead of hash
  • PlayerModal - Added needsHlsStream to useEffect dependencies
  • extract route - Simplified URL storage, removed thumbnailHash assignment

[1.2.4] - 2026-01-11 β€” BiliBili Support + JSON Cleanup

πŸš€ What's New

  • BiliBili Video+Audio Merge - Downloads now include audio! FFmpeg merges video and audio streams from BiliBili DASH segments
  • BiliBili Playback - Video playback with audio via server-side FFmpeg merge
  • Cleaner JSON Response - Removed hash from source objects to reduce payload size

πŸ› Fixes

  • BiliBili Download - Fixed video downloads being video-only (no audio)
  • BiliBili Playback - Fixed playback having no audio by merging video+audio in FFmpeg
  • BiliBili Thumbnail - Added bstarstatic.com to stream endpoint Referer check

πŸ”§ Technical Changes

  • hls-stream endpoint - Added audioUrl parameter for BiliBili video+audio merge (2 FFmpeg inputs)
  • FormatList - Passes audio URL for BiliBili video downloads
  • PlayerModal - Includes audioUrl in hls-stream request for BiliBili
  • extract route - Removed hash from source objects (URLs still stored for validation)

[1.2.3] - 2026-01-11 β€” YouTube HLS Video Playback Fix

πŸ› Fixes

  • YouTube Video Playback - Fixed video playing without audio. FFmpeg now includes audio track (-c:a aac) instead of stripping it (-an)
  • Separate Audio Logic - Fixed FormatList incorrectly passing separate audio for YouTube HLS streams. Now only uses separate audio when hasAudio === false explicitly
  • Video Sizing - Video player now fits modal properly with max-h-[80vh] object-contain

πŸ”§ Technical Changes

  • hls-stream endpoint - Video output now remuxes both video and audio from HLS source
  • FormatList - Simplified audio detection logic, removed URL-based guessing
  • PlayerModal - Cleaned up console.log statements

[1.2.2] - 2026-01-10 β€” HLS Opus Streaming + YouTube

πŸš€ What's New

  • YouTube Support - Video extraction via yt-dlp with quality selection, codec info, fps
  • HLS Opus Playback - SoundCloud Opus HLS streams now play in browser via server-side FFmpeg transcoding
  • Experimental Badge - HLS and Opus formats marked with ⚑ Experimental badge
  • Audio Preview - Thumbnail preview with "🎡 Audio Only" badge for audio playback
  • New Endpoint - /api/v1/hls-stream converts HLS to progressive MP3 stream

πŸ”§ Technical Changes

  • FFmpeg Integration - Uses ffmpeg-static package for server-side transcoding
  • Path Resolution Fix - Fixed FFmpeg binary path in Next.js bundled environment
  • Middleware Update - Added /api/v1/hls-stream to public routes
  • YouTube Metadata - Includes codec (H.264/VP9/AV1), fps, hasAudio flags
  • YouTube Simplified Output - 1 format per resolution (max 4: 1080p, 720p, 480p, 360p), priority: hasAudio > H.264 > VP9 > AV1
  • YouTube Audio Simplified - Only 2 audio formats: AAC ~128kbps + Opus ~128kbps

πŸ“‹ New Dependencies

{
  "ffmpeg-static": "^5.3.0"
}

[1.2.1] - 2026-01-10 β€” Python Extractors

πŸš€ What's New

  • Python Extractors - Added support for 6 new platforms via yt-dlp and gallery-dl
  • SoundCloud - Audio extraction with multiple formats (MP3, AAC, Opus)
  • BiliBili - Video extraction with multiple qualities (144p-720p)
  • Reddit - Video/image/gallery extraction
  • Pixiv - Artwork extraction (NSFW support with cookies)
  • Eporner - Adult video extraction (NSFW)
  • Rule34Video - Adult video extraction (NSFW)

πŸ”§ Technical Changes

  • Vercel Python Functions - Flask-based serverless functions alongside Next.js
  • Hybrid Architecture - TypeScript extractors (Twitter, Instagram, TikTok, Facebook) + Python extractors (new platforms)
  • Unified API - All platforms accessible via same /api/v1/extract endpoint
  • NSFW Flag - Response includes isNsfw: true for adult platforms
  • Concurrent Dev - npm run dev runs both Next.js and Python Flask server

πŸ“¦ New Files

api/py/
└── extract.py         # Flask-based Python extractor

src/lib/extractors/
└── python-platforms.ts  # Platform detection for Python routing

πŸ“‹ New Dependencies

# Python (requirements.txt)
yt-dlp
gallery-dl
flask

# Node (devDependencies)
concurrently

πŸ”§ Config Changes

  • vercel.json - Python 3.12 runtime configuration
  • next.config.ts - Dev rewrites for Python server proxy
  • package.json - New dev scripts with concurrently

[1.2.0] - 2026-01-10 β€” Next.js Migration + Streaming Fixes

πŸš€ What's New

  • Next.js 16 with App Router and Turbopack
  • TypeScript full codebase conversion with strict types
  • React frontend with custom hooks (useExtract, useStatus)
  • Server Components for optimal performance
  • 5-minute warm status - extended keep-alive timeout from 30s to 5min
  • HLS.js Integration - Better audio/video playback support

πŸ”§ Streaming & Download Fixes

  • Stream Mode - Added streamMode option to disable body timeout for large files
  • Progressive Streaming - Audio/video now streams progressively instead of buffering entire file
  • HLS Support - Added HLS.js library for m3u8 playlist playback
  • Better Error Handling - PlayerModal shows specific error codes (MEDIA_ERR_DECODE, etc.)
  • MIME Mappings - Comprehensive audio MIME type mappings (opus, ogg, flac, wav, aac)

🎡 Python Extractor Improvements

  • Extension Field - Source objects now include extension field
  • Filename Generation - Auto-generated filenames: Author_Title_Quality.ext
  • Audio Metadata - Bitrate and filesize fields for audio sources
  • Sanitized Output - Proper filename sanitization for special characters

πŸ”„ What's Different

  • Framework: Fastify β†’ Next.js 16 App Router
  • Language: JavaScript β†’ TypeScript
  • Frontend: Vanilla JS β†’ React components
  • Logging: Pino β†’ Console-based logger (simplified)
  • SSE: Custom implementation β†’ ReadableStream API
  • Middleware: Fastify hooks β†’ Next.js middleware.ts
  • Cookie handling: Frontend no longer auto-sends cookies, backend decides

πŸ› Fixes

  • File size detection - Fixed undici maxRedirections not supported error
  • Image extraction - Removed overly aggressive skip patterns (.webp, _s\d+x\d+) that blocked valid Facebook images
  • Content type detection - Fixed permalink.php URLs not detected as posts
  • Filename format - Updated to Author_Description_Quality_[DownAria].ext
  • Playback timeout - Fixed audio/video timeout on large files (bodyTimeout: 0 for streams)

πŸ“¦ New Components

src/components/
β”œβ”€β”€ ExtractForm.tsx    # URL input with paste button
β”œβ”€β”€ ResultCard.tsx     # Metadata display
β”œβ”€β”€ FormatList.tsx     # Download options
β”œβ”€β”€ PlayerModal.tsx    # Video/audio player with HLS.js
β”œβ”€β”€ CookieModal.tsx    # Cookie management
β”œβ”€β”€ StatusBadge.tsx    # SSE warm/cold indicator
└── JsonOutput.tsx     # Collapsible JSON with copy

πŸ“‹ New Dependencies

{
  "hls.js": "^1.6.15"  // HLS streaming support
}

πŸ”§ API Changes

No breaking changes - all endpoints remain the same:

  • POST /api/v1/extract
  • GET /api/v1/stream
  • GET /api/v1/download
  • GET /api/v1/status
  • GET /api/v1/events
  • GET /api/health

[1.0.0] - 2026-01-8 β€” Pre-Migration Release (Fastify)

πŸŽ‰ Initial Complete Release

This is the complete Fastify-based version before migrating to Next.js.


Core Architecture

  • Fastify web framework with streaming support
  • Undici HTTP client with connection pooling
  • Pino structured logging system
  • Environment-based configuration (.env)
  • ES Modules (type: module)

Facebook Extractor

  • Full support: Videos, Reels, Stories, Posts, Galleries, Groups
  • Cookie authentication (Netscape + JSON + raw string formats)
  • Smart retry strategy: guest first β†’ cookies on auth error
  • Stories always use cookies (required)
  • Engagement stats extraction from JSON + title fallback
  • Content issue detection (age-restricted, private, deleted, expired)
  • URL resolution for short links (fb.watch, fb.me)

API Endpoints (v1)

  • POST /v1/extract - Main extraction with cookie support
  • GET /v1/stream - Video/audio streaming proxy (range requests)
  • GET /v1/download - File download proxy
  • GET /v1/status - Server status
  • GET /v1/events - SSE real-time status updates
  • GET /health - Health check

Security

  • Path traversal protection - canonical path validation, encoding bypass prevention
  • SSRF protection - internal IPs, metadata endpoints, numeric/octal/hex IP blocking
  • XSS sanitization - using xss library
  • SQL injection patterns - defense in depth
  • CRLF/header injection - blocked
  • Request size limit - 1MB max
  • Rate limiting - 100 req/min per IP
  • Security headers - via @fastify/helmet
  • Frontend sanitization - backticks, template literals blocked

Cookie System

  • Priority: Client cookie β†’ Server cookie (future) β†’ File cookie
  • Auto-detect format: JSON array, JSON object, Netscape, raw string
  • parseCookies() universal parser
  • getFacebookCookies({ clientCookie }) with priority handling

Response Builder

  • Standardized API response format
  • meta object: responseTime, accessMode, publicContent
  • stats normalization for all platforms
  • buildResponse(), buildErrorResponse() helpers

Filename Generation

  • Universal addFilenames() at route level
  • Format: author_contentType_title(25)_quality.ext
  • Unicode support (Chinese, Japanese, etc.)
  • Uses getExtensionFromMime() helper

Self-Registering Extractors

  • Each extractor defines static patterns and static match()
  • Registry auto-discovers extractors
  • url.utils.js is platform-agnostic

Frontend (Vanilla JS)

  • Dark theme with Tailwind CSS (CDN)
  • Extract form with paste button
  • Result card with thumbnail, metadata, stats
  • Format list with Play, Open, Download buttons
  • Player modal (video/audio) using proxy stream
  • Cookie modal with platform tabs (Facebook, Instagram, TikTok)
  • Copy JSON button
  • SSE status indicator (Warm/Cold)
  • Download All with progress
  • Input sanitization (XSS, command injection)

Output Format

{
  "success": true,
  "platform": "facebook",
  "contentType": "reel",
  "title": "...",
  "author": "...",
  "stats": { "views": 9700, "likes": 340 },
  "items": [{
    "index": 0,
    "type": "video",
    "thumbnail": "...",
    "sources": [{
      "quality": "hd",
      "url": "...",
      "resolution": "1280x720",
      "mime": "video/mp4",
      "size": 4493909,
      "filename": "author_reel_title_hd.mp4"
    }]
  }],
  "meta": {
    "responseTime": 2049,
    "accessMode": "public",
    "publicContent": true
  }
}

Dependencies

fastify, @fastify/cors, @fastify/helmet, @fastify/static
undici, node-html-parser, xss, validator
pino, pino-pretty, dotenv

Placeholders (Not Implemented)

  • YouTube extractor
  • Instagram extractor
  • TikTok extractor
  • Twitter/X extractor

Pre-1.0.0 Development Notes

2026-01-10 β€” Session Summary

  • Removed legacy /api/* routes (replaced by /v1/*)
  • Created /v1/stream with range request support
  • Added player modal for video/audio playback
  • Implemented SSE for status updates (replaced polling)
  • Self-registering extractors pattern
  • Comprehensive security hardening
  • Cookie system with client upload support
  • Response builder with meta/stats
  • Filename generation utility
  • Stats extraction with title fallback
  • Copy JSON button fix (CSP compliance)