Development

.gitignore

@@ -19,4 +19,5 @@ tags
 env/
 .idea/
 venv/
-test.py
+test.py
+uv.lock

AGENTS.md

@@ -0,0 +1,47 @@
+# Repository Guidelines
+
+## Project structure & module organization
+This repository is a Python CLI application packaged as `yewtube`.
+Core code lives in `mps_youtube/`, with command handlers in
+`mps_youtube/commands/`, player backends in `mps_youtube/players/`, and
+list rendering helpers in `mps_youtube/listview/`. Project configuration is in
+`pyproject.toml`, dependency locking is in `uv.lock`, and packaging metadata is
+in `yewtube.egg-info/`.
+
+## Build, test, and development commands
+Use `uv` for local workflows.
+
+- `uv sync` installs runtime and dev dependencies from `pyproject.toml` and
+  `uv.lock`.
+- `uv run yt` runs the CLI entry point (`yt = mps_youtube.main:main`).
+- `uv run ruff check .` runs lint checks.
+- `uv run ruff check . --fix` auto-fixes safe lint issues.
+- `uv run python -m mps_youtube.main` runs the app directly during debugging.
+
+## Coding style & naming conventions
+Follow the existing Python style in `mps_youtube/`.
+
+- Use 4-space indentation and keep lines readable around the configured
+  80-character target.
+- Keep module and function names `snake_case`, classes `PascalCase`, and
+  constants `UPPER_SNAKE_CASE`.
+- Prefer small, focused command modules under `mps_youtube/commands/`.
+- Run `ruff` before committing; lint rules are configured in
+  `pyproject.toml`.
+
+## Testing guidelines
+There is currently no dedicated automated test suite in this repository.
+
+- Add new tests under a top-level `tests/` directory using `test_*.py` naming.
+- For behavior changes, include at least one automated test or document manual
+  verification steps in your pull request.
+- Minimum manual check: run `uv run yt` and verify the changed command path.
+
+## Commit & pull request guidelines
+The current Git history uses short, imperative commit messages
+(for example, `Capture current state for review`, `initial commit`).
+
+- Write concise, imperative commits focused on one logical change.
+- In pull requests, include: what changed, why, and how you validated it.
+- Link related issues when available and include terminal output or screenshots
+  for user-visible CLI behavior changes.

GEMINI.md

@@ -0,0 +1,46 @@
+# yewtube (yt) - Terminal YouTube Player
+
+This directory is a specialized workspace for **yewtube** (a fork of `mps-youtube`), a terminal-based YouTube player and downloader that does not require a YouTube API key.
+
+## Project Overview
+
+- **Purpose**: Search, stream, and download YouTube content directly from the command line.
+- **Core Technologies**: Python 3.x, `yt-dlp`, `requests`, `pafy`, and `youtube-search-python`.
+- **Playback Engines**: Supports `mpv`, `mplayer`, and `VLC`.
+- **Architecture**: A terminal-based interactive CLI that interfaces with YouTube's web interface (via `pafy` and `yt-dlp`) to bypass API requirements.
+
+## Environment & Tools
+
+The project is managed via a `uv` virtual environment located in `.venv/`.
+
+### Key Executables
+- `yt`: The main interactive terminal interface.
+- `yt-dlp`: The underlying engine used for downloading and extracting media information.
+
+## Usage Guide
+
+To start the interface, run:
+```bash
+./.venv/bin/yt
+```
+
+### Common Commands (Internal to `yt`)
+- `/ <search terms>`: Search for videos.
+- `// <search terms>`: Search for playlists.
+- `1-5`: Play videos 1 through 5.
+- `d 1`: Download video 1.
+- `set <option> <value>`: Configure settings (e.g., `set player mpv`).
+- `h`: Show help.
+
+## Development Context
+
+- **Fork Status**: This is a fork of the original `mps-youtube` project, maintained to ensure continued functionality without official API keys.
+- **Configuration**: User configuration and local playlists are typically stored in `~/.config/mps-youtube` or `~/.config/yewtube`.
+- **Dependencies**: Relies heavily on `yt-dlp` for extracting streams. If playback fails, updating `yt-dlp` is often the first step:
+  ```bash
+  ./.venv/bin/python -m pip install -U yt-dlp
+  ```
+
+## Troubleshooting
+
+If you encounter `ModuleNotFoundError: No module named 'pip'` when running `yt`, it may be due to the specific `uv` environment configuration. Ensure all dependencies are correctly linked in `site-packages`.

IMPROVEMENTS.md

@@ -0,0 +1,119 @@
+# Improvements and Modernization of yewtube
+
+This document tracks the significant refactoring and modernizations applied to the yewtube (mps-youtube fork) terminal application.
+
+## 1. Dependency Modernization (Completed)
+- **Metadata Management**: Replaced legacy `pkg_resources` (via `pip._vendor`) with `importlib.metadata`, ensuring compatibility with Python 3.12+ and environments without `pip` internals.
+- **Environment Cleanup**: Removed unused or redundant pip modules including `gnureadline` (replaced by `prompt_toolkit`), `requests`, `charset-normalizer`, and `urllib3`, reducing the application footprint.
+
+## 2. CLI & Interactive Shell (Completed)
+- **Framework Migration**: Migrated from manual argument parsing to `click`, providing robust help generation and standard CLI patterns.
+- **Shell Upgrade**: Replaced standard `readline` with `prompt_toolkit` for all interactive input.
+- **Intelligent Completion**: Implemented a `CommandCompleter` with context-aware logic:
+    *   Suggests `true`/`false` only for boolean configuration items.
+    *   Provides playlist name completions for `open`, `view`, etc.
+    *   Provides command-specific sub-argument completions (e.g., for `sort`).
+- **Enhanced Exit Flow**: Improved the Ctrl-C exit confirmation prompt to support command history and tab completion.
+
+## 3. Playback Engine & mpv Integration (Completed)
+- **Robust IPC Communication**: Rewrote the `mpv` integration to use IPC sockets with improved connection retry logic and secure temporary directory management.
+- **Asynchronous Error Capture**: Implemented a background thread to capture `mpv` stderr, allowing the application to detect and report fatal initialization errors (e.g., driver failures) to the UI.
+- **Persistent Error Reporting**: Modified the playback loop to display captured errors for 3 seconds before refreshing, ensuring users can diagnose failures.
+- **Output Management**:
+    *   Muted standard diagnostic output (stdout) to keep the terminal clean.
+    *   Unified video suppression logic (fixed crashes when passing both `--no-video` and `--vid=no`).
+    *   Removed redundant double-quoting in subprocess arguments.
+- **Extensibility**: Added the `aux_mpv_cli_config` setting to allow users to pass arbitrary additional command-line arguments to `mpv`.
+- **Keyboard Interactivity**: Restored standard input flow to `mpv` while maintaining clean terminal output, ensuring Space (pause) and Arrow keys (seek) work during playback.
+
+## 4. Metadata & Sorting (Completed)
+- **Search Metadata**: Patched `youtubesearchpython` to correctly extract `publishedTime` from YouTube's varying renderer formats.
+- **Chronological Sorting**: Implemented a weight-based heuristic to convert relative upload strings (e.g., "2 weeks ago", "5 months ago") into sortable numeric values, fixing alphabetical sorting bugs.
+- **Matching Accuracy**: Improved the `best_song_match` algorithm in `album_search` and `spotify_playlist` by fixing duration variance scoring and preventing negative score components.
+
+## 5. Security & Reliability Fixs (Completed)
+- **Resource Management**: Implemented context managers for file downloads to prevent file handle leaks during network interruptions.
+- **Insecure Temp Files**: Replaced `tempfile.mktemp` with secure `tempfile.mkdtemp` in both `mpv` and `mplayer` backends to prevent race conditions.
+- **Safe Process Control**: Refactored the `VLC` backend to use granular process termination (`terminate()`) instead of aggressive system-wide `pkill`.
+- **API Modernization**: Replaced all remaining legacy `pafy.call_gdata` calls with modern search and metadata retrieval methods across `generate_playlist`, `spotify_playlist`, and `description_parser`.
+- **Input Robustness**: Fixed potential `IndexError` in the duration parser and `ZeroDivisionError` in metadata matching logic.
+- **Shadowing Fixes**: Resolved variable shadowing (e.g., `c` loop variable vs `c` color module) to prevent subtle logic errors.
+
+## 6. Code Quality (Completed)
+- **Linting & Formatting**: Adopted `ruff` for fast linting and formatting.
+- **Consistency**: Standardized on an 80-character line limit and project-wide formatting.
+- **Type Safety**: Converted bare `except` blocks to explicit `except Exception:` to prevent catching system signals or masking fatal errors.
+
+## 7. User Interface & Playback Control (Completed)
+- **Clean Playback UI**: Streamlined the status bar by replacing the scrolling track size and YouTube ID with a static track title (truncated to 20 characters), providing a cleaner and more readable playback experience.
+- **Sequential Autoplay**: Overhauled the `AUTOPLAY` logic to implement a predictable sequential flow through the current song list. This removed redundant random-related-video recursion in favor of stable list traversal.
+- **Advanced Navigation**:
+    - **Skip to Next**: Re-mapped the `q` key to skip to the next track when `AUTOPLAY` is enabled, rather than stopping playback.
+    - **Previous Track ('p')**: Added the `p` key for previous track navigation in both `mpv` and `mplayer` backends.
+    - **Help Legend**: Updated the in-player help text to dynamically show `[p] prev [q] next` when autoplay is active.
+- **Context-Aware Playback**: Refactored the playback command to provide the player with full list context and a `start_index`. This ensures that navigation keys (`p`/`q`) can correctly traverse the entire original list even when a single track is selected for play.
+- **MPlayer Modernization**: Restored standard terminal interactivity (`stdin`) to the `mplayer` backend, ensuring parity with `mpv` for keyboard controls like pause and seek.
+- **Diagnostic Transparency**: Improved exception handling in metadata retrieval to surface original error messages, aiding in the diagnosis of private or geo-restricted content.
+
+## 8. Extractor Modernization (yt-dlp Transition)
+- **Unified Extraction Layer**: Replaced the legacy `pafy` module with a modern `extractor.py` abstraction. This layer directly interfaces with `yt-dlp` and `youtubesearchpython`, providing a stable and high-performance foundation for all media operations.
+- **VideoInfo Abstraction**: Implemented a standardized `VideoInfo` class that ensures consistent metadata access across the entire application, eliminating brittle dictionary-key dependencies.
+- **Enhanced Stream Selection**:
+    - Leveraged `yt-dlp`'s detailed format metadata for more accurate stream identification (audio vs mixed vs video-only).
+    - Improved resolution-based filtering and bitrate-aware sorting for higher quality playback.
+    - Added resilient expiry parsing for stream URLs to optimize playback start times.
+- **Global terminology migration**: Conducted a project-wide cleanup to remove all legacy `pafy` naming from imports, variables, and global caches (e.g., `g.pafs` -> `g.metadata_cache`).
+- **Improved Metadata Retrieval**: Unified search and playlist metadata processing to use modern scraping techniques, significantly reducing "no results" or "extraction failed" errors.
+- **Search Fallback Mechanism**: Implemented a robust fallback to `yt-dlp` for video searches when `youtubesearchpython` crashes or fails (common with high-profile artists like 'Lana Del Rey' due to complex shelf renderers).
+
+## 9. Full yt-dlp Migration (Completed)
+- **Eliminated Scraping Dependencies**: Successfully migrated the entire application from `youtube-search-python` to an internal `yt-dlp` and direct API-based implementation. This removes fragile HTML scraping dependencies that frequently broke on YouTube UI updates.
+- **Unified Search Engine**: All search types (video, playlist, and channel) now use the robust `yt-dlp` search backend with optimized filters (e.g., `sp=EgIQAw%3D%3D` for playlists).
+- **High-Performance Discovery**: 
+    - **Playlist Loading**: Re-implemented `get_playlist` using `yt-dlp`'s `extract_flat` mode, providing significantly faster loading of large playlists.
+    - **Channel Content**: Refactored channel video and playlist retrieval to use `yt-dlp`, ensuring stable metadata across the discovery workflow.
+- **Lightweight Suggestions**: Replaced the legacy `Suggestions` class with a direct, asynchronous-ready API call to the Google Suggest service, reducing overhead and improving response times.
+- **Advanced Metadata**: 
+    - **Comment Extraction**: Re-integrated video comment viewing using `yt-dlp`'s `getcomments` capability.
+    - **Dislike Restoration**: Maintained integration with the `Return YouTube Dislike` API alongside `yt-dlp` metadata.
+
+## 10. Search Performance Optimization (Completed)
+- **Extractor Filtering**: Optimized `yt-dlp` initialization by restricting allowed extractors to `youtube:search` and `youtube`, cutting search startup time by reducing unnecessary extractor checks.
+- **Lazy Extraction**: Enabled `lazy_extract` for all search types, allowing the application to fetch result lists without waiting for detailed metadata for every individual entry.
+- **Disk I/O Reduction**: Disabled the `yt-dlp` cache directory for search operations to eliminate unnecessary disk writes during ephemeral queries.
+
+## 11. Architectural Refinements (Completed)
+- **Robust Metadata Mapping**: Implemented a `MockPlaylist` compatibility layer to bridge `yt-dlp` raw output with the existing UI, preventing attribute errors during display.
+- **Safe Network Handling**: Added timeouts and comprehensive error handling to all network-bound operations (e.g., content-length retrieval and search suggestions).
+- **Resolution Resilience**: Hardened the stream selection logic to handle non-standard or malformed resolution strings (e.g., "720p" vs "1280x720") common in `yt-dlp` results.
+- **Initialization Integrity**: Improved the main entry point to ensure the application stops immediately on initialization failures, preventing cascaded state errors.
+- **Pydantic Configuration**: Migrated the legacy custom configuration system to Pydantic `BaseSettings`. This provides robust type validation, environment variable support, and modern JSON serialization while maintaining full backward compatibility with the existing interactive CLI through a bridge layer.
+
+## 12. Recent Stabilization Updates (Completed)
+- **Argparse Unification**: Standardized CLI parsing on `argparse` end-to-end, removing the mixed parser state that previously caused runtime namespace mismatches.
+- **CLI Cookie Support**: Added first-class authentication switches:
+    - `--cookies <path>` for Netscape-format cookie files.
+    - `--cookies-from-browser <browser[:profile]>` for direct browser cookie loading.
+- **Cookie Source Hardening**: Removed cookie loading from environment variables and switched to explicit CLI-driven configuration (with `~/.config/mps-youtube/cookies.txt` fallback support retained in extractor logic).
+- **Browser-like Request Headers**: Added a shared browser User-Agent/header profile and applied it consistently to `yt-dlp` and direct HTTP requests to reduce anti-bot false positives.
+- **Search Error Noise Reduction**: Suppressed the specific YouTube bot-check warning when search results are still successfully returned, while preserving errors for empty/failed searches.
+- **Runtime Regression Fixes**:
+    - Restored `g.text` binding to fix `module 'mps_youtube.g' has no attribute 'text'`.
+    - Hardened CLI argument access in initialization to avoid missing-attribute crashes.
+    - Replaced stray runtime `print()` calls with debug logging in playback/config paths.
+- **Lint & Consistency Pass**: Ran a project-wide `ruff --fix` cleanup and resolved remaining manual lint issues.
+
+## 13. Hybrid Discovery Engine (Completed)
+- **Migrated to `youtube-search-next`**: Switched discovery (search, playlists, channels) to the `youtube-search-next` library for high-speed, maintained access to the YouTube internal API.
+- **Unified Extractor**: Updated `extractor.py` to prioritize library-based discovery while retaining `yt-dlp` as a robust fallback for complex metadata and restricted content.
+- **Redundant Code Removal**: Deleted the manual `mps_youtube/innertube.py` implementation as its functionality is now superiorly handled by the external library.
+
+## 14. Playback & UI Reliability Updates (Completed)
+- **MPRIS Navigation Enhancement**: Added dedicated exit codes (44, 45) for Next and Previous commands. This allows the playback loop to distinguish manual MPRIS skips from natural player exits, ensuring reliable queue progression.
+- **Fixed Track Date Display**: 
+    - Improved metadata parsing to extract upload ages (e.g., "5 months ago") from `accessibility` labels when standard fields are missing.
+    - Increased default column sizes for "Date" and "Time" from 8 to 14 characters to prevent truncation of relative date strings.
+- **Stream Extraction Stability**: 
+    - Refactored `StreamURLFetcher` to use the `yt-dlp` backend for URL resolution, eliminating 403 Forbidden errors encountered with the library-based fetcher.
+    - Fixed a critical playback bug by ensuring `vcodec` and `acodec` are always normalized to lowercase strings (e.g., `'none'`), preventing empty stream lists.
+- **Full Channel Metadata**: Implemented a direct Innertube `browse` call for the `user <name>` command to retrieve complete video metadata (age, view counts) which is often stripped in flat playlist listings.

mps_youtube/.gitignore

@@ -0,0 +1,2 @@
+.venv
+__pycache__

mps_youtube/__init__.py

@@ -1,7 +1,10 @@
-from pip._vendor import pkg_resources
+import importlib.metadata
+
+try:
+    __version__ = importlib.metadata.version("yewtube")
+except importlib.metadata.PackageNotFoundError:
+    __version__ = "unable to determine"
 
-__version__ = next((p.version for p in pkg_resources.working_set if p.project_name.lower() == 'yewtube'), "unable to determine")
 __author__ = "iamtalhaasghar"
 __license__ = "GPLv3"
 __url__ = "https://github.com/mps-youtube/yewtube"
-

mps_youtube/c.py

@@ -1,4 +1,4 @@
-""" Module for holding colour code values. """
+"""Module for holding colour code values."""
 
 import os
 import re
@@ -8,23 +8,25 @@
     white = "\x1b[%sm" % 0
     ul = "\x1b[%sm" * 2 % (2, 4)
 
-    if not os.getenv('NO_COLOR'):
+    if not os.getenv("NO_COLOR"):
         ul += "\x1b[%33m"
 
     cols = ["\x1b[%sm" % n for n in range(91, 96)]
-    red, green, yellow, blue, pink = [""]*5 if os.getenv('NO_COLOR') else cols
+    red, green, yellow, blue, pink = [""] * 5 if os.getenv("NO_COLOR") else cols
 else:
     ul = red = green = yellow = blue = pink = white = ""
 
 r, g, y, b, p, w = red, green, yellow, blue, pink, white
 
-ansirx = re.compile(r'\x1b\[\d*m', re.UNICODE)
+ansirx = re.compile(r"\x1b\[\d*m", re.UNICODE)
+
 
 def c(colour, text):
-    """ Return coloured text. """
-    colours = {'r': r, 'g': g, 'y': y, 'b':b, 'p':p}
+    """Return coloured text."""
+    colours = {"r": r, "g": g, "y": y, "b": b, "p": p}
     return colours[colour] + text + w
 
+
 def charcount(s):
-    """ Return number of characters in string, with ANSI color codes excluded. """
-    return len(ansirx.sub('', s))
+    """Return number of characters in string, with ANSI color codes excluded."""
+    return len(ansirx.sub("", s))