Add HLS streaming for instant playback while downloading

[hauxir]

Summary

• Add an HLS streaming pipeline that pipes sequential torrent bytes into ffmpeg so playback can start before the download/MP4 conversion completes (MKV/AVI/TS containers; MP4/MOV still wait for full download since the mov demuxer needs seeking).
• Keep the existing MP4 conversion pipeline as the primary output and seamlessly switch the player from HLS to MP4 once conversion finishes, preserving playback position and subtitle selection.
• Expose subtitles via the HLS master playlist (EXT-X-MEDIA TYPE=SUBTITLES) so hls.js manages text tracks natively, with language normalization and disambiguation for duplicate-language tracks.
• Harden error handling: codec tagging for HEVC fmp4, frag_keyframe + reset_timestamps for clean seeking, retry/recovery on hls.js network/media errors, and a 50MB sequential-byte threshold gating the stream button.

Test plan

• Stream an MKV torrent before it finishes downloading and confirm playback starts once 50MB is sequential.
• Verify seamless HLS→MP4 switch when conversion completes (position + selected subtitle preserved).
• Subtitle language preference applies on initial load and survives the HLS→MP4 switch; duplicate-language tracks are distinguishable.
• HEVC source plays in Chrome (hvc1 tag) and falls back cleanly to the loading screen if HLS fails.
• MP4 source files do not show the stream button (need seeking, can't pipe).

🤖 Generated with Claude Code

[qodo-free-for-open-source-projects]

Review Summary by Qodo

Add HLS streaming for instant playback while downloading non-MP4 video files

✨ Enhancement

Walkthroughs

Description

• Add HLS streaming pipeline for instant video playback while downloading non-MP4 files
  - Pipe sequential torrent bytes into ffmpeg for fMP4 segment generation
  - Support MKV/AVI/TS containers; MP4/MOV wait for full download (require seeking)
  - Seamlessly switch from HLS to MP4 once conversion completes, preserving playback position
• Expose subtitles via HLS master playlist with EXT-X-MEDIA tracks for native hls.js management
  - Extract embedded subtitles as VTT files during streaming
  - Language normalization and disambiguation for duplicate-language tracks
• Harden error handling and add recovery mechanisms
  - Codec tagging for HEVC (hvc1) and H.264 (avc1) fMP4 segments
  - Retry/recovery on hls.js network and media errors with max recovery limit
  - Delete incomplete m3u8 on ffmpeg failure to trigger daemon retry
  - Self-healing: finalize orphaned m3u8 files by switching EVENT→VOD and appending ENDLIST
• Add UI stream button (▶) for eligible files with 50MB sequential-byte threshold
  - Show download progress badge in player topbar during HLS playback
  - Disable stream button for MP4 files (cannot pipe, require seeking)

Diagram

flowchart LR
  A["Torrent Download"] -->|Sequential Bytes| B["HLS Streamer"]
  B -->|ffmpeg stdin| C["fMP4 Segments"]
  C -->|hls.js Player| D["Early Playback"]
  A -->|Full Download| E["MP4 Converter"]
  E -->|MP4 Ready| F["Seamless Switch"]
  D -->|Position Preserved| F
  B -->|Extract VTT| G["Subtitles"]
  G -->|Master Playlist| H["Track Selection"]

Loading

File Changes

1. app/app.py ✨ Enhancement +13/-1

Add HLS streaming fields and endpoint to API responses

app/app.py

---

2. app/rapidbaydaemon.py ✨ Enhancement +160/-31

Implement HLS streaming orchestration and subtitle extraction

app/rapidbaydaemon.py

---

3. app/settings.py ⚙️ Configuration changes +4/-0

Add HLS segment duration and start threshold configuration

app/settings.py

---

View more (8)

4. app/torrent.py ✨ Enhancement +28/-0

Add sequential byte calculation and sequential download flag

app/torrent.py

---

5. app/video_conversion.py ✨ Enhancement +258/-5

Implement HLSStreamer class with pipe feeding and master playlist generation

app/video_conversion.py

---

6. app/frontend/app.js ✨ Enhancement +161/-43

Add hls.js integration with error recovery and subtitle track management

app/frontend/app.js

---

7. app/frontend/index.html ✨ Enhancement +8/-3

Load hls.js library and add stream button to download screen

app/frontend/index.html

---

8. app/frontend/style.css ✨ Enhancement +1/-0

Add download progress badge styling for player topbar

app/frontend/style.css

---

9. app/run.sh ⚙️ Configuration changes +5/-3

Make listen port configurable via environment variable

app/run.sh

---

10. Dockerfile ⚙️ Configuration changes +1/-0

Remove default nginx site configuration

Dockerfile

---

11. nginx.conf ⚙️ Configuration changes +11/-1

Add MIME types for HLS and video segments, make port configurable

nginx.conf

---

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

1. Shell ffmpeg injection ☑ 🐞 Bug ⛨ Security

Description

RapidBayDaemon._download_external_subtitles builds an ffmpeg command string using
filesystem-derived paths and executes it with shell=True, allowing command injection if a
torrent-derived basename or subtitle filename contains quotes/shell metacharacters. This can lead to
arbitrary command execution under the app user.

Code

app/rapidbaydaemon.py[R410-426]

+        # Copy downloaded .srt files to output dir as .vtt for HLS playback
+        dirname = os.path.dirname(filepath)
+        basename = os.path.basename(filepath)
+        basename_without_ext = os.path.splitext(basename)[0]
+        for srt_file in os.listdir(dirname):
+            if srt_file.endswith(".srt") and srt_file.startswith(basename_without_ext):
+                srt_path = os.path.join(dirname, srt_file)
+                srt_stem = os.path.splitext(srt_file)[0]
+                lang = srt_stem[len(basename_without_ext):].lstrip(".")
+                vtt_name = f"{basename_without_ext}_{lang}.vtt" if lang else srt_stem + ".vtt"
+                vtt_path = os.path.join(output_dir, vtt_name)
+                if not os.path.isfile(vtt_path):
+                    from subprocess import Popen
+                    Popen(
+                        f'ffmpeg -nostdin -v quiet -i "{srt_path}" "{vtt_path}"',
+                        shell=True,
+                    ).wait()

Evidence

The PR introduces a new shell=True invocation where the interpolated arguments are derived from
filepath (torrent filename influences basename) and directory listing results, making them
attacker-controlled in the torrent threat model.

app/rapidbaydaemon.py[401-427]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`_download_external_subtitles()` executes ffmpeg via a shell string with paths derived from torrent-controlled filenames. A filename containing quotes or shell metacharacters can break out of the quoted string and inject commands.
### Issue Context
This code runs on the server and processes filenames originating from torrents / downloaded subtitles, which should be treated as untrusted.
### Fix
- Replace the `Popen(f'ffmpeg ...', shell=True)` call with a argv-list invocation (`shell=False`).
- Consider switching to `subprocess.run([...], check=False)` for simplicity.
- Ensure paths are passed as raw arguments (no manual quoting).
### Fix Focus Areas
- app/rapidbaydaemon.py[401-427]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Stream start may lie ☑ 🐞 Bug ≡ Correctness

Description

start_hls_stream() returns True once the sequential-byte threshold is met even though
HLSStreamer.start_stream() may immediately return without marking the stream active when
MAX_PARALLEL_CONVERSIONS capacity is reached. This makes /api/.../stream report success when no
stream will run.

Code

app/rapidbaydaemon.py[R355-382]

+    def start_hls_stream(self, magnet_hash: str, filename: str) -> bool:
+        """Start HLS streaming for a file. Returns True if stream was started or already active."""
+        is_video = os.path.splitext(filename)[1][1:] in settings.VIDEO_EXTENSIONS
+        if not is_video:
+            return False
+        m3u8 = _m3u8_path(magnet_hash, filename)
+        if os.path.isfile(m3u8) or self.hls_streamer.active_streams.get(m3u8):
+            return True  # Already streaming or complete
+        h = self.torrent_client.torrents.get(magnet_hash)
+        if not h or not h.has_metadata():
+            return False
+        i, f = torrent.get_index_and_file_from_files(h, filename)
+        if i is None or f is None:
+            return False
+        available_bytes = self._get_available_bytes(magnet_hash, filename)
+        if available_bytes < settings.HLS_START_THRESHOLD:
+            return False
+        filepath = os.path.join(settings.DOWNLOAD_DIR, magnet_hash, f.path)
+        output_dir = _get_output_dir(magnet_hash)
+        os.makedirs(output_dir, exist_ok=True)
+        self.hls_streamer.start_stream(
+            filepath,
+            output_dir,
+            lambda _mh=magnet_hash, _fn=filename: self._get_available_bytes(_mh, _fn),
+            total_file_size=f.size,
+            m3u8_filename=_m3u8_filename(filename),
+        )
+        return True

Evidence

The daemon endpoint always returns True after invoking the threaded start, but the streamer has an
early capacity return before it adds an entry to active_streams, meaning the stream is neither
started nor active in that case.

app/rapidbaydaemon.py[355-382]
app/video_conversion.py[225-247]
app/settings.py[50-66]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`start_hls_stream()` reports `started=True` even when the stream thread will immediately exit due to `MAX_PARALLEL_CONVERSIONS`.
### Issue Context
`HLSStreamer.start_stream()` is `@threaded`, so the caller can’t observe the early-return decision unless it performs the capacity check itself (or the streamer exposes a synchronous "reserve slot" API).
### Fix
- Add a capacity check in `RapidBayDaemon.start_hls_stream()` before calling `hls_streamer.start_stream()` (e.g., compare `len(active_streams)` against `MAX_PARALLEL_CONVERSIONS`).
- Alternatively, refactor `HLSStreamer` so the capacity check and `active_streams[m3u8_path]=True` reservation happen synchronously in a non-threaded method, and only the ffmpeg work is threaded.
- Only return `True` when a stream is already active/complete or was successfully reserved/started.
### Fix Focus Areas
- app/rapidbaydaemon.py[355-382]
- app/video_conversion.py[225-247]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Pipe feeder tight loop ☑ 🐞 Bug ➹ Performance

Description

HLSStreamer._pipe_feeder breaks its inner loop when f.read() returns empty while `available >
bytes_written`, and then immediately retries without any sleep, which can cause a busy loop. This
can happen when the sequential-byte estimator exceeds the bytes currently flushed/available via the
file read.

Code

app/video_conversion.py[R350-356]

+                    to_read = available - bytes_written
+                    while to_read > 0:
+                        chunk_size = min(to_read, PIPE_READ_CHUNK)
+                        data = f.read(chunk_size)
+                        if not data:
+                            break
+                        try:

Evidence

When available > bytes_written, the outer loop skips the time.sleep(0.5) guard. If f.read()
returns b'' (EOF because file isn’t extended yet), the code breaks only the inner loop and
immediately recomputes to_read again, potentially spinning. The sequential-byte computation is
piece-based and does not reference current on-disk file size, so it can overestimate relative to
readable bytes.

app/video_conversion.py[339-366]
app/torrent.py[255-280]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The pipe feeder can enter a tight retry loop when it expects more bytes (`available > bytes_written`) but the file read returns empty (EOF).
### Issue Context
`available` is derived from torrent piece state / HTTP progress, not from the file’s current readable size.
### Fix
- If `f.read()` returns empty, add a small sleep and `continue` the outer loop (instead of immediately recomputing without delay).
- Consider capping `available` by `os.path.getsize(filepath)` to ensure feeder never tries to read past current file size.
- Optionally track “no progress” duration and abort the stream if stalled.
### Fix Focus Areas
- app/video_conversion.py[325-366]
- app/torrent.py[255-280]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. No HLS wait timeout ☑ 🐞 Bug ☼ Reliability

Description

For non-pipe inputs, HLSStreamer.start_stream waits in a loop until `get_sequential_bytes() >=
total_file_size with no timeout, while the stream is already counted in active_streams`. A stalled
download can keep a stream slot occupied indefinitely, preventing additional streams from starting.

Code

app/video_conversion.py[R291-294]

+                    # MP4/MOV need seeking — wait for full file, then use direct input
+                    while get_sequential_bytes() < total_file_size:
+                        time.sleep(1)
+                    proc = Popen(ffmpeg_cmd, stdin=DEVNULL, stdout=DEVNULL, stderr=stderr_log)

Evidence

The code marks the stream active before entering a potentially unbounded wait loop, and
active_streams entries are only removed in finally, so a never-completing wait keeps capacity
consumed (the streamer also enforces len(active_streams) >= MAX_PARALLEL_CONVERSIONS).

app/video_conversion.py[239-247]
app/video_conversion.py[290-295]
app/video_conversion.py[321-323]
app/settings.py[55-66]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Non-pipe HLS startup blocks waiting for the full file with no timeout, keeping the stream marked active and consuming the limited active-stream capacity.
### Issue Context
Even if the UI doesn’t expose streaming for MP4/MOV, the backend API can still be called, and stalled downloads can accumulate waiting threads/active slots.
### Fix
- Add a max wait timeout (and/or max “no progress” duration) to the `while get_sequential_bytes() < total_file_size` loop.
- If timed out, remove the `active_streams` entry and exit cleanly.
- Consider refusing HLS for non-pipe formats at the daemon API level unless explicitly allowed.
### Fix Focus Areas
- app/video_conversion.py[239-247]
- app/video_conversion.py[290-295]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---