Skip to content

Add in-video overlays #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BlinkStrike wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Add in-video overlays #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f703dbd
add overlays + api debugging option
BlinkStrike Aug 10, 2025
9d62313
feat: add metadata utility functions for stream management and proces…
BlinkStrike Aug 10, 2025
baed33b
add locales
BlinkStrike Oct 2, 2025
25076ee
try changing video player
BlinkStrike Oct 2, 2025
9984d8b
restore the player as it was
BlinkStrike Oct 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
NONPUBLIC
.DS_Store
.VSCodeCounter
.env
.env.local
.env.development.local
.env.test.local
Expand Down
66 changes: 66 additions & 0 deletions docs/pr/2025-08-11-overlay-ffmpeg.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Add FFmpeg overlay support in createInputsOutputs (in‑video image overlays)

## Summary
- Implement overlay rendering via FFmpeg `-filter_complex` in `src/utils/metadata.js#createInputsOutputs()`.
- Add overlay image inputs with proper input options (`-loop 1`, `-framerate <n>`; default 30).
- Build a single filter graph that:
- Scales the base video (reuse existing profile scale if present; otherwise default `scale=1280:720`).
- Scales each overlay individually.
- Composites overlays in sequence with configurable positions.
- Appends `format=yuv420p` for broad player compatibility.
- Place `-filter_complex` in `outputs[0].options` and map the graph output with `-map [v]`.
- Preserve existing audio mapping/encoders. No behavior change when no overlays are configured.

## Background
The UI exposes overlay configuration (image path, scale, position, framerate), but previously the generated FFmpeg command did not merge overlays into the video pipeline. This PR wires the UI overlay config into the FFmpeg graph so overlays render in the encoded output.

## Changes
- `src/utils/metadata.js`
- Detect when overlays are present and add each overlay as a separate FFmpeg input with `-loop 1` and `-framerate` (default 30).
- Construct a filter graph:
- Overlay scaling: `[{overlayInput}:v]scale=<width>:-1[logoN]` (width defaults to 320 when not specified).
- Base video scaling:
- If `profile.video.filter.graph` contains a scale, reuse it.
- Else default to `scale=1280:720`.
- Use type-based stream specifier `[<inputIndex>:v]` (not `[<inputIndex>:0]`).
- Sequential compositing: `${current}[logoN]overlay=x=<x>:y=<y>`; labels chained until final `[v]`.
- Defaults: `x=(W-w)/2`, `y=(H-h)/2`.
- If `y` not provided for the first overlay, default to bottom with padding: `y=H-h-24`.
- Append `format=yuv420p` to the last link in the chain: `...,format=yuv420p[v]`.
- Attach `-filter_complex` to `outputs[0].options` (not global options) and replace the video `-map` with `-map [v]`.
- Keep audio options and mappings intact.

## Example (resulting options excerpt)
```sh
-filter_complex "[0:v]scale=1280:720[bg];[1:v]scale=320:-1[logo1];[bg][logo1]overlay=x=(W-w)/2:y=H-h-24[v],format=yuv420p[v]" \
-map [v] ...
```

## Implementation Notes
- The overlay pipeline is only activated when overlays are present. Otherwise the previous per-stream filter behavior is preserved.
- Stream selectors were normalized to type-based (`:v`) to reduce brittleness and to match the working example.
- `-filter_complex` is inserted at the beginning of `outputs[0].options` to ensure correct option ordering.

## Testing
- Add one or more overlays in the Edit view and save.
- Verify the API payload (enable request logging with `REACT_APP_API_DEBUG=true`) includes:
- Separate overlay inputs with `-loop 1` and `-framerate`.
- A single `-filter_complex` string with base video scale, per-overlay scale, overlay chaining, and final `format=yuv420p`.
- `-map [v]` for the video stream in `outputs[0].options`.
- Confirm visual correctness by starting a process and checking the output preview/target player.

## Limitations / Follow‑ups
- Overlays are applied to the first video profile/output. Multi-profile overlay routing is not yet implemented.
- Only image overlays are handled; animated overlays (e.g., GIF/MOV) may require enabling/disabling `-loop 1` based on media type.
- Additional positioning presets and per-overlay alpha controls can be added in the UI in future PRs.

## Risk and Rollback
- Low risk when overlays are disabled (old behavior preserved).
- Rollback by reverting the commits in `src/utils/metadata.js`.

## Checklist
- [x] Overlay inputs added with correct input options.
- [x] Filter graph composes base scale, overlay scales, and positions.
- [x] Final `format=yuv420p` appended and mapped with `-map [v]`.
- [x] Audio mapping unchanged.
- [x] Verified API payload with debug logging.
5 changes: 3 additions & 2 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,5 +94,6 @@
"prettier": "^3.3.3",
"react-error-overlay": "^6.0.11"
},
"resolutions": {}
}
"resolutions": {},
"packageManager": "[email protected]+sha512.a6b2f7906b721bba3d67d4aff083df04dad64c399707841b7acf00f6b133b7ac24255f2652fa22ae3534329dc6180534e98d17432037ff6fd140556e2bb3137e"
}
7 changes: 7 additions & 0 deletions src/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,18 @@ import CssBaseline from '@mui/material/CssBaseline';
import theme from './theme';
import RestreamerUI from './RestreamerUI';

const ENV_ADDRESS = process.env.REACT_APP_RESTREAMER_ADDRESS;

let address = window.location.protocol + '//' + window.location.host;
if (window.location.pathname.endsWith('/ui/')) {
address += window.location.pathname.replace(/ui\/$/, '');
}

// Override with env var if provided at build-time
if (ENV_ADDRESS && ENV_ADDRESS.length > 0) {
address = ENV_ADDRESS;
}

const urlParams = new URLSearchParams(window.location.search.substring(1));
if (urlParams.has('address') === true) {
address = urlParams.get('address');
Expand Down
2 changes: 1 addition & 1 deletion src/locales/da/messages.js
View file
Open in desktop

Large diffs are not rendered by default.

Loading