Exploring the feasibilty of AI Video Upscaling

[sathish-a]

Hello Video.js maintainers and community,

I am exploring the idea of building (or extending) an open-source web video player with an optional video upscaling feature powered by a super-resolution model, and I am evaluating whether Video.js would be a suitable platform for this work.

The high-level concept is to expose user-selectable options (for example, 2× and 4× upscaling) within the quality or resolution settings. Based on the selected option, a super-resolution model would be invoked to upscale the video after decoding. The approach I am considering involves buffering decoded frames, performing GPU-based inference for upscaling, and rendering the enhanced frames during playback.

To ensure fast startup and good interactivity, playback would initially begin at the native resolution to avoid any cold-start overhead associated with model initialization. In parallel, buffered frames could be pre-emptively upscaled in the background. During seek operations, the player would temporarily fall back to native-resolution playback to preserve responsiveness and low latency.

At this stage, I am not looking for code-level guidance. I would appreciate high-level input from the maintainers and community on:
• Whether Video.js’s plugin architecture and rendering pipeline can support this kind of decoded-frame processing
• Which extension points or APIs would be most appropriate for experimenting with GPU-based frame enhancement
• Any architectural constraints or best practices to consider before proceeding

I am mainly looking to brainstorm and validate feasibility before investing further effort. Thank you for your guidance and feedback.