Next video optimization

[pedrohenriquebraga]

I wish there was something like next/image but for videos. I don't know if it would be viable since it would require a lot of processing (I think), but maybe you will find a way to make it viable.

As I said, it would be something like next/image, it would reduce the quality of the video, convert it to webm, maybe even stream the files (like uploading YouTube in parties).

I hope you enjoyed the suggestion and are able to add it. Good luck!

[BrunoSette]

Great idea!

[guhandelta]

I am not clearly sure if my idea(s) would be a good one(s), but I hope they could use some cool tech like socket programming(it has been there for ages, and should there be something equivalent that can be implemented using JS stack) to optimize real-time video rendering. If not, they could also try to utilize web assembly in any other new ways, to churn up a technique to optimize videos.

It might seem crazy, but it impossible these days to limit the abilities of any tech. Who knows, someday PHP might become a huge deal as like the world of JS and Python.

[AxelReid]

Absolutely! Stream + webm would be so sweet candy

[0xsot]

agreed!

[slavakurilyak]

What's missing from Next.js? Video components and video optimizations!

For inspiration, check out the following open-source projects.

Inspiration 1

Look at @cloudinary open-source work for rendering video components using React.js:

• npmjs.com/package/cloudinary-react
• npmjs.com/package/@cloudinary/react

You can specify transformations using the Video element, which automatically generates an HTML5 video tag including the URL sources for the main formats supported by web browsers (webm, mp4 and ogv), as well as a poster thumbnail image, which is automatically generated from a frame in the video. This enables the browser to automatically select and play the video format it supports. The video files are created dynamically when first accessed by your users.

For example,

<Video cloudName="my_cloud" publicId="watchme">
  <Transformation overlay="text:arial_60:watchme" gravity="north" y="20" />
</Video>

Will be compiled by React to:

<video>
  <source src="https://res.cloudinary.com/my_cloud/video/upload/g_north,l_text:arial_60:watchme,y_20/watchme.webm" type="video/webm">
  <source src="https://res.cloudinary.com/my_cloud/video/upload/g_north,l_text:arial_60:watchme,y_20/watchme.mp4" type="video/mp4">
  <source src="https://res.cloudinary.com/my_cloud/video/upload/g_north,l_text:arial_60:watchme,y_20/watchme.ogv" type="video/ogg">
</video>

Inspiration 2

Check out @imagekit-developer work on URL-based transformations (video transformation) and video optimization:

• npmjs.com/package/imagekitio-react

Here's an simple URL structure:

        URL-endpoint        transformation   video path                                    
┌──────────────────────────┐┌────────────┐┌───────────────┐
https://ik.imagekit.io/demo/tr:w-300,h-300/sample-video.mp4

[slavakurilyak]

Videos make up the majority of all consumer internet traffic.

It's time to implement a <Video> component in Next.js!

For inspiration, check out:

1. Offical Mux API wrapper for Node projects: Github Repo
2. Offical api.video API wrapper for Node projects: Github Repo

[dylanjha]

Love this idea ❤️ and would love to help. We have the Mux Player in beta now:

React example: https://codesandbox.io/s/mux-player-react-uqz58z
Docs: https://docs.mux.com/guides/video/mux-player

[lachlanjc]

^ highly recommend Mux for this need! Would be so amazing to have a Vercel <> Next.js <> Mux integration that provided this functionality without a separate upload step/env vars/etc.

[DonHuskini]

If your video has autoplay=false, then you can use preload=none but if you're using it as a gif autoplay=true, then it'll be preloaded independently if it's in the viewport or not. This is usually undesirable and would be better to lazy load it with the Intersection Observer API. Here's a solution for this case:

/* /src/scripts/video-lazyload.jsx */

import Script from 'next/script';

export default function VideoLazyLoad() {
  return (
    <Script id="show-banner" strategy="afterInteractive">
      {`!function(){var e=[].slice.call(document.querySelectorAll("video.lazy"));if("IntersectionObserver"in window){var r=new IntersectionObserver(function(e,t){e.forEach(function(e){if(e.isIntersecting){for(var t in e.target.children){var a=e.target.children[t];"string"==typeof a.tagName&&"SOURCE"===a.tagName&&(a.src=a.dataset.src)}e.target.load(),e.target.classList.remove("lazy"),r.unobserve(e.target)}})});e.forEach(function(e){r.observe(e)})}}();`}
    </Script>
  );
}

/* src/pages/_app.jsx */

import VideoLazyLoad from 'scripts/video-lazyload';

export default function App({ Component, pageProps }: AppProps) {
  return (
    <VideoLazyLoad />
    <Component {...pageProps} />
  )
}

Then the videos you want to lazy load, add the lazy class on the video element and data-source attribute on the source.

<video class="lazy">
  <source data-src={src}  />
</video>

References:

• https://www.roboleary.net/frontend/2022/03/28/why-doesnt-the-video-element-have-native-lazy-loading.html
• https://web.dev/lazy-loading-video/

[pedrohenriquebraga]

All the answers are amazing and useful, but they are a little off the original idea. The idea is to add INSIDE the NextJs a solution for video optimization using NextJs/Vercel itself, not using paid tools.

I'm not saying the answers given aren't useful or can't be used, but it's not what I had in mind when I opened this discussion.

[dylanjha]

The idea is to add INSIDE the NextJs a solution for video optimization using NextJs/Vercel itself, not using paid tools.

I'd love to see something like this directly in Next.js. The first time I used next/image blew my mind a little bit with how simple and powerful it is. Doing the same for video would be great for everyone building with Next.

A few of the challenges with this (and why video is not quite as easy as images):

• Images are small enough to be checked into source control (couple hundred KB, maybe a couple MB at most). Video files are way too big to be sitting in version control in a git repo (many MBs or GBs). If anything you would have to use Git LFS for large files. That comes with some extra considerations to think about.
• Even if you do get them into source control with Git LFS, the processing required on images is might lighter weight. You can actually run it in the next.js process on your laptop, in real-time when the user request the image file. Video processing takes a lot more time and CPU. A single 10 minute video transcoded to 5 different bitrates for HLS delivery in 6 second segments would put out 500 video files and that process can take several minutes to run via something like ffmpeg. It's sort of impractical for everyone running a dev server to keep re-transcoding the same videos over and over again like it does with images.

That's not to say that any of these challenges can't be overcome, and especially if we're talking about very short videos that are say, under a minute long. Things you might use for background hero videos or small graphics on the page, then that gets significantly easier to figure out.

[haensl]

While this might not be directly related to video optimization, I see potential for next/image in conjunction with <video poster="" />.

As far as I can tell, there is currently no way to elegantly load an image from next.js's optimization cache when it isn't used as an element on the page. However, video thumbnails, i.e. the poster attribute of the <video> element, would be an awesome place to make use of Next.js' amazing image optimization.

[ViceStudio]

i have been unable to come up with a simple boiler plate solution for this. I have some code where i have the video play on hover and then resets off hover. The poster image is great for what I'm trying to do but the images are not optimized. I have tried CSS and tailwind solution of swapping the Image and Video elements to create a poster like solution but I can't get it to work right and sort of janky even at the 80 percent solution. Not sure if there is someone that created a boiler plate solution for using nextjs image as poster for this type of solution. i have looked but didn't find it.

[joshbedo]

+1 next/video should 100% exist

[dylanjha]

Folks on this thread might be interested in this RFC for next/video

[shuence]

Would love to have a NPM package related to it, let me try building one.