|
7 | 7 | Accelerated video decoding on GPUs with CUDA and NVDEC |
8 | 8 | ================================================================ |
9 | 9 |
|
10 | | -.. _ndecoderec_tutorial: |
11 | | -
|
12 | 10 | TorchCodec can use supported Nvidia hardware (see support matrix |
13 | 11 | `here <https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new>`_) to speed-up |
14 | 12 | video decoding. This is called "CUDA Decoding" and it uses Nvidia's |
|
45 | 43 | #. An Nvidia GPU that supports decoding the video format you want to decode. See |
46 | 44 | the support matrix `here <https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new>`_ |
47 | 45 | #. `CUDA-enabled pytorch <https://pytorch.org/get-started/locally/>`_ |
48 | | -#. FFmpeg binaries that support NdecoderEC-enabled codecs |
| 46 | +#. FFmpeg binaries that support |
| 47 | + `NVDEC-enabled <https://docs.nvidia.com/video-technologies/video-codec-sdk/12.0/ffmpeg-with-nvidia-gpu/index.html>`_ |
| 48 | + codecs |
49 | 49 | #. libnpp and nvrtc (these are usually installed when you install the full cuda-toolkit) |
50 | 50 |
|
51 | 51 |
|
52 | | -FFmpeg versions 5, 6 and 7 from conda-forge are built with NdecoderEC support and you can |
53 | | -install them with conda. For example, to install FFmpeg version 7: |
| 52 | +FFmpeg versions 5, 6 and 7 from conda-forge are built with |
| 53 | +`NVDEC support <https://docs.nvidia.com/video-technologies/video-codec-sdk/12.0/ffmpeg-with-nvidia-gpu/index.html>`_ |
| 54 | +and you can install them with conda. For example, to install FFmpeg version 7: |
54 | 55 |
|
55 | 56 |
|
56 | 57 | .. code-block:: bash |
|
117 | 118 | # |
118 | 119 | # The video frames are decoded and returned as tensor of NCHW format. |
119 | 120 |
|
120 | | -print(frame.data.shape, frame.data.dtype) |
| 121 | +print(frame.shape, frame.dtype) |
121 | 122 |
|
122 | 123 | # %% |
123 | 124 | # |
|
132 | 133 | # |
133 | 134 | # Let's look at the frames decoded by CUDA decoder and compare them |
134 | 135 | # against equivalent results from the CPU decoders. |
135 | | -import matplotlib.pyplot as plt |
136 | | -from torchvision.transforms.v2.functional import to_pil_image |
137 | | - |
138 | | - |
139 | | -def get_frames(timestamps: list[float], device: str): |
140 | | - decoder = VideoDecoder(video_file, device=device) |
141 | | - return [decoder.get_frame_played_at(ts).data for ts in timestamps] |
142 | | - |
143 | | - |
144 | 136 | timestamps = [12, 19, 45, 131, 180] |
145 | | -cpu_frames = get_frames(timestamps, device="cpu") |
146 | | -cuda_frames = get_frames(timestamps, device="cuda") |
| 137 | +cpu_decoder = VideoDecoder(video_file, device="cpu") |
| 138 | +cuda_decoder = VideoDecoder(video_file, device="cuda") |
| 139 | +cpu_frames = cpu_decoder.get_frames_played_at(timestamps).data |
| 140 | +cuda_frames = cuda_decoder.get_frames_played_at(timestamps).data |
147 | 141 |
|
148 | 142 |
|
149 | 143 | def plot_cpu_and_cuda_frames( |
150 | 144 | cpu_frames: list[torch.Tensor], cuda_frames: list[torch.Tensor] |
151 | 145 | ): |
| 146 | + try: |
| 147 | + import matplotlib.pyplot as plt |
| 148 | + from torchvision.transforms.v2.functional import to_pil_image |
| 149 | + except ImportError: |
| 150 | + print("Cannot plot, please run `pip install torchvision matplotlib`") |
| 151 | + return |
152 | 152 | n_rows = len(timestamps) |
153 | 153 | fig, axes = plt.subplots(n_rows, 2, figsize=[12.8, 16.0]) |
154 | 154 | for i in range(n_rows): |
|
0 commit comments