Update Video Encoder and tests for 6 container formats #913
Conversation
meta-cla
bot
added
the
CLA Signed
Dan-Flores
force-pushed
the
encoder_accuracy
branch
3 times, most recently
from
f414d0b to
c29dee3
Compare
Dan-Flores
force-pushed
the
encoder_accuracy
branch
from
fe8fb87 to
4516b35
Compare
|
@Dan-Flores has imported this pull request. If you are a Meta employee, you can view this in D84393092. |
There was a problem hiding this comment.
Nice, thanks for the PR @Dan-Flores ! Approving with some minor comments below, nothing really blocking as long as CI is green
test/test_ops.py
Outdated
| ) | ||
|
|
||
| def decode(self, file_path) -> torch.Tensor: | ||
| def decode(self, file_path, device="cpu") -> torch.Tensor: |
There was a problem hiding this comment.
I think "cpu" is always the default, isn't it? We should be able to revert 163c5c2 ?
test/test_ops.py
Outdated
| # If FFmpeg selects a codec or pixel format that uses qscale (not crf), | ||
| # the VideoEncoder outputs *slightly* different frames. | ||
| # There may be additional subtle differences in the encoder. | ||
| percentage = 94 if ffmpeg_version == 6 or format in ("avi") else 99 |
There was a problem hiding this comment.
The above works but technically,
format in ("avi")
returns True if format is "avi" but also "a", "v", "i", "av", and "vi".
| percentage = 94 if ffmpeg_version == 6 or format in ("avi") else 99 | |
| percentage = 94 if ffmpeg_version == 6 or format == "avi" else 99 |
| 0); | ||
| } | ||
| int status = avcodec_open2(avCodecContext_.get(), avCodec, &options); | ||
| av_dict_free(&options); |
There was a problem hiding this comment.
I guess this is OK, if we start using AVDictionary in more places we should consider creating a UniqueAVDictionary smart pointer, like we did for the other objects e.g.
torchcodec/src/torchcodec/_core/FFMPEGCommon.h
Lines 74 to 75 in e5b2eef
This reverts commit 163c5c2.
2 participants
This PR updates the VideoEncoder to support encoding for common video container formats, nearly identically to the FFmpeg CLI.
Changes:
Some changes are made to align with the design in #907:
crfas an option on the C++ side to enable round trip testsTesting
test_video_encoder_round_trip: Ensures that a video's decoded frames are the same after encoding then decoding.mov,mp4,mkv,webmtest_video_encoder_against_ffmpeg_cli: Ensures that the VideoEncoder frames are the same as the FFmpeg CLI.mov,mp4,avi,mkv,webm,flv,gifTesting caveats
crfparameter is needed to test lossless encoding in the round trip test. For formats that do not supportcrf, the round trip test is not availablle.assert_tensor_close_on_at_leastwith a lower percentage match (96-99), and a higheratol(2-15).