Skip to content

Commit 708ca25

Browse files
walidbarakatwalidbarakat
committed
initial draft for direct API access for ffmpeg
initial draft for direct API access for ffmpeg Signed-off-by: Aly, Walid <[email protected]>
1 parent 5d6a774 commit 708ca25

File tree

4 files changed

+501
-0
lines changed

4 files changed

+501
-0
lines changed

‎FFmpeg_direct_API/README.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
```
2+
The start_processing() function handles the main processing loop:
3+
```
4+
5+
- Opens input files using the `mtl_st20p` plugin and sets the necessary options.
6+
- Finds stream information.
7+
- Sets up decoders.
8+
- Sets up the filter graph.
9+
- Sets up the output format context using the `mtl_st20p` plugin and sets the necessary options.
10+
- Creates the output stream and sets up the encoder.
11+
- Opens the output file.
12+
- Writes the output file header.
13+
- Allocates frames and packets.
14+
- Reads, decodes, filters, encodes, and writes frames.
15+
- Writes the output file trailer.
16+
17+
# -filter_complex:
18+
This option specifies a complex filter graph. It allows you to define a series of filters and how they are connected, including multiple inputs and outputs.
19+
20+
```
21+
The setup_filter_graph() function sets up the filter graph, including buffer source filters, hwupload filters, scale_qsv filters, xstack_qsv filter, and format filters.
22+
```
23+
command from [multiviewer_process.sh](https://github.com/OpenVisualCloud/Intel-Tiber-Broadcast-Suite/blob/main/pipelines/multiviewer_process.sh) :
24+
25+
```
26+
-filter_complex "[0:v]hwupload,scale_qsv=iw/4:ih/2[out0]; \
27+
[1:v]hwupload,scale_qsv=iw/4:ih/2[out1]; \
28+
[2:v]hwupload,scale_qsv=iw/4:ih/2[out2]; \
29+
[3:v]hwupload,scale_qsv=iw/4:ih/2[out3]; \
30+
[4:v]hwupload,scale_qsv=iw/4:ih/2[out4]; \
31+
[5:v]hwupload,scale_qsv=iw/4:ih/2[out5]; \
32+
[6:v]hwupload,scale_qsv=iw/4:ih/2[out6]; \
33+
[7:v]hwupload,scale_qsv=iw/4:ih/2[out7]; \
34+
[out0][out1][out2][out3] \
35+
[out4][out5][out6][out7] \
36+
xstack_qsv=inputs=8:\
37+
layout=0_0|w0_0|0_h0|w0_h0|w0+w1_0|w0+w1+w2_0|w0+w1_h0|w0+w1+w2_h0, \
38+
format=y210le,format=yuv422p10le" \
39+
```
40+
41+
**Input Streams:**
42+
43+
- `[0:v], [1:v], [2:v], [3:v], [4:v], [5:v], [6:v], [7:v]:`
44+
These are the video streams from the input files. The numbers (0, 1, 2, etc.) refer to the input file indices, and v indicates that these are video streams.
45+
46+
**Filters:**
47+
48+
**hwupload:**
49+
50+
- This filter uploads the video frames to the GPU for hardware acceleration. It is used to prepare the frames for further processing by hardware-accelerated filters.
51+
52+
**scale_qsv=iw/4:ih/2:**
53+
54+
- This filter scales the video frames using Intel's Quick Sync Video (QSV) hardware acceleration. The iw/4 and ih/2 specify the new width and height of the frames, which are one-fourth and one-half of the original width and height, respectively.
55+
56+
**Output Labels:**
57+
58+
- `[out0], [out1], [out2], [out3], [out4], [out5], [out6], [out7]:`
59+
These labels are used to name the outputs of the scale_qsv filters. They are used as inputs to the next filter in the chain.
60+
61+
**Stacking Filter:**
62+
63+
- `xstack_qsv=inputs=8:layout=0_0|w0_0|0_h0|w0_h0|w0+w1_0|w0+w1+w2_0|w0+w1_h0|w0+w1+w2_h0:`
64+
This filter stacks multiple video frames together using Intel's QSV hardware acceleration. The inputs=8 specifies that there are 8 input streams. The layout parameter defines the layout of the stacked frames. The layout
65+
66+
**positions the frames in a grid-like pattern:**
67+
- `0_0:` The first frame is placed at the top-left corner.
68+
- `w0_0:` The second frame is placed to the right of the first frame.
69+
- `0_h0:` The third frame is placed below the first frame.
70+
- `w0_h0:` The fourth frame is placed to the right of the third frame.
71+
- `w0+w1_0:` The fifth frame is placed to the right of the second frame.
72+
- `w0+w1+w2_0:` The sixth frame is placed to the right of the fifth frame.
73+
- `w0+w1_h0:` The seventh frame is placed below the fifth frame.
74+
- `w0+w1+w2_h0:` The eighth frame is placed to the right of the seventh frame.
75+
76+
**Format Conversion:**
77+
- `format=y210le:`
78+
This filter converts the pixel format of the video frames to y210le, which is a 10-bit YUV 4:2:2 format with little-endian byte order.
79+
80+
- `format=yuv422p10le:`
81+
This filter converts the pixel format of the video frames to yuv422p10le, which is another 10-bit YUV 4:2:2 format with little-endian byte order.

‎FFmpeg_direct_API/filter_graph_complex.c‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
#include <libavfilter/avfilter.h>
2+
3+
#include "filter_graph_complex.h"
4+
5+
// Function to set up the filter graph
6+
int setup_filter_graph(AVFilterGraph **graph, AVFilterContext **buffersrc_ctx, AVFilterContext **buffersink_ctx, AVCodecContext *dec_ctx[], int num_inputs) {
7+
AVFilterGraph *filter_graph = avfilter_graph_alloc();
8+
if (!filter_graph) {
9+
fprintf(stderr, "Could not allocate filter graph\n");
10+
return -1;
11+
}
12+
13+
char args[512];
14+
AVFilterContext *hwupload_ctx[num_inputs];
15+
AVFilterContext *scale_ctx[num_inputs];
16+
AVFilterContext *xstack_ctx;
17+
const AVFilter *buffersrc = avfilter_get_by_name("buffer");
18+
const AVFilter *buffersink = avfilter_get_by_name("buffersink");
19+
const AVFilter *hwupload = avfilter_get_by_name("hwupload");
20+
const AVFilter *scale_qsv = avfilter_get_by_name("scale_qsv");
21+
const AVFilter *xstack_qsv = avfilter_get_by_name("xstack_qsv");
22+
const AVFilter *format = avfilter_get_by_name("format");
23+
24+
// Create buffer source filters
25+
for (int i = 0; i < num_inputs; i++) {
26+
snprintf(args, sizeof(args),
27+
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
28+
dec_ctx[i]->width, dec_ctx[i]->height, dec_ctx[i]->pix_fmt,
29+
dec_ctx[i]->time_base.num, dec_ctx[i]->time_base.den,
30+
dec_ctx[i]->sample_aspect_ratio.num, dec_ctx[i]->sample_aspect_ratio.den);
31+
if (avfilter_graph_create_filter(&buffersrc_ctx[i], buffersrc, NULL, args, NULL, filter_graph) < 0) {
32+
fprintf(stderr, "Could not create buffer source filter for input %d\n", i);
33+
avfilter_graph_free(&filter_graph);
34+
return -1;
35+
}
36+
}
37+
38+
// Create hwupload and scale filters
39+
for (int i = 0; i < num_inputs; i++) {
40+
if (avfilter_graph_create_filter(&hwupload_ctx[i], hwupload, NULL, NULL, NULL, filter_graph) < 0) {
41+
fprintf(stderr, "Could not create hwupload filter for input %d\n", i);
42+
avfilter_graph_free(&filter_graph);
43+
return -1;
44+
}
45+
snprintf(args, sizeof(args), "iw/4:ih/2");
46+
if (avfilter_graph_create_filter(&scale_ctx[i], scale_qsv, NULL, args, NULL, filter_graph) < 0) {
47+
fprintf(stderr, "Could not create scale_qsv filter for input %d\n", i);
48+
avfilter_graph_free(&filter_graph);
49+
return -1;
50+
}
51+
}
52+
53+
// Create xstack filter
54+
if (avfilter_graph_create_filter(&xstack_ctx, xstack_qsv, NULL, "inputs=8:layout=0_0|w0_0|0_h0|w0_h0|w0+w1_0|w0+w1+w2_0|w0+w1_h0|w0+w1+w2_h0", NULL, filter_graph) < 0) {
55+
fprintf(stderr, "Could not create xstack_qsv filter\n");
56+
avfilter_graph_free(&filter_graph);
57+
return -1;
58+
}
59+
60+
// Create format filters
61+
AVFilterContext *format_ctx1;
62+
if (avfilter_graph_create_filter(&format_ctx1, format, NULL, "y210le", NULL, filter_graph) < 0) {
63+
fprintf(stderr, "Could not create format filter y210le\n");
64+
avfilter_graph_free(&filter_graph);
65+
return -1;
66+
}
67+
AVFilterContext *format_ctx2;
68+
if (avfilter_graph_create_filter(&format_ctx2, format, NULL, "yuv422p10le", NULL, filter_graph) < 0) {
69+
fprintf(stderr, "Could not create format filter yuv422p10le\n");
70+
avfilter_graph_free(&filter_graph);
71+
return -1;
72+
}
73+
74+
// Create buffer sink filter
75+
if (avfilter_graph_create_filter(buffersink_ctx, buffersink, NULL, NULL, NULL, filter_graph) < 0) {
76+
fprintf(stderr, "Could not create buffer sink filter\n");
77+
avfilter_graph_free(&filter_graph);
78+
return -1;
79+
}
80+
81+
// Link filters
82+
for (int i = 0; i < num_inputs; i++) {
83+
if (avfilter_link(buffersrc_ctx[i], 0, hwupload_ctx[i], 0) < 0 ||
84+
avfilter_link(hwupload_ctx[i], 0, scale_ctx[i], 0) < 0 ||
85+
avfilter_link(scale_ctx[i], 0, xstack_ctx, i) < 0) {
86+
fprintf(stderr, "Error linking filters for input %d\n", i);
87+
avfilter_graph_free(&filter_graph);
88+
return -1;
89+
}
90+
}
91+
if (avfilter_link(xstack_ctx, 0, format_ctx1, 0) < 0 ||
92+
avfilter_link(format_ctx1, 0, format_ctx2, 0) < 0 ||
93+
avfilter_link(format_ctx2, 0, *buffersink_ctx, 0) < 0) {
94+
fprintf(stderr, "Error linking xstack and format filters\n");
95+
avfilter_graph_free(&filter_graph);
96+
return -1;
97+
}
98+
99+
// Configure the filter graph
100+
if (avfilter_graph_config(filter_graph, NULL) < 0) {
101+
fprintf(stderr, "Error configuring the filter graph\n");
102+
avfilter_graph_free(&filter_graph);
103+
return -1;
104+
}
105+
106+
*graph = filter_graph;
107+
return 0;
108+
}

‎FFmpeg_direct_API/filter_graph_complex.h‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#ifndef _FILTER_GRAPH_COMPLEX_H_
2+
#define _FILTER_GRAPH_COMPLEX_H_
3+
4+
#include <libavfilter/avfilter.h>
5+
#include <libavfilter/buffersink.h>
6+
#include <libavfilter/buffersrc.h>
7+
8+
// Function to set up the filter graph
9+
int setup_filter_graph(AVFilterGraph **graph, AVFilterContext **buffersrc_ctx,
10+
AVFilterContext **buffersink_ctx, AVCodecContext *dec_ctx[],
11+
int num_inputs);
12+
13+
#endif _FILTER_GRAPH_COMPLEX_H_

0 commit comments

Comments
 (0)