Skip to content

Conversation

@benedikt-grl
Copy link
Contributor

FFmpeg allows accessing the quantization parameters for a few video codecs via side data. This PR adds support for accessing the VideoEncParams and VideoBlockParams.
Something similar was proposed in #779, but has not been implemented yet.

After setting stream.codec_context.options = {"export_side_data": "venc_params"}, the user can do something like this:

if sd.type == av.sidedata.sidedata.Type.VIDEO_ENC_PARAMS:
    for block_idx in range(sd.nb_blocks):
        b = sd.block_params(block_idx)
        # b has properties src_x, src_y, w, h, delta_qp

I also added a convenience method sd.qp_map() that arranges the quantization parameters in a 2-D grid. The quantization parameters are calculated as sd.qp + b.delta_qp, analogously to ff_qp_table_extract.

Full example:

import av
import matplotlib.pyplot as plt
import numpy as np


def draw_qp_map(qp_map):
    # Determine figure size
    map_height, map_width = qp_map.shape
    width_px = map_width * 16
    height_px = map_height * 16
    font_size = 7

    # Create figure
    fig, ax = plt.subplots(1, 1, figsize=(width_px / 50, height_px / 50))

    # Display QP map
    ax.imshow(qp_map, vmin=0, vmax=51)

    # Annotate each macroblock with its quantization parameter
    for y in range(map_height):
        for x in range(map_width):
            ax.text(x, y, qp_map[y, x], horizontalalignment="center", verticalalignment="center", color="black", fontsize=font_size)

    # Show macroblock coordinates
    ax.set_xticks(np.arange(map_width))
    ax.set_yticks(np.arange(map_height))

    # Show macroblock indices at all sides
    ax.tick_params(
        top=True,
        bottom=True,
        left=True,
        right=True,
        labeltop=True,
        labelbottom=True,
        labelright=True,
    )

    return fig, ax


video_filepath = "video.mp4"
container = av.open(video_filepath)

stream = container.streams.video[0]
stream.codec_context.options = {"export_side_data": "venc_params"}

frame_iter = container.decode(video=0)

# Select frame by index
frame_idx = 0

# Skip ahead to the frame of interest
for i in range(frame_idx + 1):
    frame = next(frame_iter)

qp_map = None
for sd in frame.side_data:
    if sd.type == av.sidedata.sidedata.Type.VIDEO_ENC_PARAMS:
        qp_map = sd.qp_map()

fig, ax = draw_qp_map(qp_map)
plt.show()

@WyattBlue WyattBlue added the needs tests This PR needs a test label Jan 6, 2026
@WyattBlue WyattBlue removed the needs tests This PR needs a test label Jan 7, 2026
@WyattBlue WyattBlue merged commit a14f730 into PyAV-Org:main Jan 7, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants