Expose `AVFrame.metadata` through `Frame.metadata`

[mockodin]

Summary

FFmpeg filters can write structured results to AVFrame.metadata, but PyAV does not expose that dictionary on AudioFrame or VideoFrame.

This makes metadata-producing filters such as ebur128, ssim, silencedetect, blackdetect, and select incomplete when used through an in-process PyAV filter graph. The filtered frame is returned, but the lavfi.* values attached to it cannot be read from Python.

This was previously requested in #477, which was closed automatically as stale without an implementation.

Reproducer

Tested with PyAV 18.0.0:

from fractions import Fraction

import av
import numpy as np

sample_rate = 48_000
graph = av.filter.Graph()
source = graph.add_abuffer(
    sample_rate=sample_rate,
    format="fltp",
    layout="stereo",
    time_base=Fraction(1, sample_rate),
)
analyzer = graph.add("ebur128", "metadata=1")
sink = graph.add("abuffersink")
source.link_to(analyzer)
analyzer.link_to(sink)
graph.configure()

frame = av.AudioFrame.from_ndarray(
    np.zeros((2, 4_800), dtype=np.float32),
    format="fltp",
    layout="stereo",
)
frame.sample_rate = sample_rate
frame.time_base = Fraction(1, sample_rate)
frame.pts = 0

source.push(frame)
source.push(None)
output = sink.pull()

print("PyAV:", av.__version__)
print("output:", type(output).__name__)
print("has metadata:", hasattr(output, "metadata"))
print("side data:", list(output.side_data))
print("opaque:", output.opaque)

Actual output:

PyAV: 18.0.0
output: AudioFrame
has metadata: False
side data: []
opaque: None

FFmpeg's ebur128=metadata=1 filter writes keys such as lavfi.r128.I, lavfi.r128.LRA, and lavfi.r128.true_peak to the output frame's AVFrame.metadata dictionary. The frame reaches Python, but those values are inaccessible.

Expected behavior

A read-only Frame.metadata property should expose a copy of the underlying dictionary:

print(output.metadata)
# {
#     "lavfi.r128.M": "...",
#     "lavfi.r128.S": "...",
#     "lavfi.r128.I": "...",
#     "lavfi.r128.LRA": "...",
#     ...
# }

Frame.side_data is not an alternative. It exposes AVFrameSideData entries and their metadata, while these filters write directly to AVFrame.metadata.

Proposed API

A minimal implementation appears to be a generic read-only property on the base Frame class:

from cython.cimports.av.utils import avdict_to_dict

@property
def metadata(self):
    return avdict_to_dict(self.ptr.metadata, "utf-8", "strict")

With the corresponding type declaration:

metadata: dict[str, str]

This keeps the binding filter-agnostic and avoids filter-specific parsing, temporary stats files, log capture, or an ffmpeg/ffprobe subprocess.

Would the maintainers accept a PR adding this read-only property and a filter-graph regression test?

References

• Previous request: Python access to AVFrame metadata #477
• FFmpeg AVFrame.metadata: https://ffmpeg.org/doxygen/8.0/structAVFrame.html
• Recent property-preservation work, which preserves but does not expose frame metadata: Preserve frame props after hardware downloads #2321

[mockodin]

Thanks @WyattBlue !