Skip to content

feat: decode HEVC and AV1 in hardware in the media player - #331

Open
devopvoid wants to merge 2 commits into
mainfrom
feat/media-hevc-av1-hw-decode
Open

devopvoid wants to merge 2 commits into
mainfrom
feat/media-hevc-av1-hw-decode

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

Summary

The media player decoded only H.264 and VP9 on the GPU. This adds HEVC and AV1:

  • FFmpeg is built with the hardware decoders of HEVC and AV1: Direct3D 11 and 12 on Windows, NVDEC on Linux, VideoToolbox on macOS.
  • FFmpeg's own AV1 decoder has no software path, so AV1 gets dav1d 1.5.4 as its software decoder. It is a new submodule, built with Meson before FFmpeg into a static library that is linked into libavcodec. Its licence and a notice go into the platform jar. On Windows it is built with the static C runtime, which is the one FFmpeg is built with.
  • FFmpeg prefers libdav1d to its native AV1 decoder, and only the native one has hardware configurations. VideoDecoder therefore picks a decoder that has one when hardware decoding is asked for, and otherwise keeps the preferred one.
  • dav1d's version and options are recorded in components.txt, so changing them rebuilds FFmpeg on its own.
  • Meson and Ninja are new build prerequisites. The CI preparation steps install them; the README and the docs say so.

Testing

On Windows x64 (AMD Radeon RX 9070 XT), with the hardware decoders required (webrtc.test.hardwareDecoding and the new webrtc.test.hardwareAv1Decoding), all 81 tests of the media module pass. HardwareDecodingTest now compares every frame decoded in hardware with the software decoder for HEVC and AV1, at 320x240 and 1080p (four new test assets of 19 to 362 KB).

Not verified

Only windows-x86_64 has been built. The Meson cross files for the other targets (Linux x64/ARM64/ARM32, macOS x86-64 and ARM64, Windows ARM64) are generated from the toolchain files and the variables FFmpeg is configured with, but have not been run. The CI builds are the first to exercise them. 10-bit and other profiles were not tested, nor were streams other than the synthetic test assets.

The player decoded only H.264 and VP9 on the GPU. FFmpeg is now built with the
hardware decoders of HEVC and AV1 as well: Direct3D 11 and 12 on Windows,
NVDEC on Linux, VideoToolbox on macOS.

FFmpeg's own AV1 decoder has no software path, so AV1 gets dav1d as its
software decoder. It is a new submodule (1.5.4), built with Meson before FFmpeg
into a static library that is linked into libavcodec. On Windows that is with
the static C runtime, which is the one FFmpeg is built with. Its version and
options are recorded in components.txt, so changing them rebuilds on its own.
Meson and Ninja are new build prerequisites, installed by the CI preparation
steps. Cross targets get a Meson cross file made from the same variables
FFmpeg is configured with; only windows-x86_64 has been built so far.

FFmpeg prefers libdav1d to its native AV1 decoder, and only the native one has
hardware configurations, so VideoDecoder picks a decoder that has one when
hardware decoding is asked for, and otherwise keeps the preferred one.

The licence of dav1d and a notice go into the platform jar.

Tests compare every frame decoded in hardware with the software decoder for
HEVC and AV1, at 320x240 and 1080p. AV1 has its own property,
webrtc.test.hardwareAv1Decoding, since older GPUs cannot decode it.
Meson finds the compiler by its path inside the Xcode toolchain, which skips
the /usr/bin/cc shim that points SDKROOT at an SDK, so on the macOS ARM64
runner it reported that the compiler cannot compile programs ("library
'System' not found"). The SDK is named for the Meson run now. FFmpeg's own
configure gets it from the shim, so SDKROOT is unset again before it runs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant