Skip to content

build: patch FFmpeg so that Direct3D 12 decoding works on AMD and Intel - #330

Merged
devopvoid merged 1 commit into
mainfrom
build/patch-ffmpeg-d3d12va
Oct 5, 2026
Merged

devopvoid merged 1 commit into
mainfrom
build/patch-ffmpeg-d3d12va

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

Summary

FFmpeg 8.1 cannot decode through Direct3D 12 for long on a driver that needs reference-only allocations (AMD, Intel). In libavcodec/d3d12va_decode.c, get_reference_only_resource() takes a second reference-only resource each time the frame pool hands out a texture it already has one for. Nothing frees the extra, and prepare_reference_only_resources() then gives the decoder the older one as the reference. The decoder fails with No space for new Reference frame! after as many pictures as it has reference slots (10 for VP9, 18 for H.264), and before that it can decode against a stale reference. The map is also allocated with max_num_ref + 1 entries while every loop over it stops at max_num_ref.

Since the media player tries Direct3D 12 first (#326), hardware decoding fell back to software for every stream on an AMD Radeon RX 9070 XT, and 8 cases of HardwareDecodingTest failed where hardware is required.

patches/0001-d3d12va-reuse-the-reference-only-resource-of-a-texture.patch makes a texture keep the resource it has and makes the loops cover the whole map.

How the build uses it

  • dependencies/ffmpeg/CMakeLists.txt applies every patches/*.patch, in the order of their names, to the submodule for the length of the FFmpeg build and takes them off again afterwards, whether or not the build worked. The submodule is clean after a build.
  • A patch that does not apply stops the build and says so. That is how a bump of FFmpeg shows that the fix has arrived and the patch can go. If an earlier build was cut short, a patch that is applied already is recognised.
  • The name and SHA-256 of each patch are part of components.txt, so an install built without a patch, or with another version of it, is built again.
  • The CI cache key and restore-key of the FFmpeg cache include patches/*.patch.
  • patches/README.md describes how patches are made and kept.

The patch is applied on every platform, though it only matters on Windows, which means the first CI run rebuilds FFmpeg once on all of them.

The fix is not in FFmpeg upstream as far as I know. It should go there too; I have not done that.

Test plan

  • Default install on the AMD machine rebuilt by itself (built with other components, rebuilding), the patch was applied, and the submodule was clean afterwards
  • Media module suite with -Dwebrtc.test.hardwareDecoding=true, Direct3D 12 first: 76 tests, 0 failures (the 8 HardwareDecodingTest cases that fail on main pass)
  • A standalone decode of a 14315 frame 1080p H.264 stream, and of every test asset, runs to the end on Direct3D 12
  • A second patch that does not apply stops the build with a clear message, and the first one is taken off again
  • A patch that is applied already is recognised, and the submodule is clean after the build

FFmpeg 8.1 cannot decode through Direct3D 12 for long on a driver that needs
reference-only allocations: it takes a second reference-only resource each
time its frame pool hands out a texture it already has one for, nothing
frees the extra, and the decoder fails with "No space for new Reference
frame!" after as many pictures as it has reference slots, 10 for VP9 and 18
for H.264. Since the media player tries Direct3D 12 first, hardware decoding
fell back to software on an AMD Radeon RX 9070 XT for every stream, and
8 tests of HardwareDecodingTest failed where hardware is required.

patches/0001-d3d12va-reuse-the-reference-only-resource-of-a-texture.patch
makes a texture keep the resource it has, and makes the loops over the map
cover all the entries it is allocated with. With it the same 27 tests pass
on that GPU with Direct3D 12 first, and a 14315 frame 1080p H.264 stream
decodes to its end.

The build applies the patches in that directory to the submodule for the
length of the FFmpeg build and takes them off again, so the submodule is
clean afterwards, whether the build worked or not. A patch that does not
apply stops the build, which is how a bump of FFmpeg shows that the fix has
arrived and the patch can go. Their names and hashes are part of
components.txt, and of the key of the CI cache, so that an install built
without a patch is built again.
@devopvoid
devopvoid merged commit 4d205e9 into main Oct 5, 2026
20 of 22 checks passed
@devopvoid
devopvoid deleted the build/patch-ffmpeg-d3d12va branch October 5, 2026 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant