build: patch FFmpeg so that Direct3D 12 decoding works on AMD and Intel - #330
Merged
Merged
Conversation
FFmpeg 8.1 cannot decode through Direct3D 12 for long on a driver that needs reference-only allocations: it takes a second reference-only resource each time its frame pool hands out a texture it already has one for, nothing frees the extra, and the decoder fails with "No space for new Reference frame!" after as many pictures as it has reference slots, 10 for VP9 and 18 for H.264. Since the media player tries Direct3D 12 first, hardware decoding fell back to software on an AMD Radeon RX 9070 XT for every stream, and 8 tests of HardwareDecodingTest failed where hardware is required. patches/0001-d3d12va-reuse-the-reference-only-resource-of-a-texture.patch makes a texture keep the resource it has, and makes the loops over the map cover all the entries it is allocated with. With it the same 27 tests pass on that GPU with Direct3D 12 first, and a 14315 frame 1080p H.264 stream decodes to its end. The build applies the patches in that directory to the submodule for the length of the FFmpeg build and takes them off again, so the submodule is clean afterwards, whether the build worked or not. A patch that does not apply stops the build, which is how a bump of FFmpeg shows that the fix has arrived and the patch can go. Their names and hashes are part of components.txt, and of the key of the CI cache, so that an install built without a patch is built again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FFmpeg 8.1 cannot decode through Direct3D 12 for long on a driver that needs reference-only allocations (AMD, Intel). In
libavcodec/d3d12va_decode.c,get_reference_only_resource()takes a second reference-only resource each time the frame pool hands out a texture it already has one for. Nothing frees the extra, andprepare_reference_only_resources()then gives the decoder the older one as the reference. The decoder fails withNo space for new Reference frame!after as many pictures as it has reference slots (10 for VP9, 18 for H.264), and before that it can decode against a stale reference. The map is also allocated withmax_num_ref + 1entries while every loop over it stops atmax_num_ref.Since the media player tries Direct3D 12 first (#326), hardware decoding fell back to software for every stream on an AMD Radeon RX 9070 XT, and 8 cases of
HardwareDecodingTestfailed where hardware is required.patches/0001-d3d12va-reuse-the-reference-only-resource-of-a-texture.patchmakes a texture keep the resource it has and makes the loops cover the whole map.How the build uses it
dependencies/ffmpeg/CMakeLists.txtapplies everypatches/*.patch, in the order of their names, to the submodule for the length of the FFmpeg build and takes them off again afterwards, whether or not the build worked. The submodule is clean after a build.components.txt, so an install built without a patch, or with another version of it, is built again.patches/*.patch.patches/README.mddescribes how patches are made and kept.The patch is applied on every platform, though it only matters on Windows, which means the first CI run rebuilds FFmpeg once on all of them.
The fix is not in FFmpeg upstream as far as I know. It should go there too; I have not done that.
Test plan
built with other components, rebuilding), the patch was applied, and the submodule was clean afterwards-Dwebrtc.test.hardwareDecoding=true, Direct3D 12 first: 76 tests, 0 failures (the 8HardwareDecodingTestcases that fail onmainpass)