perf: cut the processor time of hardware decoding on Windows - #326
Merged
Merged
Conversation
devopvoid
force-pushed
the
perf/d3d12-hardware-decoding
branch
from
October 4, 2026 18:52
d37a15e to
8378072
Compare
devopvoid
added this pull request to stack #327
October 4, 2026 19:10
For every picture the hardware path dropped the system-memory frame the picture was read back into, and allocated a new I420 frame for the conversion. At 4K that is two fresh allocations of 12 MB per picture, and a fresh allocation costs more to fault in than the copy into it. The read-back frame now stays between pictures, with a new buffer only when the size or the pixel format changes (the transfer copies no more than the smaller of the two frames), and the I420 frames come from a buffer pool, so a picture lands in memory that is already mapped. The conversion of other formats in software uses the pool as well. CPU cycles on the decoding thread per picture, on an NVIDIA GeForce RTX 3080 through Direct3D 11: the read-back went from 11.0 to 5.3 million at 4K and from 3.3 to 1.8 million at 1080p, the conversion from 10.4 to 5.7 and from 2.2 to 1.3.
FFmpeg 8.1 has Direct3D 12 hwaccels for H.264 and VP9. The Windows build now
enables them (h264_d3d12va, vp9_d3d12va), and the player tries a Direct3D 12
device before the Direct3D 11 one it used. A machine without Windows 10 2004
or a driver that decodes through Direct3D 12 has no such device, and goes on
with Direct3D 11 as before. The pictures are the same: every test of the
hardware path compares them with software decoding.
Reading a picture back is what costs processor time on this path, and
Direct3D 12 does it for about half of what Direct3D 11 does. Process CPU per
frame in the player on an NVIDIA GeForce RTX 3080, median of four runs of
steady playback, one machine and synthetic media:
software Direct3D 11 Direct3D 12
1080p H.264 3.6 ms 4.2 ms 2.1 ms
4K H.264 6.6 ms 12.4 ms 5.4 ms
Direct3D 11 used more processor time than software at 4K, Direct3D 12 less.
The component list changed, so the next build rebuilds FFmpeg on Windows.
devopvoid
force-pushed
the
perf/d3d12-hardware-decoding
branch
from
October 4, 2026 19:10
8378072 to
ffc55dd
Compare
This was referenced Oct 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #325 (which is on #323): the base is its branch, so the diff is these two commits only. Retarget to
mainonce those are merged.Hardware decoding in the media player saved little processor time on Windows, and at 4K cost more than software. Profiling the decoding thread (cycles per picture,
QueryThreadCycleTime) showed why, and led to two changes.1. Reuse the buffers of hardware-decoded frames. For every picture the hardware path dropped the system-memory frame it read the picture back into, and allocated a new I420 frame for the conversion: two fresh allocations of 12 MB per picture at 4K, and a fresh allocation costs more to fault in than the copy into it.
2. Direct3D 12 first on Windows. FFmpeg 8.1 has
h264_d3d12vaandvp9_d3d12va. The Windows build enables them andDeviceTypes()tries a Direct3D 12 device before Direct3D 11. A machine without Windows 10 2004 or a driver that decodes through Direct3D 12 goes on with Direct3D 11 as before. The component list changed, so the next Windows build rebuilds FFmpeg.Process CPU per frame in the player on an NVIDIA GeForce RTX 3080, median of four runs of steady playback:
Direct3D 12 uses about 40% less than software at 1080p and 18% less at 4K, and about half of what Direct3D 11 does. The 7 to 8 times of the Apple M2 does not carry over. One machine, synthetic media and large run-to-run noise (the four runs of a setting differ by up to a factor of 2), so these are indications, not a promise; the guide says so and gives these figures.
The pictures are unchanged: all 67 tests of
webrtc-java-mediapass withwebrtc.test.hardwareDecodingset, including the 1080p, 4K, 1366x768 and fallback cases of #325 that compare every frame with software decoding. A log line confirmed the Direct3D 12 device was the one created; it is not part of the change.Not done:
-Xcheck:jnion the media module.--enable-d3d12vamakes FFmpeg's configure fail where the Windows SDK has nod3d12video.h. The CI runners most likely have it, but I have not checked, so watch the Windows jobs.