{"id":1996,"date":"2020-01-18T12:34:00","date_gmt":"2020-01-18T10:34:00","guid":{"rendered":"https:\/\/alax.info\/blog\/?p=1996"},"modified":"2020-01-19T11:47:46","modified_gmt":"2020-01-19T09:47:46","slug":"on-efficiency-of-hardware-assisted-jpeg-decoding-amd-mft-mjpeg-decoder","status":"publish","type":"post","link":"https:\/\/alax.info\/blog\/1996","title":{"rendered":"On efficiency of hardware-assisted JPEG decoding (AMD MFT MJPEG Decoder)"},"content":{"rendered":"\n

The previous post<\/a> was focusing on problems with the hardware MFT decoder provided as a part of video driver package. This time I am going to mention some data about how the inefficiency affects performance of video capture using a high frame rate 260 FPS camera as a test stand. Apparently the effect is better visible with high frame rates because CPU and GPU hardware is fast enough already to process less complicated signal.<\/p>\n\n\n\n

There is already some interest from AMD end<\/a> (deserves a separate post why this is exceptional on its own), and some bug fixes are already under the way.<\/p>\n\n\n\n

The performance problem is less visible because the decoder is overall performing without fatal issues and provides expected output: no failures, error codes, no deadlocks, neither CPU or GPU engine is peaked out, so things are more or less fine at first glance… The test application uses Media Foundation and Source Reader API<\/a> to read textures in hardware MFT enabled mode and discards the textures just printing out the frame rate.<\/p>\n\n\n\n

AMD MFT MJPEG Decoder<\/h2>\n\n\n\n
C:\\...\\MjpgCameraReader\\bin\\x64\\Release>MjpgCameraReader.exe\n Using camera HD USB Camera\n Using adapter Radeon RX 570 Series\n Using video capture format 640x360@260.004 MFVideoFormat_MJPG\n Using hardware decoder MFT AMD MFT MJPEG Decoder\n Using video frame format 640x384@260.004 MFVideoFormat_YUY2\n 72.500 video samples per second captured\n 134.000 video samples per second captured\n 135.000 video samples per second captured\n 134.500 video samples per second captured\n<\/strong> 135.500 video samples per second captured\n 134.000 video samples per second captured\n 134.000 video samples per second captured\n 135.000 video samples per second captured\n 134.500 video samples per second captured\n 133.500 video samples per second captured\n 134.000 video samples per second captured<\/pre>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
With no sign of hitting a bottleneck the reader process produces ~134 FPS from the video capture device.<\/p>\n\n\n\n
Alax.Info MJPG Video Decoder for AMD Hardware<\/h2>\n\n\n\n
My replacement for hardware decoder MFT is doing the decoding of the same signal, and, generally, shares a lot with AMD’s own decoder: both MFTs are built on top of Advanced Media Framework (AMF) SDK<\/a>. Driver package installs runtime for this SDK and installs a decoder MFT which is linked against a copy of the runtime (according to AMD representative, the static link copy shares the same codebase). <\/p>\n\n\n\n
C:\\...\\MjpgCameraReader\\bin\\x64\\Release>MjpgCameraReader.exe\n Using camera HD USB Camera\n Using adapter Radeon RX 570 Series\n Using video capture format 640x360@260.004 MFVideoFormat_MJPG\n Using substitute decoder Alax.Info MJPG Video Decoder for AMD Hardware\n Using video frame format 640x360@260.004 MFVideoFormat_YUY2\n 74.000 video samples per second captured\n 261.000 video samples per second captured\n 261.000 video samples per second captured\n 261.000 video samples per second captured\n<\/strong> 261.000 video samples per second captured\n 260.500 video samples per second captured\n 261.000 video samples per second captured\n 261.000 video samples per second captured\n 261.000 video samples per second captured\n 261.000 video samples per second captured\n 260.500 video samples per second captured<\/pre>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
Similar CPU and GPU utilization levels with higher frame rate. Actually, with the expected frame rate because it is the rate the camera is supposed to operate at.<\/p>\n\n\n\n\n\n\n\n
1280×720@120 Mode<\/h2>\n\n\n\n
Interestingly, at lower FPS mode the AMD MFT threading issues are present, and, more to that the MFT exhibits two other issues (one of them is “just ignore” one per AMD comment). At the same time video capture rate is no longer reduced: the horsepower of the hardware is hiding the implementation inefficiency.<\/p>\n\n\n\n
 Using camera HD USB Camera\n Using adapter Radeon RX 570 Series\n Using video capture format 1280x720@120.000 MFVideoFormat_MJPG\n Using hardware decoder MFT AMD MFT MJPEG Decoder\n Using video frame format 1280x736@120.000 MFVideoFormat_YUY2\n 18.500 video samples per second captured\n 120.000 video samples per second captured\n 120.000 video samples per second captured\n 120.000 video samples per second captured\n<\/strong> 120.000 video samples per second captured\n 120.000 video samples per second captured\n 120.000 video samples per second captured\n 120.000 video samples per second captured\n 120.000 video samples per second captured<\/pre>\n\n\n\n
Intel Hardware M-JPEG Decoder MFT<\/h2>\n\n\n\n
AMD is not the only one GPU vendor out there and my development system is equipped with integrated GPU from Intel as well, so why not give it a try?<\/p>\n\n\n\n
To AMD defence, Intel’s decoder is exhibiting a subpar performance:<\/p>\n\n\n\n
C:\\...\\MjpgCameraReader\\bin\\x64\\Release>MjpgCameraReader.exe\n Using camera HD USB Camera\n Using adapter Intel(R) UHD Graphics 630\n Using video capture format 640x360@260.004 MFVideoFormat_MJPG\n Using hardware decoder MFT Intel\u00d0\u00a0\u00d1\u2022 Hardware M-JPEG Decoder MFT\n Using video frame format 640x368@260.004 MFVideoFormat_YUY2\n 24.000 video samples per second captured\n 63.500 video samples per second captured\n 63.500 video samples per second captured\n 64.000 video samples per second captured\n<\/strong> 63.500 video samples per second captured\n 63.000 video samples per second captured\n 63.500 video samples per second captured\n 62.000 video samples per second captured\n 63.500 video samples per second captured\n 64.000 video samples per second captured\n 63.500 video samples per second captured<\/pre>\n\n\n\n
\"\"<\/figure>\n\n\n\n
At lower relative utilization levels and, again, without hitting any bottleneck visibly, the capture rate is reduced.<\/p>\n\n\n\n
And this happens even without the threading problem I could at least see in the AMD’s case.<\/p>\n\n\n\n
120 FPS mode is doing good:<\/p>\n\n\n\n
C:\\...\\MjpgCameraReader\\bin\\x64\\Release>MjpgCameraReader.exe\n Using camera HD USB Camera\n Using adapter Intel(R) UHD Graphics 630\n Using video capture format 1280x720@120.000 MFVideoFormat_MJPG\n Using hardware decoder MFT Intel\u00d0\u00be Hardware M-JPEG Decoder MFT\n Using video frame format 1280x720@120.000 MFVideoFormat_YUY2\n 77.000 video samples per second captured\n 119.000 video samples per second captured\n 120.000 video samples per second captured\n 121.000 video samples per second captured\n<\/strong> 119.000 video samples per second captured\n 121.000 video samples per second captured\n 120.000 video samples per second captured\n 120.000 video samples per second captured\n 120.500 video samples per second captured\n 119.500 video samples per second captured\n 120.000 video samples per second captured<\/pre>\n\n\n\n
That is, there is an obvious performance issue in Intel’s implementation since they fail to process lower resolution signal at original rate and even at rate they are showing for higher resolution signal!<\/p>\n\n\n\n
So does 1920×1080@60:<\/p>\n\n\n\n
C:\\...\\MjpgCameraReader\\bin\\x64\\Release>MjpgCameraReader.exe\n Using camera HD USB Camera\n Using adapter Intel(R) UHD Graphics 630\n Using video capture format 1920x1080@60.000 MFVideoFormat_MJPG\n Using hardware decoder MFT Intel\u00d0\u00be Hardware M-JPEG Decoder MFT\n Using video frame format 1920x1088@60.000 MFVideoFormat_YUY2\n 49.500 video samples per second captured\n 60.500 video samples per second captured\n 59.500 video samples per second captured\n 60.000 video samples per second captured\n<\/strong> 60.000 video samples per second captured\n 60.000 video samples per second captured\n 60.000 video samples per second captured\n 60.000 video samples per second captured\n 60.000 video samples per second captured\n 60.000 video samples per second captured\n 60.000 video samples per second captured<\/pre>\n\n\n\n
In closing<\/h2>\n\n\n\n
Bottom line is that hardware ASICs are generally good, but the quality of software MFT layer is not something GPU vendors care much of.<\/p>\n\n\n\n
The application below does the testing on first available GPU and it assumes you have a video capture compatible to Media Foundation API. The application uses highest frame rate MJPG format of the camera and uses a hardware decoder MFT associated with the GPU.<\/p>\n\n\n\n
One more thing to mention is that video capture takes place through so called Microsoft Windows Camera Frame Server (FrameServer) Service, notorious<\/a> and not documented. Frame Server virtualizes video capture device adding processing overhead and cross-process synchronization. <\/p>\n\n\n\n
Some time later I will compare performance of capturing around Frame Server and around Media Foundation default implementation of video capture device proxy. I expect though that there is no visible performance difference as those are, eventually, done well.<\/p>\n\n\n\n
Download links<\/h3>\n\n\n\n
Binaries:<\/p>\n\n\n\n