Greetings from H.265 / HEVC Video Decoder Media Foundation Transform

H.265 / HEVC Video Decoder Media Foundation has been around for a while, but using Media Foundation overall one step off straightforward basic paths is like walking a minefield.

A twenty-liner below hits memory access violation inside IMFTransform::GetInputStreamInfo “Exception thrown at 0x6D1E71E7 (hevcdecoder.dll) in MfHevcDecoder01.exe: 0xC0000005: Access violation reading location 0x00000000.”:

#include "stdafx.h"
#include <mfapi.h>
#include <mftransform.h>
#include <wmcodecdsp.h>
#include <atlbase.h>
#include <atlcom.h>

#pragma comment(lib, "mfplat.lib")
#pragma comment(lib, "mfuuid.lib")
#pragma comment(lib, "wmcodecdspuuid.lib")

int main()
{
    ATLVERIFY(SUCCEEDED(CoInitialize(NULL)));
    ATLVERIFY(SUCCEEDED(MFStartup(MF_VERSION)));
    CComPtr<IMFTransform> pTransform;
#if 1
    static const MFT_REGISTER_TYPE_INFO InputTypeInformation = { MFMediaType_Video, MFVideoFormat_HEVC };
    IMFActivate** ppActivates;
    UINT32 nActivateCount = 0;
    ATLVERIFY(SUCCEEDED(MFTEnumEx(MFT_CATEGORY_VIDEO_DECODER, 0, &InputTypeInformation, NULL, &ppActivates, &nActivateCount)));
    ATLASSERT(nActivateCount > 0);
    ATLVERIFY(SUCCEEDED(ppActivates[0]->ActivateObject(__uuidof(IMFTransform), (VOID**) &pTransform)));
#else
    ATLVERIFY(SUCCEEDED(pTransform.CoCreateInstance(CLSID_CMSH265EncoderMFT)));
#endif
    MFT_INPUT_STREAM_INFO InputInformation;
    ATLVERIFY(SUCCEEDED(pTransform->GetInputStreamInfo(0, &InputInformation)));
    return 0;
}

Interestingly, alternative path around IMFActivate (see #if above) seems to be working fine.

 

AMD started offering hardware H.265/HEVC video encoder for Media Foundation

It should be good news for those interested in hardware assisted video encoding as AMD extends offering in their new hardware and offers H.265 encoder in already well-known form factor: as a Microsoft Media Foundation Transform “AMDh265Encoder”:

# System

* Version: 10.0.14393, Windows 10, VER_SUITE_SINGLEUSERTS, VER_NT_WORKSTATION
* Product: PRODUCT_PROFESSIONAL

[…]

# Display Devices

* AMD Radeon (TM) RX 480
* Instance: PCI\VEN_1002&DEV_67DF&SUBSYS_0B371002&REV_C7\4&2D78AB8F&0&0008
* DEVPKEY_Device_Manufacturer: Advanced Micro Devices, Inc.
* DEVPKEY_Device_DriverVersion: 21.19.137.514

[…]

# Category `MFT_CATEGORY_VIDEO_ENCODER`

[…]

## AMDh265Encoder

15 Attributes:

* MFT_TRANSFORM_CLSID_Attribute: {5FD65104-A924-4835-AB71-09A223E3E37B} (Type VT_CLSID)
* MF_TRANSFORM_FLAGS_Attribute: MFT_ENUM_FLAG_HARDWARE
* MFT_ENUM_HARDWARE_VENDOR_ID_Attribute: VEN_1002 (Type VT_LPWSTR)
* MFT_ENUM_HARDWARE_URL_Attribute: AMDh265Encoder (Type VT_LPWSTR)
* MFT_INPUT_TYPES_Attributes: MFVideoFormat_NV12, MFVideoFormat_ARGB32
* MFT_OUTPUT_TYPES_Attributes: MFVideoFormat_HEVC
* MFT_CODEC_MERIT_Attribute: 8 (Type VT_UI4)
* MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE: 1 (Type VT_UI4)
* MF_TRANSFORM_ASYNC: 1 (Type VT_UI4)
* MF_SA_D3D11_AWARE: 1 (Type VT_UI4)
* MF_SA_D3D_AWARE: 1 (Type VT_UI4)
* MF_TRANSFORM_ASYNC_UNLOCK: 0 (Type VT_UI4)
* MFT_GFX_DRIVER_VERSION_ID_Attribute: 1.2.3.4

This follows Intel’s H.265/HEVC hardware compression offering also available in MFT form factor:

## Intel® Hardware H265 Encoder MFT

12 Attributes:

* MFT_TRANSFORM_CLSID_Attribute: {BC10864D-2B34-408F-912A-102B1B867B6C} (Type VT_CLSID)
* MF_TRANSFORM_FLAGS_Attribute: MFT_ENUM_FLAG_HARDWARE
* MFT_ENUM_HARDWARE_VENDOR_ID_Attribute: VEN_8086 (Type VT_LPWSTR)
* MFT_ENUM_HARDWARE_URL_Attribute: AA243E5D-2F73-48c7-97F7-F6FA17651651 (Type VT_LPWSTR)
* MFT_INPUT_TYPES_Attributes: {3231564E-3961-42AE-BA67-FF47CCC13EED}, MFVideoFormat_NV12, MFVideoFormat_ARGB32
* MFT_OUTPUT_TYPES_Attributes: MFVideoFormat_HEVC
* MFT_CODEC_MERIT_Attribute: 7 (Type VT_UI4)
* MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE: 1 (Type VT_UI4)
* MF_TRANSFORM_ASYNC: 1 (Type VT_UI4)
* MFT_GFX_DRIVER_VERSION_ID_Attribute: 0.0.0.3

MediaFoundationVideoEncoderTransforms: Detecting support for hardware H.264 video encoders

H.264 (MPEG-4 Part 10 also known as MPEG-4 AVC) video encoding was never packaged into DirectShow filter by Microsoft and instead they offered Media Foundation Transform (MFT) for the job: H.264 Video Encoder. Further development gave us third party video encoders also packaged as MFTs. Hardware backed MFTs became a part of video hardware drivers and nevertheless are part of Media Foundation API, can also be very well used standalone.

Detection of H.264 encoding capabilities? MediaFoundationVideoEncoderTransforms is here to help.

MediaFoundationVideoEncoderTransforms UI

H.264 options include:

  • H264 Encoder MFT
  • Intel® Quick Sync Video H.264 Encoder MFT
  • NVIDIA H.264 Encoder MFT
  • AMDh264Encoder

H.265 encoding will apparently be (already is) using the same encoder packaging.

The tools enumerates the transforms and provides details, similar to Enumerating Media Foundation Transforms (MFTs) application (source code available).

# System

[…]

# Display Devices <<– Same as in Device Manager

* AMD Radeon R7 200 Series
* Instance: PCI\VEN_1002&DEV_6610&SUBSYS_22BF1458&REV_00\4&2DB3ECDA&0&0008
* DEVPKEY_Device_Manufacturer: Advanced Micro Devices, Inc.
* DEVPKEY_Device_DriverVersion: 15.201.1151.1008
* Intel(R) HD Graphics 530 <<– Video adapter Intel QSV is available through (if you don’t see Intel video, then maybe it needs to be turned on in BIOS)
* Instance: PCI\VEN_8086&DEV_1912&SUBSYS_D0001458&REV_06\3&11583659&0&10
* DEVPKEY_Device_Manufacturer: Intel Corporation
* DEVPKEY_Device_DriverVersion: 20.19.15.4331

# Category `MFT_CATEGORY_VIDEO_ENCODER`

NOTE: Detail is limited to H.264 video encoders.

## Intel® Quick Sync Video H.264 Encoder MFT <<– The guy that does Intel QSV encoding

[…]

## H264 Encoder MFT <<– Stock Microsoft software H.264 implementation, available since Windows 7

[…]

(Other encoders like those from AMD, Nvidia will also be listed)

## Intel® Hardware H265 Encoder MFT <<– Intel also provides now H.265 hardware encoding option

[…]

## H265 Encoder MFT

[…]

Download links

Follow up: mixed parallel H.264 encoding, Intel® Quick Sync Video H.264 Encoder MFT + NVIDIA H.264 Encoder MFT

A scenario which was dropped out from previous post is mixed simultaneous encoding using both hardware encoders. Rationale: Intel QSV encoder might exist as a “free” capability of the motherboard (provided with onboard video adapter), The other one might be available with the video adapter intentionally plugged in (including for other reasons, such as to power dual monitor system etc).

From this standpoint, it might be interesting if one can benefit from using of both encoders.

Intel QSV Filter Graph

Nvidia NVEVC Filter Graph

Two filter graphs are set to produce 60 seconds of 1080p60 video as soon as possible, and are started simultaneously. The chart below show completion time, side by side with those for runs of one and two sessions of each encoder separately.

Completion Times: Intel® Quick Sync Video H.264 Encoder MFT + NVIDIA H.264 Encoder MFT

Informational: in single stream runs CPU load was around 30%, two session runs – around 50%, of which the part that synthesizes and converts the video to compatible MFT input format took 5-6% of CPU time overall. Or, if computed against 60 seconds of CPU time of eight core CPU, the synthesis-and-conversion itself consumed <4% CPU time for one stream, and <7% for dual stream runs.

 

Enumerating Media Foundation Transforms (MFTs)

Matthew van Eerde already made a similar wrapper over MFTEnumEx in How to enumerate Media Foundation transforms on your system, and this one extends it with enumeration of attributes, also listing them in human friendly way.
This sort of code should perhaps have been in Media Foundation SDK Samples, however we have what we have.

Media Foundation Transforms (MFTs) – they are registered and accessed through the registry, being available for enumeration with and without qualifying criteria. Some of the transforms are dual, DMO/MFT, some are MFT only which make their useful functionality not available directly for DirectShow pipeline. Luckily, the interface is similar to those of DMOs and making it reasonably possible to wrap one into another. Comparison of MFTs and DMOs shows how the two form factors compare one to the other.

Enumeration tool/utility shows availability of registered MFTs in the system. In Windows 7. For example, the output in Windows 7 workstation in provided below.

The output is a good cheat sheet for seeing support of media types in Windows components.

Download links:

Continue reading →