Intel’s hardware H.264 encoder MFT fails to change encoding settings as encoding goes

If I recall correctly, Intel was the first vendor to supply H.264 hardware video encoder as a Media Foundation Transform (since Windows 7) and overall Intel Quick Sync Video (QSV) was the earliest widely available implementation of hardware assisted encoding. However one of the aspects of such encoding seems to be missing: update of encoder parameters during encoding session.

The task itself is pretty typical and outside Media Foundation is handled for example by libx264:

/* x264_encoder_reconfig:
 * various parameters from x264_param_t are copied.
 * this takes effect immediately, on whichever frame is encoded next;
 * due to delay, this may not be the next frame passed to encoder_encode.
 * if the change should apply to some particular frame, use x264_picture_t->param instead.
 * returns 0 on success, negative on parameter validation error.
 * not all parameters can be changed; see the actual function for a detailed breakdown.
 *
 * since not all parameters can be changed, moving from preset to preset may not always
 * fully copy all relevant parameters, but should still work usably in practice. however,
 * more so than for other presets, many of the speed shortcuts used in ultrafast cannot be
 * switched out of; using reconfig to switch between ultrafast and other presets is not
 * recommended without a more fine-grained breakdown of parameters to take this into account. */

Even though Intel’s comment on Media Foundation interface for Intel QSV is such that MFT offers a subset of the available functionality:

These hardware MFT (HMFT) are provided and distributed as part of our graphics drivers. These provide Quick sync (hardware acceleration) support on platforms. Quick Sync Video H.264 Encoder MFT is a fixed function implementation, hence there is limited flexibility to make any changes to the pipeline. Yes, HMFT distributed via graphic driver on win7 includes dx9 support, Win8/8.1 dx11 and Win10 dx12.

Update of setting as encoding goes have been a must for some time. Not only MSDN has a mention of this, e.g. here in Codec Properties detail:

In Windows 8, this property can be set at any time during encoding. Changes are applied starting at the next input frame.

But also it looks like Microsoft implements a related feature in their software video encoder without properly documenting it AND Nvidia follows that in their MFT implementation. This makes me think that certain materials exist such as reference H.264 MFT implementation, which vendors use as a sample and Intel in particular does not include this in their code.

More technical detail in my question on Intel Developer Zone site: Certified Hardware Encoder for H.264 video encoding ignores quality changes:

I am trying to identify how Intel’s hardware Media Foundation Transform for H.264 video encoding is handling quality changes during video encoding process.

Some introductory material follows that shows that what is expected to work does not really work.

Microsoft defines H.264 encoder MFT behavior here in H.264 Video Encoder and provides a software only version of the codec in their operating systems. It is expected that vendors like Intel provide their own hardware codecs as “Certified Hardware Encoders”: such encoders follow the described behavior and operating system gives them a priority as to more efficient implementation.

Intel does supply Certified Hardware Encoder for H.264 implemented in e.g mfx_mft_h264ve_64.dll (just an example, names might vary, I am referring specifically to version 7.16.10.20 in this post, the version I have as current) and this DLL along with the dependencies is Intel’s implementation of such Certified Hardware Encoder.

Certified Hardware Encoder is expected to implement certain subset of functionality defined by Microsoft:

If a certified hardware encoder is present, it will generally be used instead of the inbox system encoder for Media Foundation related scenarios.
Certified encoders are required to support a certain set of ICodecAPI properties and can optionally support another set of properties.
The certification process should guarantee that the required properties are properly supported and, if an optional property is supported, that it is also properly supported.

MSDN follows that by mentioning required properties:

The following Windows 8 and Windows 8.1 ICodecAPI properties are required:

CODECAPI_AVEncCommonRateControlMode
CODECAPI_AVEncCommonQuality
CODECAPI_AVEncCommonQualityVsSpeed

Even though Intel’s encoder does mostly implement the properties, there is one aspect that does not really seem to work. From the same MSDN page:

┬áCODECAPI_AVEncCommonQuality […]

In Windows 8, this property can be set at any time during encoding. Changes are applied starting at the next input frame.

The behavior that I see (in Windows 10 Fall Creators Update) is that Intel’s Certified Hardware Encoder is ignoring quality changes during encoding. That is, the behavior I see is that encoder keeps encoding applying the initially set properties.

More to that Intel’s implementation does not replicate Microsoft’s own codec behavior via IMFTransform.ProcessEvent call which Intel seems to be stubbing with a no-op.

[…]

It is worth saying that Microsoft’s own implementation (apart from lacking proper documentation even though the feature was announced for as long ago as for Windows 8!) does not look consistent and well thought of. Let us have a look at the simplest scenario: eAVEncCommonRateControlMode_Quality. The documentation suggests that it is:

Quality-based VBR encoding. The encoder selects the bit rate to match a specified quality level. To specify the quality level, set the AVEncCommonQuality property.

Same time, CODECAPI_AVEncVideoEncodeQP property is described as a one step lower level relative for the CODECAPI_AVEncCommonQuality property:

This property applies when the rate control mode is eAVEncCommonRateControlMode_Quality.

This property configures the same encoding setting as AVEncCommonQuality. However, AVEncVideoEncodeQP enables the application to specify the value of QP directly. If both properties are set, AVEncVideoEncodeQP overrides.

Then why the heck dynamic quality update checks AVEncCommonQuality but not AVEncVideoEncodeQP? If the latter is a priority then apparently both should have been checked, and this is not what is happening. When Intel is going to fix their MFT, they have a good chance to make it right.

Leave a Reply