Effects of IMFVideoProcessorControl2::EnableHardwareEffects

Posted on by Roman

IMFVideoProcessorControl2::EnableHardwareEffects method:

Enables effects that were implemented with IDirectXVideoProcessor::VideoProcessorBlt.

[…] Specifies whether effects are to be enabled. TRUE specifies to enable effects. FALSE specifies to disable effects.

All right, it is apparently not IDirectXVideoProcessor and MSDN link behind the identifier takes one to correct Direct3D 11 API method: ID3D11VideoContext::VideoProcessorBlt.

Worse news is that having the effects “enabled”, the transform (the whole thing belongs to Media Foundation’s Swiss knife of conversion [with just one blade and no scissors though] – Video Processing MFT) fails to deliver proper output and produces green black fill instead of proper image.

Or, perhaps, this counts as a hardware effect.

No conversion with MF_CONNECT_ALLOW_CONVERTER

Posted on by Roman

Microsoft Media Foundation Media Session API topology resolution is way less clear compared to DirectShow. The API takes away a part of component connection process and makes it less transparent to API consumer. Then, while DirectShow Intelligent Connect is use scenario agnostic,  Media Foundation Media Session apparently targets playback scenarios, and its topology resolution process is tuned respectively.

MF_CONNECT_ALLOW_DECODER

Add a decoder transform upstream upstream from this node, if needed to complete the connection. The numeric value of this flag includes the MF_CONNECT_ALLOW_CONVERTER flag. Therefore, setting the MF_CONNECT_ALLOW_DECODER flag sets the MF_CONNECT_ALLOW_CONVERTER flag as well.

[…] If this attribute is not set, the default value is MF_CONNECT_ALLOW_DECODER.

Well, that’s double upstream and suggests that the thing is impressively reliable. However it is not.

In a non-playback topology, if a direct connection is impossible and required conversion is not typical for playback, MF_CONNECT_ALLOW_CONVERTER flag seems to be not helpful for topology resolution.

Apparently, Microsoft does have suitable code, esp. used in Sink Writer API, however it does not seem to be available in any form other than a packet deal with Sink Writer object and its own limitations. Media Session API does not implement this (non-playback, that is) style of topology resolution and node connection. Transcode API too has the necessary topology resolution code, but again it comes with its own constraints making it useless unless you want to do something really simple.

 

Reference Signal Source: audio as Media Foundation source

Posted on by Roman

Reference signal source for DirectShow in its video part already received Media Foundation Source interface earlier.

This time, the update implements a separate Media Foundation source for audio. MfGenerate2 sample code gives an idea on how to initialize the source:

using namespace AlaxInfoDirectShowReferenceSource;
CComPtr<IAudioMediaSource> pSource;
__C(pSource.CoCreateInstance(__uuidof(AudioMediaSource)));
__C(pSource->SetMediaType(NULL, g_nSampleRate, g_nChannelCount, g_nBitDepth));
__C(pSource->put_Duration((DOUBLE) g_nDuration));
CComPtr<IMFMediaSource> pAudioMediaSource = pSource;

The source can be given specific format using Media Foundation stream descriptor’s media type handler, or set up via private COM interface.

The source can be given sampling rate, channel count (all channels receive the same signal) and bit depth for PCM audio formats (8, 16..32), or 32-bit IEEE floating point format.

Video and audio streams can also be combined into aggregate source (video+audio) to produce a multi-track output. The MfGenerate2 sample shows the approach as well:

__D(pVideoMediaSource || pAudioMediaSource, E_UNNAMED);
if(pVideoMediaSource && pAudioMediaSource)
{
    CComPtr<IMFCollection> pCollection;
    __C(MFCreateCollection(&pCollection));
    __C(pCollection->AddElement(pVideoMediaSource));
    __C(pCollection->AddElement(pAudioMediaSource));
    __C(MFCreateAggregateSource(pCollection, &pMediaSource));
} else
    pMediaSource = pVideoMediaSource ? pVideoMediaSource : pAudioMediaSource;
_A(pMediaSource);

The sample project is capable to produce output of this kind:

Download links

IMFAttributes::CopyAllItems freeze on copying to self

Posted on by Roman

An attempt to copy Media Foundation attribute collection to itself results in a deadlock. Well, it’s not a good idea and a practical one to do a nonsense like this, but the implementation should be resistant to such use either, esp. avoiding the unexpected deadlock.

#include "stdafx.h"
#include <mfapi.h>

#pragma comment(lib, "mfplat.lib")
#pragma comment(lib, "mfuuid.lib")

int main()
{
    _ATLTRY
    {
        ATLENSURE_SUCCEEDED(MFStartup(MF_VERSION));
        CComPtr<IMFAttributes> pAttributes;
        ATLENSURE_SUCCEEDED(MFCreateAttributes(&pAttributes, 1));
        ATLENSURE_SUCCEEDED(pAttributes->SetGUID(MF_MT_SUBTYPE, GUID_NULL));
        ATLENSURE_SUCCEEDED(pAttributes->CopyAllItems(pAttributes)); // <<--- Freeze
    }
    _ATLCATCHALL()
    {
    }
    return 0;
}

The freeze takes place around SRW locks, that is presumably the implementation attempts to lock the data for reading and then once again for writing immediately afterwards.

    ntdll.dll!_NtWaitForAlertByThreadId@8()    Unknown
    ntdll.dll!RtlAcquireSRWLockExclusive()  Unknown
    mfplat.dll!CMFAttributesImpl<struct IMFAttributes,class CMFSRWLock>::DeleteAllItems(void)   Unknown
    mfplat.dll!CMFAttributesImpl<struct IMFAttributes,class CMFSRWLock>::_CloneAllAttributes(struct IMFAttributes *)    Unknown
    mfplat.dll!CMFAttributesImpl<struct IMFAttributes,class CMFSRWLock>::CopyAllItems(struct IMFAttributes *)   Unknown
>   MfSample01.exe!main() Line 15   C++

Some of high resolution HDMI video capture hardware

Posted on by Roman

Recent submissions of DirectShow and Media Foundation capture capabilities brought references to new interesting hardware.

1. Magewell USB Capture HDMI Gen 2 AKA XI100DUSB-HDMI — USB\VID_2935&PID_0001

Reports ability to capture up to 1920×1080 video at 60 frames per second in YUY2 and RGB pixel formats.

2. Inogeni 4K HDMI to USB 3.0 Converter AKA 2A34-INOGENI 4K2USB3 — USB\VID_2997&PID_0004

Goes beyond that and promises resolutions up to 4096×2160@24, 2560×1440@60

Also, not only with Windows 10, but it seems that it can work well with older OSes like Windows 7 as well.

3. Magewell Pro Capture HDMI 4K — PCI\VEN_1CD7&DEV_0012

A PCI version of HDMI capture board with a rich offering of media types and 2K resolutions

4. Some simpler noname HDMICap USB dongle for up to 1920×1080 YUV and RGB capture — USB\VID_EB1A&PID_7595

(I did not have the actual hardware, I just see reported specifications)

D3D11_VIDEO_DECODER_BUFFER_DESC Documentation Quality

Posted on by Roman

Direct3D 11 DXVA decoding documentation lacks accuracy. The API, sadly, lacks other things too but it is a different story.

D3D11_VIDEO_DECODER_BUFFER_DESC as defined in Windows SDK:

MSDN documentation lost DataSize field, which is – ironically – the most important one along with buffer type.

Related D3D11_1DDI_VIDEO_DECODERR_BUFFER_DESC structure has both fields but the Members section has an obvious copy/paste typo:

The structure and the API itself is presumably not so popular.