Some of high resolution HDMI video capture hardware

Roman Leave a comment

Recent submissions of DirectShow and Media Foundation capture capabilities brought references to new interesting hardware.

1. Magewell USB Capture HDMI Gen 2 AKA XI100DUSB-HDMI — USB\VID_2935&PID_0001

Reports ability to capture up to 1920×1080 video at 60 frames per second in YUY2 and RGB pixel formats.

2. Inogeni 4K HDMI to USB 3.0 Converter AKA 2A34-INOGENI 4K2USB3 — USB\VID_2997&PID_0004

Goes beyond that and promises resolutions up to 4096×2160@24, 2560×1440@60

Also, not only with Windows 10, but it seems that it can work well with older OSes like Windows 7 as well.

3. Magewell Pro Capture HDMI 4K — PCI\VEN_1CD7&DEV_0012

A PCI version of HDMI capture board with a rich offering of media types and 2K resolutions

4. Some simpler noname HDMICap USB dongle for up to 1920×1080 YUV and RGB capture — USB\VID_EB1A&PID_7595

(I did not have the actual hardware, I just see reported specifications)

D3D11_VIDEO_DECODER_BUFFER_DESC Documentation Quality

Roman Leave a comment

Direct3D 11 DXVA decoding documentation lacks accuracy. The API, sadly, lacks other things too but it is a different story.

D3D11_VIDEO_DECODER_BUFFER_DESC as defined in Windows SDK:

MSDN documentation lost DataSize field, which is – ironically – the most important one along with buffer type.

Related D3D11_1DDI_VIDEO_DECODERR_BUFFER_DESC structure has both fields but the Members section has an obvious copy/paste typo:

The structure and the API itself is presumably not so popular.

Media Foundation’s MFT_MESSAGE_SET_D3D_MANAGER with Frame Rate Converter DSP

Roman Leave a comment

It might look weird why would someone try Direct3D mode with a DSP, which is not supposed to be Direct3D aware, but still. I am omitting the part why I even got to such scenario. The documentation says a few things about MFT_MESSAGE_SET_D3D_MANAGER:

  • This message applies only to video transforms. The client should not send this message unless the MFT returns TRUE for the MF_SA_D3D_AWARE attribute (MF_SA_D3D11_AWARE for Direct3D 11).
  • Do not send this message to an MFT with multiple outputs.
  • An MFT should support this message only if the MFT uses DirectX Video Acceleration for video processing or decoding.
  • If an MFT supports this message, it should also implement the IMFTransform::GetAttributes method and return the value TRUE…
  • If an MFT does not support this message, it should return E_NOTIMPL from ProcessMessage. This is an exception to the general rule that an MFT can return S_OK from any message it ignores.

Frame Rate Converter DSP is a hybrid DMO/MFT, which in turn basically means that its “legacy” DMO upgraded to MFT using specialized wrapper. It is not supposed to be Direct3D aware, not documented as such.

However it could presumably normalize frame rate of Direct3D aware samples by dropping/duplicating samples respectively. It could easily be Direct3D aware since it does not need, in its simplest implementation, to change the data. It is easy to see that the MFT satisfies the other conditions: it is single output video transform.

The MFT correctly and expectedly does not advertise itself as Direct3D aware. It does not have transform attributes.

However, it fails to comply with documented behavior on returning E_NOTIMPL in MFT_MESSAGE_SET_D3D_MANAGER message. The message is defined to be an exception, however DSP implementation seems to be ignoring that. The wrapper could possibly be created even before the exception was introduced in first place.

The DSP does not make an exception, returns success code as if it does handle the message, and does not act as documented.

AMD started offering hardware H.265/HEVC video encoder for Media Foundation

Roman Leave a comment

It should be good news for those interested in hardware assisted video encoding as AMD extends offering in their new hardware and offers H.265 encoder in already well-known form factor: as a Microsoft Media Foundation Transform “AMDh265Encoder”:

# System

* Version: 10.0.14393, Windows 10, VER_SUITE_SINGLEUSERTS, VER_NT_WORKSTATION
* Product: PRODUCT_PROFESSIONAL

[…]

# Display Devices

* AMD Radeon (TM) RX 480
* Instance: PCI\VEN_1002&DEV_67DF&SUBSYS_0B371002&REV_C7\4&2D78AB8F&0&0008
* DEVPKEY_Device_Manufacturer: Advanced Micro Devices, Inc.
* DEVPKEY_Device_DriverVersion: 21.19.137.514

[…]

# Category `MFT_CATEGORY_VIDEO_ENCODER`

[…]

## AMDh265Encoder

15 Attributes:

* MFT_TRANSFORM_CLSID_Attribute: {5FD65104-A924-4835-AB71-09A223E3E37B} (Type VT_CLSID)
* MF_TRANSFORM_FLAGS_Attribute: MFT_ENUM_FLAG_HARDWARE
* MFT_ENUM_HARDWARE_VENDOR_ID_Attribute: VEN_1002 (Type VT_LPWSTR)
* MFT_ENUM_HARDWARE_URL_Attribute: AMDh265Encoder (Type VT_LPWSTR)
* MFT_INPUT_TYPES_Attributes: MFVideoFormat_NV12, MFVideoFormat_ARGB32
* MFT_OUTPUT_TYPES_Attributes: MFVideoFormat_HEVC
* MFT_CODEC_MERIT_Attribute: 8 (Type VT_UI4)
* MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE: 1 (Type VT_UI4)
* MF_TRANSFORM_ASYNC: 1 (Type VT_UI4)
* MF_SA_D3D11_AWARE: 1 (Type VT_UI4)
* MF_SA_D3D_AWARE: 1 (Type VT_UI4)
* MF_TRANSFORM_ASYNC_UNLOCK: 0 (Type VT_UI4)
* MFT_GFX_DRIVER_VERSION_ID_Attribute: 1.2.3.4

This follows Intel’s H.265/HEVC hardware compression offering also available in MFT form factor:

## Intel® Hardware H265 Encoder MFT

12 Attributes:

* MFT_TRANSFORM_CLSID_Attribute: {BC10864D-2B34-408F-912A-102B1B867B6C} (Type VT_CLSID)
* MF_TRANSFORM_FLAGS_Attribute: MFT_ENUM_FLAG_HARDWARE
* MFT_ENUM_HARDWARE_VENDOR_ID_Attribute: VEN_8086 (Type VT_LPWSTR)
* MFT_ENUM_HARDWARE_URL_Attribute: AA243E5D-2F73-48c7-97F7-F6FA17651651 (Type VT_LPWSTR)
* MFT_INPUT_TYPES_Attributes: {3231564E-3961-42AE-BA67-FF47CCC13EED}, MFVideoFormat_NV12, MFVideoFormat_ARGB32
* MFT_OUTPUT_TYPES_Attributes: MFVideoFormat_HEVC
* MFT_CODEC_MERIT_Attribute: 7 (Type VT_UI4)
* MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE: 1 (Type VT_UI4)
* MF_TRANSFORM_ASYNC: 1 (Type VT_UI4)
* MFT_GFX_DRIVER_VERSION_ID_Attribute: 0.0.0.3

How to control IP Video Source JPEG video filter from C# programmatically

Roman Leave a comment

User’s question about Alax.Info IP Video Source DirectShow filter for JPEG/M-JPEG video:

As it stands now, in order to configure the video capture source filter, I have to instantiate it […], set the properties (camera URL, and video dimensions), then save the filter state to a data array which is then compiled into the application. That doesn’t provide a feasible system for user configuration at run time: Telling our customers we need to have the IP address(es) of their camera(s) so we can build a custom version of our software for them isn’t going to be very marketable.

[…]

I looked on the Alax.Info Web site, and didn’t see anything that looks like an API for configuration. I also didn’t see anything inside […] that looks like configuration parameters. I find it hard to believe this isn’t a common requirement, so I suspect I’m overlooking something.

Is there an “easy” way to configure the parameters of the Alax.info video capture filter at run time? Where is it documented?

Is there a structure that describes the state info that is captured [into raw configuration data array]?

Are there any other considerations I’ve overlooked?

C++ and C# sample code (Trac, SVN) appears to be not easy no discover. The code snippets include PlayJpegLocation (C++) and PlayJpegLocationSharp (C#) projects/snippets demonstrating import of IP Video Source COM interfaces via COM and use them from code.

IP Video Source comes with a COM type library which is well suited for import by development environments, specifically by Visual Studio as a reference for .NET projects. To avoid bitness issues, it is recommended to have both 32 and 64 bit versions of IP Video Source installed.

C# integration is straightforward:

First, one needs to add a reference to AlaxInfoIpVideoSource type library, the same way DirectShow.NET is added to the project. The type library is imported and is available via .NET COM Interop at this point (specifically, visible in Object Browser). In code, the definitions are in AlaxInfoIpVideoSource namespace, JpegVideoSourceFilter represents a filter class, with IJpegVideoSourceFilter interface offering filter configuration.

The filter is also compatible with generic COM persistence: it implements standard IPersistStream, IPersistStreamInit, IPersistPropertyBag and other interfaces, its state can be saved to stream or property bad and loaded back.

Intel Quick Sync Video Consumption by Applications

Roman Leave a comment

I wrote a few posts on hardware H.264 encoding (e.g. this and the latest one Applying Hardsubs to H.264 Video with GPU). A blog reader asked a question regarding availability of the mentioned Intel Quick Sync Video support with low end Intel x5-Z8300 CPU.

[…] Intel has advertised that the Cherry Trail CPUs support H264 encoding and / or QSV, but nowhere have I seen a demo of this being used […].
What did you use to encode the video? Is the QSV codec available in the x5-z8300 for possible 720p realtime encoding? I’d like to see this checked in regards to using software like FFmpeg with qsv_h264 -codec and OBS. […]

A picture below explains how applications are consuming Intel’s hardware video compression offering in Windows.

Intel QSV includes hardware implementation of the encoder and corresponding drivers which provide a frontend API to software. This includes a component which integrates the codec with Microsoft’s Media Foundation API. Applications are to choose between interfacing the codec using Windows API – this is the way stock Microsoft applications work, and this is the way I used for video encoding development mentioned on the blog. Other applications prefer to interface through Intel Media SDK, which is an alternate route ending up at the same hardware-backed services.

Intel x5-Z8300 system in question has H.264 video encoding support integrated into Windows API and the services can be consumed without additional Intel runtime and/or development kit. The codec, according to the benchmarks made earlier, is fast enough to handle real-time 720p video encoding nevertheless the device is a budget thing.

Bug in Media Foundation MPEG-4 File Source related to timestamping video frames of a fragmented MP4 file

Roman 4 Comments

Some recent update in Media Foundation platform introduced a new bug related to fragmented MP4 files and H.264 video. The bug shows up consistently with file versions:

  • mfplat.dll – 10.0.14393.351 (rs1_release_inmarket.161014-1755)    15-Oct-16 05:48
  • mfmp4srcsnk.dll – 10.0.14393.351 (rs1_release_inmarket.161014-1755)    15-Oct-16 05:45

The nature of the problem is that MPEG-4 File Source is incorrectly time stamping the data: frame time stamps are incorrect, they seems to be getting wrong durations and increments, then quickly jumps into future… and on playback this leads to unobvious playback freezes. As Media Foundation is used by Windows Media Player, Windows 10 Movies & TV Player, the bug is present there as well.

The original report is on MSDN Forums.

Presumably it is possible to roll certain Windows Update package back, or alternatively one has to wait for Microsoft to fix the problem and deliver a new update deploying the fix.