It is really great news that Windows 7 offers quite some goodies for MPEG-4 support right out of the box, but some things appear to be made not so accurate.

The component responsible for video decoding is Microsoft DTV-DVD Video Decoder (also known as Microsoft MPEG-2 Video Decoder), available with DirectShow interface. It offers video decoding capabilities for video resolutions up to 1920×1088 (Full HD), including DXVA version 2 mode. The decoder is a “preferred decoder” and is given a first chance to be taken when it comes for Intelligent Connect to render a video stream.

Technology went ahead though, stepped behind 1920×1088 in particular. Microsoft DTV-DVD Video Decoder DirectShow filter still remains the first take in H.264 video decoding, and what however is taking place – the decoder accepts the format (media type) which is clearly cannot decode, neither according to specification, nor in real life. As soon as pin connection succeeds, DirectShow is not trying to look for another decoder, which can truly decode the feed. Result: failure to decode video.

The problem takes place in a quite surprising way. Given 5 megapixel video (2560×1592, H.264, Baseline profile, from RTSP-enabled IP camera), the decoder advertises 720×480 NV12 output media type. While consuming input data, the decoder outputs nothing and keeps video window black.

It does not issue any filter graph events, no. The filter is persistent and works hard in assumption that a new piece of knowledge will come from Mars and provide new capabilities to start decoding video, or filter’s input will give up and provide a new SPS to lower resolution down to something less complex.

Well then, so is this advanced technology yet to be discovered by science? It appears that no. CoreAVC Video Decoder, applied to the same H.264 video decodes it well and nice:

Update 2011-09-26: According to Microsoft classification, this is a bug on a released product. That is, if anyone is experiencing severe issues as a result of this behavior, one might need to contact Microsoft Support directly in search of a solution.

Actually, as resolutions over Full HD (1920×1080) are not so much common at the moment, the issue side effects are not yet likely. Still moving on with time, I think the issues are likely to come up, without being severe though.

Crime

An application which builds a DirectShow graph unexpectedly started failing with VFW_E_NOT_CONNECTED (0×80040209) error code.

Scene

The problem takes place during DirectShow graph building, yet in stopped state. Specific call which appeared to be giving out the error in first place appears to be EVR input pin’s IPin::ConnectionMediaType, and the problem is also specific to Enhanced video Renderer (Windows 7, but not necessarily only this version).

Investigation

The problem does not appear to be persistent. On the contrary, it is taking place for just a few milliseconds after pin connection. After the problem is gone, it does not seem to ever come up again unless the filter graph is built again from the beginning.

EVR pin connection is always reporting success, so the following error code stating VFW_E_NOT_CONNECTED “The operation cannot be performed because the pins are not connected.” goes against documented behavior, and is thus a bug.

Depending on time between pin connection and media type polling, the call can reach EVR:

• before it starts showing the problem – stage A
• at the time the call fails – stage B
• after the failure time interval, when the call is successful from then on – stage C

Thus, the problem is limited to specific use cases:

• the application should care about media type on EVR input
• unexpected failure takes place when the call reaches in stage B
• also found: the clipping window for the EVR has to belong to a non-primary monitor

If an application keep polling for media type in a loop, the result may be about the following:

UINT nStageA = 0, nStageB = 0, nStageC = 0;
// [...]
for(; ; )
{
    AM_MEDIA_TYPE MediaType;
    ZeroMemory(&MediaType, sizeof MediaType);
    const HRESULT nConnectionMediaTypeResult = pInputPin->ConnectionMediaType(&MediaType);
    if(SUCCEEDED(nConnectionMediaTypeResult))
    {
        if(nStageB)
        {
            nStageC++;
            break;
        } else
            nStageA++;
    } else
    {
        ATLASSERT(nConnectionMediaTypeResult == VFW_E_NOT_CONNECTED);
        nStageB++;
    }
    CoTaskMemFree(MediaType.pbFormat);
}
// [...]
CString sMessage;
sMessage.Format(_T("Bingo!\r\n\r\n") _T("nStageA %d, nStageB %d - 0x%08x, nStageC %d\n"), nStageA, nStageB, nResult, nStageC);
AtlMessageBox(m_hWnd, (LPCTSTR) sMessage, _T("Result"), MB_ICONERROR);

Workaround

An obvious straightforward workaround is to follow EVR connection with a wait for Stage B to pass, or timeout – whichever takes place first.

Also, vote for the bug on Microsoft Connect.

More Details

Video renderer filter are notorious for re-agreeing media type and being fretful as for memory allocators and media types (for a good reason though!). So it makes sense to suggest that the problem takes place when the filter is doing something related, such as it starts background activity immediately after connection in order to discover upstream peer capabilities.

In order to possibly get details on this, it is possible to raise an exception as soon as Stage B is detected and take a look at thread states using a debugger. Indeed, on of the background threads is engaged in EVR reconnection activity:

Yes it does the reconnection, but nevertheless it is expected to do the things undercover and transparently, it still allows a failure on the outer API.

     evr.dll!GetSourceRectFromMediaType() + 0x37 bytes    
     evr.dll!CEVRInputPin::CheckMediaType() + 0x81 bytes    
     evr.dll!CBasePin::ReceiveConnection() + 0x61 bytes    
     evr.dll!CEVRInputPin::ReceiveConnection() + 0x1fc2d bytes    
     quartz.dll!CBasePin::AttemptConnection() - 0x21 bytes    
     quartz.dll!CBasePin::TryMediaTypes() + 0x60 bytes    
     quartz.dll!CBasePin::AgreeMediaType() + 0x54 bytes    
     quartz.dll!CBasePin::Connect() + 0x46 bytes    
     quartz.dll!CFilterGraph::ConnectDirectInternal() + 0x83 bytes    
     quartz.dll!CFilterGraph::ConnectRecursively() + 0x2c bytes    
     quartz.dll!CFilterGraph::ConnectInternal() + 0xde bytes    
     quartz.dll!CFilterGraph::Connect() + 0x17 bytes    
     quartz.dll!CFGControl::WorkerDisplayChanged() + 0xf1 bytes    
     quartz.dll!CFGControl::CGraphWindow::OnReceiveMessage() + 0x2e2a bytes    
>    quartz.dll!WndProc() + 0x3e bytes    
     user32.dll!_InternalCallWinProc@20() + 0x23 bytes    
     user32.dll!_UserCallWinProcCheckWow@32() + 0xb7 bytes    
     user32.dll!_DispatchMessageWorker@8() + 0xed bytes    
     user32.dll!_DispatchMessageW@4() + 0xf bytes    
     quartz.dll!ObjectThread() + 0x65 bytes

A test Visual C++ .NET 2010 application is available from SVN. The code requires a media file, and refers to 352×288 I420.avi, which is included into ZIP file attached to MS Connect Feedback.

A customer complained on video playback problems which had symptoms of software compatibility issue. At certain operation modes software froze leaving the only option to kill process and restart.

I appeared that the system has a third party DLL installed, a new hero name’s – picapmk4.dll. The DLL registers itself as Video for Windows codec and is clearly indicating capabilities it does not really support.

The DLL itself has no indication of its source, no valid version info. Instead it has GPL v2 embedded, XviD logo and about box resource template showing up as “PMK4 Video Codec”.

Looks like crapware, which is hostile to DirectShow environment, with possibly GPL compliance issues.

Recent IP Video Source class/module is not limited to use via registration as a video input/capture/source device. The filter is also registered as a regular filter and can be used programmatically in a regular way: CoCreateInstance, AddFilter etc.

A C++ code snippet below shows how to import interface definition, create an instance of the filter, set it up and start video from a camera:

#include <dshow.h>
#pragma comment(lib, "strmiids.lib")

#import "libid:BDCE8B49-8895-4605-8278-E9A1FBC889AC" no_namespace raw_interfaces_only raw_dispinterfaces named_guids // IpVideoSource

// [...]

inline VOID ATLENSURE_INLINE_SUCCEEDED(HRESULT nResult)
{
    ATLENSURE_SUCCEEDED(nResult);
}

// [...]

CComPtr<IBaseFilter> pBaseFilter;
ATLENSURE_INLINE_SUCCEEDED(pBaseFilter.CoCreateInstance(CLSID_JpegVideoSourceFilter));
CComQIPtr<IJpegVideoSourceFilter> pJpegVideoSourceFilter = pBaseFilter;
ATLENSURE_THROW(pJpegVideoSourceFilter, E_NOINTERFACE);
ATLENSURE_INLINE_SUCCEEDED(pFilterGraph->AddFilter(pBaseFilter, L"Source"));

ATLENSURE_INLINE_SUCCEEDED(pJpegVideoSourceFilter->put_Location(CComBSTR(_T("http://p.viewnetcam.com:60001/nphMotionJpeg?Resolution=640x480&Quality=Standard"))));
ATLENSURE_INLINE_SUCCEEDED(pJpegVideoSourceFilter->put_Width(640));
ATLENSURE_INLINE_SUCCEEDED(pJpegVideoSourceFilter->put_Height(480));

#pragma region Render Output Pin
CComPtr<IEnumPins> pEnumPins;
ATLENSURE_INLINE_SUCCEEDED(pBaseFilter->EnumPins(&pEnumPins));
CComPtr<IPin> pPin;
ATLENSURE_THROW(pEnumPins->Next(1, &pPin, NULL) == S_OK, E_FAIL);
ATLASSERT(pPin);
ATLENSURE_INLINE_SUCCEEDED(pFilterGraph->Render(pPin));
#pragma endregion

This starts M-JPEG video from Panasonic BB-HCM381A camera available to public online at http://p.viewnetcam.com:60001/nphMotionJpeg?Resolution=640×480&Quality=Standard

UPDATE: ATLENSURE_SUCCEEDED macro in the snippet replaced with ATLENSURE_INLINE_SUCCEEDED – see additional post on this.

C#.NET code snippet will look like this (using DirectShowLib):

using DirectShowLib;
using AlaxInfoIpVideoSource;

// [...]

FilterGraph filterGraph = new FilterGraph();
IJpegVideoSourceFilter sourceFilter = Activator.CreateInstance(Type.GetTypeFromCLSID(new Guid("{A8DA2ECB-DEF6-414D-8CE2-E651640DBA4F}"))) as IJpegVideoSourceFilter;
IBaseFilter sourceBaseFilter = sourceFilter as IBaseFilter;
(filterGraph as IFilterGraph2).AddFilter(sourceBaseFilter, "Source");
sourceFilter.Location = @"http://p.viewnetcam.com:60001/nphMotionJpeg?Resolution=640x480&Quality=Standard";
sourceFilter.Width = 640;
sourceFilter.Height = 480;
IPin pin = DsFindPin.ByDirection(sourceBaseFilter, PinDirection.Output, 0);
(filterGraph as IFilterGraph2).Render(pin);
(filterGraph as IMediaControl).Run();
EventCode eventCode;
(filterGraph as IMediaEvent).WaitForCompletion(-1, out eventCode);

Two widely popular applications, which use DirectShow API appeared to be locked in a combat: Google video chat is installing a plugin which registers artificial video capture sources “Google Camera Adapter 0″ and “Google Camera Adapter 1″.

As the application does not announce any integration capabilities and is only using video for internal purposes for in-browser video, the approach with system-wide device registration is, well, questionable.

No need to mention, that devices are not operational: they are giving ERROR_DLL_INIT_FAILED (Error 0x8007045A) as soon as you try to instantiate a filter in a DirectShow video capture enabled application.

In-browser Google Video Chat itself does work and shows video from “real” DirectShow devices, including such artificial as created by IP Video Source tool. Google developers prefer still to take data off the device and as soon as possible leave DirectShow domain, as their graph looks like:

Video Capture Source -> Smart Tee Filter -> Sample Grabber Filter -> Null Renderer Filter

While Google plugin developers might need some interprocess synchronization, as the plugin runs in a child “plugin container” process (plugin-container.exe), introduction of virtual video capture devices is not necessary, but is a plain garbage for the operating system and DirectShow environment.

For some reason, Google developers decided to introduce a special thing: they mirror captured video left-to right when it comes to showing picture capture locally. Well, quite possibly it starts a new era in video conferencing: one will not see a direct copy of video sent remotely, but instead he will be given a mirrored thing.

The curious part is, however, the effect the broken devices introduce to another popular software – Skype.

As Skype is capable of video capturing from a DirectShow device, for video-enabled conferencing, it enumerates the available devices and definitely gets in touch with the aliens from Google planet, which are already known to be broken junk.

The outcome appears to be subject to chances, there are two possible outcomes and only one is such that is more or less good for a user:

• either, Skype shows other available devices and ignores Google adapters (while basically any reasonable DirectShow video capture device would still list those devices – after all they are registered as existing!)
• or, Skype stumbles upon a broken thing, stops enumeration and show no video capture devices available at all! Since Skype does enumeration only once on startup, if it fails to discover devices a user needs to close Skype completely and start it again for another try with video hardware.

It appears that both applications have severe issues.

This does not update the software with new features, but there are a few simple things worth mentioning explicitly.

The first is that virtual DirectShow camera device can be set up with both M-JPEG and JPEG URLs. That is, IP cameras which do not implement M-JPEG, or implement it in a buggy way (there is a *huge* deal of such out there) can still be set up to send video as individual video frames/images as long as they implement JPEG snapshots. This is taking place often at a lower frame rate, but still works.

The driver will automatically detect type of URL (by response from the device) and will choose best access method for the given URL.

Second is that if you are looking for IP Video Source software version, such as to check against available updates, it is here on the UI (right click the caption):

The IP Video Source update provides several improvements to the driver:

• copy/paste feature to backup, restore, or synchronize installed devices between 32-bit and 64-bit versions
• 64-bit version and .MSI
• dynamic video resizing (via Video Resizer DSP)
• Adobe FMLE compatibility

Updates in greater detail follow.

Device Copy/Paste Feature

The video device management window is providing Copy and Paste buttons, which let user transfer device information, including name and settings, through clipboard for various purposes:

• save data in order to be able to restore devices later
• restore devices from saved list, or re-create from a list saved on another machine
• duplicate a device
• synchronize devices between 32-bit and 64-bit versions

The device data is a text, one line per device, lines in comma-separated values (CSV) format.

64-Bit Version

Software is available in both 32-bit and 64-bit versions. While most of multimedia software is still 32-bit, sometimes 64-bit version is really helpful to, for example:

• interface to “Any CPU” .NET code which executes in 64-bit on a 64-bit operating system
• go beyond 32-bit code boundaries, especially in part of multiple devices and virtual address space limit

64-bit version duplicates functionality available in 32-bit version, though 32- and 64-bit modules run in completely separated environments.

Dynamic Video Resizing

The source DirectShow filter is capable of connecting with media types different from original/camera media type in resolution part for decompressed video. The filter is internally taking advantage of Video Resizer DSP, and resizes decompressed video to format of interest.

As Video Resizer DSP is available starting with Windows Vista, the functionality availability is subject to the same conditions.

The feature is absolutely required for Adobe Flash Media Live Encoder software, as it make certain assumptions as for supported resolutions, and completely ignores the information the filter makes available in standard way.

Adobe Flash Media Live Encoder Friendliness

The update make the device much closer to interoperation with Adobe product. The software works around a handful of bugs, including as ridiculous as neglecting to provide correct BI_RGB value in BITMAPINFOHEADER::biCompression structure, yet researchers are still to resolve the remained ones.

FMLE would start perfect and stream video being run under debugger, otherwise in a regular run chances are that Adobe application would internally respond with E_UNEXPECTED failure for unknown reason, stop any encoding without giving a single little notice on the GUI.

After all, the encoding runs perfectly through web based encoder, and the driver is providing a good way to stream to video distribution sites such as http://ustream.tv, http://livestream.com from an IP camera or video encoder.

Download Information

• Download (32-bit): AlaxInfoIpVideoSourceSetup.msi
• Download (64-bit): AlaxInfoIpVideoSourceSetup-x64.msi
• License: This software is free to use

This implements a DirectShow driver/wrapper over a HTTP based JPEG/M-JPEG streamed video, widely available with IP cameras. Once installed, it provides a Start Menu shortcut to manage video capture devices, where a user can add/remove devices. The devices are automatically registered with DriectShow and are available to applications.

The compatibility list includes:

• Windows SDK AmCap Sample (reference)
• VideoLan VLC
• Skype (see below)
• Google Talk Video Chat
• Luxriot (as rather an example as Luxriot has its own generic JPEG/M-JPEG device driver, however this still demonstrates compatibility and interoperability of applications)
• GraphEdit, GraphStudio and similar tools

For a created device it is required to configure:

1. JPEG or M-JPEG resource URL on the device, where the driver will access video feed or sequential snapshots
2. Image/video size on the device

Resource URLs vary from device to device and there is no standard identifier. This has to be looked up for specific model in device API reference or SDK. For JPEG snapshots it is possible to copy image shortcut from web browser and paste address into device settings.

Note that some devices, especially low end IP cameras deliver non-compliant feed which can be decoded either by manufacturer’s ActiveX control or model-specific implementation which this driver does not convert and is unlikely to ever cover. The driver will only work with hardware which honors industry standards.

For example, Stardot NetCam SC cameras make the video feed accessible using /nph-mjpeg.cgi resource, so the full URL will consist of protocol prefix “http://”, optional username and password (see screenshot above), IP address or host name e.g. “demo2.stardotcams.com”, resource identifier on the device “/nph-mjpeg.cgi”.

Hence, http://demo2.stardotcams.com/nph-mjpeg.cgi

It is necessary to provide video size of the footage in advance so that the driver could negotiate content media types without accessing the device. Future versions of the driver might implement auto-detection.

Skype Compatibility

To make the device work with Skype, it is necessary to:

1. Configure the device (using its web interface) to work at resolution 640×480
2. Add/configure device with Alax.Info IP video Source as described above, including setting of 640 by 480 pixel resolution in device configuration

Internals

Software implements and registers video input/capture devices, listed under DirectShow CLSID_VideoInputDeviceCategory category. The applications access devices from the list and treat as a regular device such as web camera.

Windows SDK AmCap sample will create the following DirectShow graph where the source filter implemented by this software:

Internally, software creates a worker filter graph in order to stream data from the device:

To decode video, software uses stock MJPEG Decompressor Filter provided with operating system.

Download Information

• Download (32-bit): AlaxInfoIpVideoSourceSetup.msi
• Download (64-bit): To Be Done/On Request
• License: This software is free to use

EnumerateVcmCodecs uses ICOpen API to enuerate available codecs and prints out information obtained from enumeration. 32-bit and 64-bit versions respectively enumerate codecs available for the platform (lists may vary because the codecs are actually provided by different DLLs, built for respective platform).

szName
  fccType: 0x63646976 (vidc), fccHandler 0x64697663 (cvid)
  dwFlags 0
  dwVersion 0x0, dwVersionICM 0x104
  szDescription ""
  szDriver "iccvid.dll"
szName
  fccType: 0x63646976 (vidc), fccHandler 0x30323469 (i420)
  dwFlags 0
  dwVersion 0x0, dwVersionICM 0x104
  szDescription ""
  szDriver "msh263.drv"
...

A binary [Win32, x64] and Visual C++ .NET 2010 source code are available from SVN.

See also:

• related topic on MSDN Forums

ExportAviResources walks through AVI video clips attached as resources to a binary file and exports them into separate files.

Such clips can be used with Animation Controls for GUI animations. You might want to run the utility against SYSTEM32/SYSWOW64 folders to see if any of stock animations are good for you:

D:\>for %i in (C:\Windows\system32\*.dll) do "..\Utilities\ExportAviResources\x64\Release\ExportAviResources.exe" "%i"

A binary [Win32, x64] and Visual C++ .NET 2010 source code are available from SVN.