Category Archives: Video

American Kestrel Partnership — The Peregrine Fund

As a part of American Kestrel Partnership project, Peregrine Fund put live cameras online to broadcast interior and exterior views of American Kestrel nest boxes in their campus in Boise, Idaho.

One of the nestboxes is not only populated, four eggs have been laid recently and hatching is expected soon. The live picture is made possible by putting IP cameras and connecting them with to Ustream.tv broadcasting service through Adobe FMLE software through my IP Video Source DirectShow driver for IP cameras (the IP camera video feed is streamed 24/7 through a virtual video device picked up by FMLE and re-encoded into uploadable stream forwarded to Ustream online service for further distribution).

For more information, see:

Watch the pet on live cameras below:

North Nestbox – Outside

http://ustre.am/Evbs


Live stream videos at Ustream

North Nestbox – Inside

http://ustre.am/J6jD


Live broadcasting by Ustream

Read more »

So, how many EVRs you can do?

Direct3D based DirectShow video renderers – Video Mixing Renderer 9 and Enhanced Video Renderer – have been notoriously known for consuming resources in a way that you can run at most X simultaneously. There has been no comment published on the topic and questions (e.g. like this: How many VMR 9 can a PC support concurrently) remain unanswered for a long time. Video Mixing Renderer 7 was a good alternative for some time in past until it was cut down to be unable to support hardware scaling (thanks Microsoft!).

The trendy way to render video nowadays is using Enhanced Video Renderer, a Media Foundation subsystem with an interface into DirectShow to take over state of the art video rendering capabilities. So, how many EVRs one can run simultaneously? Chances are that it is less than one could suppose.

The interesting part is that there is no obvious evidence on type of resource running out, which causing next EVR instance to fail to run. And not even run, the failure seem to be coming up at an earlier stage of just connecting pins in stopped state. The failure might be accompanied with errors like E_INVALIDARG, ERROR_FILE_NOT_FOUND, E_UNEXPECTED. The actual limit appear to be loosely correlating to parameters of video output, such as resolution and bitness.

Desperately waiting for clarification, I am sharing the tool to estimate the limit:

  • multiple EVR instances at once, hosted by multiple windows, which can be distributed across multiple monitors
  • choices of resolutions and formats
  • a double click on an individual renderer pops up property page set displaying effective frame rate
  • 32- and 64-bit versions

A binary [Win32x64] and partial Visual C++ .NET 2010 partial source code are available from SVN.

Oops, AMAP_3D_TARGET

Unfortunately, AMAP_3D_TARGET appears to be useless. Internally, surface allocation comes up with the following capabilities for the surface: DDSCAPS_VIDEOMEMORY | DDSCAPS_LOCALVIDMEM | DDSCAPS_OFFSCREENPLAIN | DDSCAPS_3DDEVICE. Sadly, DirectDraw responds (might respond?) with E_NOTIMPL.

While one can advance one step further by removing DDSCAPS_VIDEOMEMORY | DDSCAPS_LOCALVIDMEM, another problem is hit that you are no longer able to create a surface with is both FourCC enabled and in the same time has 3D rendering capabilities. Remove your YUV four character code (such as NV12, YV12, YUY2 you would normally have as codec’s output) or fail miserably with DDERR_INVALIDPIXELFORMAT.

Back to the original problem, the AMAP_3D_TARGET flag failing to work makes it impossible to allocate 3D-enabled DirectDraw surface with DirectShow Video Mixing Renderer. While it might sound as deprecated technology, it is yet a supported Windows SDK component, the most efficient video rendering component, the least hardware sensitive video rendering component, and – the most immediately important – the only way an SDK I am working with can deliver its video overlay.

So, there is no easy way to request a 3D enabled surface through a custom allocator-presenter, with a hook in the IVMRSurfaceAllocator::AllocateSurface updating the allocation flags. It does not make much sense to replace the whole allocator-presenter either: while an earlier DirectShow/Windows SDK provides a decent base for custom allocator-presenter, it is going to eventually hit the same problems mentioned above. And you cannot blit from 3D surface into a plain offscreen surface either without getting E_FAIL.

What appears to still be possible is allocating an addition 3D enabled surface, such as within IVMRImagePresenter::StartPresenting and using it as a replacement surface within IVMRImagePresenter::PresentImage. Having a custom allocator-presenter receive video frame, you can blit the picture into 3D-enabled surface, do your thing and pass the updated VMRPRESENTATIONINFO to default allocator-presenter so that it presents the additional update surface, not the original one.

Read more »

DirectShow SDK at Crime Scene Investigation Service

According to CSI: Crime Scene Investigation, Season 06 Episode 13 (17:04), investigators are using snake scope camera with DirectShow AMCAP tool to present the captured image:

Microsoft DTV-DVD Video Decoder at High Resolutions

It is really great news that Windows 7 offers quite some goodies for MPEG-4 support right out of the box, but some things appear to be made not so accurate.

The component responsible for video decoding is Microsoft DTV-DVD Video Decoder (also known as Microsoft MPEG-2 Video Decoder), available with DirectShow interface. It offers video decoding capabilities for video resolutions up to 1920×1088 (Full HD), including DXVA version 2 mode. The decoder is a “preferred decoder” and is given a first chance to be taken when it comes for Intelligent Connect to render a video stream.

Technology went ahead though, stepped behind 1920×1088 in particular. Microsoft DTV-DVD Video Decoder DirectShow filter still remains the first take in H.264 video decoding, and what however is taking place – the decoder accepts the format (media type) which is clearly cannot decode, neither according to specification, nor in real life. As soon as pin connection succeeds, DirectShow is not trying to look for another decoder, which can truly decode the feed. Result: failure to decode video.

The problem takes place in a quite surprising way. Given 5 megapixel video (2560×1592, H.264, Baseline profile, from RTSP-enabled IP camera), the decoder advertises 720×480 NV12 output media type. While consuming input data, the decoder outputs nothing and keeps video window black.

It does not issue any filter graph events, no. The filter is persistent and works hard in assumption that a new piece of knowledge will come from Mars and provide new capabilities to start decoding video, or filter’s input will give up and provide a new SPS to lower resolution down to something less complex.

Well then, so is this advanced technology yet to be discovered by science? It appears that no. CoreAVC Video Decoder, applied to the same H.264 video decodes it well and nice:

Update 2011-09-26: According to Microsoft classification, this is a bug on a released product. That is, if anyone is experiencing severe issues as a result of this behavior, one might need to contact Microsoft Support directly in search of a solution.

Actually, as resolutions over Full HD (1920×1080) are not so much common at the moment, the issue side effects are not yet likely. Still moving on with time, I think the issues are likely to come up, without being severe though.

A tricky EVR bug was caught up: input pin may falsely report disconnected state

Crime

An application which builds a DirectShow graph unexpectedly started failing with VFW_E_NOT_CONNECTED (0×80040209) error code.

Scene

The problem takes place during DirectShow graph building, yet in stopped state. Specific call which appeared to be giving out the error in first place appears to be EVR input pin’s IPin::ConnectionMediaType, and the problem is also specific to Enhanced video Renderer (Windows 7, but not necessarily only this version).

Investigation

The problem does not appear to be persistent. On the contrary, it is taking place for just a few milliseconds after pin connection. After the problem is gone, it does not seem to ever come up again unless the filter graph is built again from the beginning.

EVR pin connection is always reporting success, so the following error code stating VFW_E_NOT_CONNECTED “The operation cannot be performed because the pins are not connected.” goes against documented behavior, and is thus a bug.

Depending on time between pin connection and media type polling, the call can reach EVR:

  • before it starts showing the problem – stage A
  • at the time the call fails – stage B
  • after the failure time interval, when the call is successful from then on – stage C

Thus, the problem is limited to specific use cases:

  • the application should care about media type on EVR input
  • unexpected failure takes place when the call reaches in stage B
  • also found: the clipping window for the EVR has to belong to a non-primary monitor

If an application keep polling for media type in a loop, the result may be about the following:

UINT nStageA = 0, nStageB = 0, nStageC = 0;
// [...]
for(; ; )
{
    AM_MEDIA_TYPE MediaType;
    ZeroMemory(&MediaType, sizeof MediaType);
    const HRESULT nConnectionMediaTypeResult = pInputPin->ConnectionMediaType(&MediaType);
    if(SUCCEEDED(nConnectionMediaTypeResult))
    {
        if(nStageB)
        {
            nStageC++;
            break;
        } else
            nStageA++;
    } else
    {
        ATLASSERT(nConnectionMediaTypeResult == VFW_E_NOT_CONNECTED);
        nStageB++;
    }
    CoTaskMemFree(MediaType.pbFormat);
}
// [...]
CString sMessage;
sMessage.Format(_T("Bingo!\r\n\r\n") _T("nStageA %d, nStageB %d - 0x%08x, nStageC %d\n"), nStageA, nStageB, nResult, nStageC);
AtlMessageBox(m_hWnd, (LPCTSTR) sMessage, _T("Result"), MB_ICONERROR);

Workaround

An obvious straightforward workaround is to follow EVR connection with a wait for Stage B to pass, or timeout – whichever takes place first.

Also, vote for the bug on Microsoft Connect.

More Details

Video renderer filter are notorious for re-agreeing media type and being fretful as for memory allocators and media types (for a good reason though!). So it makes sense to suggest that the problem takes place when the filter is doing something related, such as it starts background activity immediately after connection in order to discover upstream peer capabilities.

In order to possibly get details on this, it is possible to raise an exception as soon as Stage B is detected and take a look at thread states using a debugger. Indeed, on of the background threads is engaged in EVR reconnection activity:

Yes it does the reconnection, but nevertheless it is expected to do the things undercover and transparently, it still allows a failure on the outer API.

     evr.dll!GetSourceRectFromMediaType() + 0x37 bytes    
     evr.dll!CEVRInputPin::CheckMediaType() + 0x81 bytes    
     evr.dll!CBasePin::ReceiveConnection() + 0x61 bytes    
     evr.dll!CEVRInputPin::ReceiveConnection() + 0x1fc2d bytes    
     quartz.dll!CBasePin::AttemptConnection() - 0x21 bytes    
     quartz.dll!CBasePin::TryMediaTypes() + 0x60 bytes    
     quartz.dll!CBasePin::AgreeMediaType() + 0x54 bytes    
     quartz.dll!CBasePin::Connect() + 0x46 bytes    
     quartz.dll!CFilterGraph::ConnectDirectInternal() + 0x83 bytes    
     quartz.dll!CFilterGraph::ConnectRecursively() + 0x2c bytes    
     quartz.dll!CFilterGraph::ConnectInternal() + 0xde bytes    
     quartz.dll!CFilterGraph::Connect() + 0x17 bytes    
     quartz.dll!CFGControl::WorkerDisplayChanged() + 0xf1 bytes    
     quartz.dll!CFGControl::CGraphWindow::OnReceiveMessage() + 0x2e2a bytes    
>    quartz.dll!WndProc() + 0x3e bytes    
     user32.dll!_InternalCallWinProc@20() + 0x23 bytes    
     user32.dll!_UserCallWinProcCheckWow@32() + 0xb7 bytes    
     user32.dll!_DispatchMessageWorker@8() + 0xed bytes    
     user32.dll!_DispatchMessageW@4() + 0xf bytes    
     quartz.dll!ObjectThread() + 0x65 bytes

A test Visual C++ .NET 2010 application is available from SVN. The code requires a media file, and refers to 352×288 I420.avi, which is included into ZIP file attached to MS Connect Feedback.

Another VFW/DirectShow Pain in the Ass: picapmk4.dll

A customer complained on video playback problems which had symptoms of software compatibility issue. At certain operation modes software froze leaving the only option to kill process and restart.

I appeared that the system has a third party DLL installed, a new hero name’s – picapmk4.dll. The DLL registers itself as Video for Windows codec and is clearly indicating capabilities it does not really support.

The DLL itself has no indication of its source, no valid version info. Instead it has GPL v2 embedded, XviD logo and about box resource template showing up as “PMK4 Video Codec”.

Looks like crapware, which is hostile to DirectShow environment, with possibly GPL compliance issues.