Published by Roman on 13 May 2010

MediaTools: Two samples to capture M-JPEG video into JPEG files and to play JPEG files back

I added two new simple samples for the MediaTools DirectShow filters to demonstrate how to capture M-JPEG video feed, esp. from an IP camera, and write the video frames into sequence of JPEG files. The other sample takes a directory on the input and plays the images back as video. If you are working on certain transformation filter, it is an easy way to make a reference feed and use it for debugging purposes.

The filters behind that empower the sample are described in another post.

RenderHttpMjpegVideoIntoFiles01 sample takes an URL on the input to generate image/jpeg JPEG or multipart/x-mixed-replace M-JPEG stream. For example, it might be http://demo1.stardotcams.com/nph-mjpeg.cgi feed from a demo StarDot Technologies IP camera.

The application will create a new directory to write files into, and will save each new video frame received into new JPEG file.

Z:\MediaTools\Samples\RenderHttpMjpegVideoIntoFiles01\Release>RenderHttpMjpegVideoIntoFiles01.exe http://demo1.stardotcams.com/nph-mjpeg.cgi
URL: http://demo1.stardotcams.com/nph-mjpeg.cgi
Writing to directory: Z:\MediaTools\Samples\RenderHttpMjpegVideoIntoFiles01\Release\2010-05-12 22-07-37
Event: nCode EC_CLOCK_CHANGED 0xD, nParameter1 0x00000000, nParameter2 0x00000000
Event: nCode EC_PAUSED 0xE, nResult 0x00000000, nParameter2 0x00000000
[...]
^C

The application will generate the files and convert media sample time stamps into file time.

RenderHttpMjpegVideoIntoFiles01 Sample Output

The DirectShow graph that implements the operation is the following:

RenderHttpMjpegVideoIntoFiles01 Filter Graph

The other sample RenderJpegFiles01 takes a directory path to look for JPEG files, e.g. generated by previous sample, and pushes them into DirectShow graph as a video feed. File times will be converted [back] to media sample times.

Continue Reading »

Published by Roman on 25 Mar 2010

ProcessSnapshot: Create process minidump for port-mortem debugging

ProcessSnapshot is a utility to take a snapshot of process call stacks, and the snapshot taken is written into a human friendly text file.

ProcessSnapshot is taking process minidump files

Additionally to this, the utility has been given a capability to create process minidump files, on user request. The minidump files can be used with debugger to analyze the context of the process using feature rich debug environment, esp. Microsoft Visual Studio. To create a minidump for a process, check a corresponding box and press “Take a Dump” button. A file named “<process-image-name> – <date> <time>.dmp” will be created in the directory of the utility executable.

See also:

A binary [Win32, x64] and partial Visual C++ .NET 2008 source code are available from SVN.

Published by Roman on 09 Oct 2009

How to use windowless Video Mixing Renderer Filter to show video fullscreen

The question is being asked from time to time. Everyone knows what is full screen video. Video renderers implement full screen capability since long ago through their IVideoWindow::put_FullScreenMode property, and even Filter Graph Manager exposes its own IVideoWindow interface to forward calls to filter’s implementation of IVideoWindow interface.

However, for Video Mixing Renderers, version 7 and 9, the preferred and recommended mode is windowless, where no IVideoWindow interface is available.

Note The IVMRWindowlessControl or IVMRWindowlessControl9 interface is now preferred over IVideoWindow. For more information, see Using Windowless Mode.

So in order to implement full screen mode it takes the application to attach Video Mixing Renderer filter to a private frameless window, to its entire client area and expand the window to entire monitor area.

The sample project FullScreenWindowlessVmrSample01 is illustrating this technique.

FullScreenWindowlessVmrSample01 Main Dialog

Continue Reading »

Published by Roman on 21 Sep 2009

How to overlay a bitmap on top of video with Video Mixing Renderer (VMR-9)

A 100-lines code snippet which illustrates how a bitmap is overlaid over displayed video with Video Mixing Renderer 9 Filter using IVMRMixerBitmap9 interface. A video clip is played (default is Windows clock.avi, but you can replace it with your longer one to see overlay is really in a loop).

http://code.assembla.com/…/VmrMixerBitmapSample01/…

VMR9AlphaBitmap AlphaBitmap;
ZeroMemory(&AlphaBitmap, sizeof AlphaBitmap);
AlphaBitmap.dwFlags = VMR9AlphaBitmap_hDC;
AlphaBitmap.hdc = Dc;
AlphaBitmap.rSrc = CRect(0, 0, 32, 32);
AlphaBitmap.rDest.left = (FLOAT) 0.75;
AlphaBitmap.rDest.top = (FLOAT) 0.75;
AlphaBitmap.rDest.right = (FLOAT) 0.95;
AlphaBitmap.rDest.bottom = (FLOAT) 0.95;
AlphaBitmap.fAlpha = 0.75;
const HRESULT nSetAlphaBitmapResult = pVmrMixerBitmap->SetAlphaBitmap(&AlphaBitmap);
ATLENSURE_SUCCEEDED(nSetAlphaBitmapResult);

With a low FPS clip like clock.avi it is clear that the overlaid image is only updated with the next “main” video frame.

IVMRMixerBitmap9 Usage Sample

Visual C++ .NET 2008 source code is available from SVN, release binary included.

Published by Roman on 22 Jun 2009

DirectShow Filter Graph Spy on Vista

I have been receiving comments that Filter Graph Spy tool does not work with Microsoft Vista operating system. I never had a moment to check until recently, and this time I realized that it really does not work. I am definitely aware of dramatic changes introduced with this operating system, and in particular UAC feature, virtualization and changes in security. No wonder this was the first guess that security was the cause, however the investigation showed there was a trail of issues underneath…

Investigation details deserve a separate post, while this one briefly outlines the issues and also accompany the repository update with a version compatible with Vista OS.

First of all, COM registration of the DLL (which definitely requires privilege elevation) succeeded on Vista. This means the registration procedure did not encounter any errors on the way, or poorly written code ignored the problem. It appeared that the source of the problem was CoTreatAsClass API, which failed to do the requested action, however returned status code indicating successful operation. This definitely looks like a bug and further comments on this particular behavior are expected to appear on Windows Applications Security MSDN forum, where I opened a topic on the matter.

With a COM TreatAs feature activated, the class’s behavior to instantiate instead of DirectShow’s CLSID_FilterGraph is restored, and in particular the DLL generates FilterGraphSpy.log log file on filter graph activity. Note that log file location is OS dependent (due to Vista’s permissions and file system virtualization):

  • pre-Vista OS: root of syste drive, typically C:\
  • starting Vista, administrator with elevated privileges: CSIDL_COMMON_APPDATA, typically C:\ProgramData (note this directory is hidden by deafult)
  • starting Vista, without elevated administrator privileges: CSIDL_LOCAL_APPDATA, typically C:\Users\$(UserName)\AppData\Local (note that AppData directory is hidden by deafult)

Still even with the log file generated and indicating activation of the spy, it was unable to connect to remote graph through the running object table (ROT). It appeared that the ROT entires are there where expected, it was OK to get an object from ROT and the problem came from QueryInterface code:

CComPtr<IUnknown> pFilterGraphUnknown;
ATLENSURE_SUCCEEDED(pRunningObjectTable->GetObject(pMoniker, &pFilterGraphUnknown));
CComQIPtr<IMyFilterGraph> pFilterGraph = pFilterGraphUnknown; // E_NOINTERFACE

The call reached the original object but COM subsystem was unable to marshal the interface through apartments to enable interprocess communication on it. The reason for this is absence of PSFactoryBuffer class (CLSID {92A3A302-DA7C-4A1F-BA7E-1802BB5D2D02}), which provides proxy/stub pairs for marshaling well known DirectShow interfaces in the Vista’s version of quartz.dll. As mentioned by Microsoft’s Mike Wasson, this class was moved from quartz.dll into Vista SDK’s proppage.dll, so in order to obtain connectivity to remote DirectShow graphs starting Vista, one needs to install this DLL with Windows SDK, or otherwise have it registered with the operating system.

Also note that DirectShow-enabled applications that have their filter graphs published on ROT will need an utility application such as GraphEdit started with the same permissions (elevated or not) in order to be able to access ROT entires.

To sum everything up, to install Alax.Info DirectShow Filter Graph Spy on Vista:

  • get the latest FilterGraphSpy.dll
  • regsvr32 FilterGraphSpy.dll on target system from administrative command prompt, with elevated privileges (note you should have a log file FilterGraphSpy.log generated in CSIDL_COMMON_APPDATA directory, with a few lines indicating registration success)
  • get Windows SDK and make $(WindowsSDK)\Bin\proppage.dll file registered on target system (also administrative regsvr32), note that it is necessary to restart DirectShow-enabled applications and GraphEdit after DLL registration to get graphs visible through ROT
  • mind the log file directories with FilterGraphSpy.log file

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

Published by Roman on 07 Feb 2009

More on DirectShow Filter Graph Spy

Having DirectShow Filter Graph Spy installed in the system, I noticed a new and weird effect that when Media Player Classic plays a list of files and each MP3 file being currently decoded shows decoder’s system tray icon, a switch to a new file leaves an old icon in place and adds a new icon for a new decoder.  I thought this might be also a decoder’s problem which I also recently updated to a new version, but then I noticed other artifacts, with other software and again with Media Player Classic, that if I drop a new file onto the player and it starts playing it, old file is still being played as if player just releases all referenced to the graph and never calls IMediaControl::Stop to force stop of playback.

It was clear at that point that it is related to filter graph spy and the problem needs a fix. The problem definitely looked like the spy leave outstanding references to the filter graph and when the application releases graph and it is supposed to be automatically destroyed, it is still being alive and possibly, such as in case of Media Player Classic, even running.

The first clue was that being published to running object table (ROT), the object gets more than one additional reference on it and as such it never gets removed from ROT until application terminates as ROT keeps a reference to the graph and there is noone to remove it. By original design spy published filter graph on ROT and removed one reference immediately after that to compensate the ROT’s reference (one only) making it a weak reference. So when all references are released from the filter graph but ROT reference, the graph gets destroyed and in FinalRelease class on the COM object we re-add the reference back and automatically remove the object from ROT as well. The whole scheme works in assumption that ROT adds one reference only, if there are more then the object is referenced by the ROT forever.

Debugging however showed that ROT adds a single reference and there is no problem from this side. However I immediately noticed  another weird behavior in the same method FinalConstruct. An underlying (real) filter graph object, i.e. CFilterGraph from quartz.dll, is adding a reference on the outer COM object when being queried for a private interface from implicit (also known as main, raw) IUnknown when created as aggregated object. And it is not exactly the behavior one would expect from aggregated object because it basically break proper reference counting. I realize that aggregating is a rather rare technique and DirectShow COM base is one of the very first COM bases at all which then stopped being developed, so perhaps this problem was not identified on time and later left unfixed at all, so in present situation there should rather be a workaround.

It is also worth mentioning that there are two scenarios involved. My CSpy object creates DirectShow CFilterGraph as an aggregated object. And CSpy in its turn may be created as aggregated too. For example, Windows Media Player creates (as most applications in this universe do) filter graph object the regular way, as a standalone COM object. However it appears that Media Player Classic creates filter graph as aggregated, for whatever it needs it for. Because of the discovered problem inner CFilterGraph adds an outstanding reference on the top level “controlling IUnknown”, which is either CSpy’s private IUnkown in case of Windows Media Player, or higher level IUnknown in case of Media Player Classic.

So the proper workaround is to release unwanted references from controlling unknown rather than self:

CComPtr<IUnknown> pUnknown;
ATLENSURE_SUCCEEDED(pClassFactory->CreateInstance(pControllingUnknown, __uuidof(IUnknown), (VOID**) &pUnknown));
// NOTE: DirectShow FilterGraph is incorrectly implementing COM aggregation adding outer reference to interfaces queried from private IUnknown
CComQIPtr<IFilterGraph2> pFilterGraph2 = pUnknown;
ATLENSURE_THROW(pFilterGraph2, E_NOINTERFACE);
pFilterGraph2.p->Release(); // <<--------------------------

It looked like the problem is finally worked around but in aggregated creation scenario it brought another problem up. Since instantiation of inner CFilterGraph takes place in FinalConstruct and CSpy is also being instantiated as aggregated, it appears that outer COM object is sensible to manipulation with its reference counter. CFilterGraph added a reference it should not, CSpy released it in compensation but on this early stage the compensating Release might zero reference counter and might initiate unexpected object destruction. It is not necessarily this way as it depends on outer COM object base, but it seems that Media Player Classic does not do DECLARE_PROTECT_FINAL_CONSTRUCT or its equavalent and things go the worst scenario.

CSpy is an inner object and we don’t have any external instance to hold a temporary reference for us while outer COM object is being created, so a workround for this is to temporarily hold a circular reference to an outer object to let it complete its instanatiation and pass another reference higher up to the controlling application. At which point we are going to be safe to release the temporary reference and normalize reference counters.

if(m_bIsAggregated)
{
    pControllingUnknown.p->AddRef();
    const ULONG nReferenceCount = pControllingUnknown.p->Release();
    if(nReferenceCount == 1)
        m_pTemporaryUnknown = pControllingUnknown;
}

The only thing is left is to release the temporary reference early enough, but we have a filter graph, don’t we? What is typically done with a filter graph, adding a filter right? And it is a controlling application which has its own reference, who adds filters so we are safe to release temporary reference in IFilterGraph::AddFilter or IGraphBuilder::AddSourceFilter (actually it would not hurt to add this in other methods as well):

STDMETHOD(AddFilter)(IBaseFilter* pFilter, LPCWSTR pszName) throw()
{
    _Z4(atlTraceCOM, 4, _T("pszName \"%s\"\n"), CString(pszName));
    ReleaseTemporaryUnknown(); // <<--------------------------
    return m_pInnerFilterGraph2->AddFilter(pFilter, pszName);
}

Additionally, the spy also implements now IMediaControl interface and traces calls to log file, it is convenient.

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

reg-FilterGraphSpy.dll.bat and unreg-FilterGraphSpy.dll.bat files in SVN are convenient batch files to register and unregister the spy with the operating system.

Published by Roman on 29 Jan 2009

DirectShow Filter Graph Spy – CLSID_FilterGraphNoThread

At the time of original implementation I intentionally left off a variation of Filter Graph Manager that runs on application thread, CLSID_FilterGraphNoThread. MSDN says:

CLSID_FilterGraphNoThread creates the Filter Graph Manager on the application’s thread. If you use this CLSID, the thread that calls CoCreateInstance must have a message loop that dispatches messages; otherwise, deadlocks can occur. Also, before the application thread exits, it must release the Filter Graph Manager and all graph objects (such as filters, pins, reference clocks, and so forth).

I have never needed to use this CLSID (by the way Windows SDK has one more CLSID – CLSID_FilterGraphPrivateThread – but it does not seem to be documented) and for this reason I left it for the time a need in it comes up. However it appeared that this CLSID was not created just for fun, there was a need in it. Filter graphs of Windows Media Player playing well known Windows’ clock.avi appeared to be not published on Running Object Table. Why?

Process Monitor showed clearly that Windows Media Player created filter graph through CLSID_FilterGraphNoThread and quite obviously it was not intercepted by Filter Graph Spy (still I wonder what made it different previous time when I could see WMP’s graph).

Continue Reading »

Next »