DirectShow Spy: Intelligent Connect Trace, Selective Process Black Listing

DirectShow Spy is updated with a few new features:

  • retroactive Intelligent Connect trace
  • log output for IAMGraphBuilderCallback-related activity
  • process name based black list to selectively exclude processes from spying

Intelligent Connect Trace

The utility received a capability to read back from its own log file (DirectShowSpy.log, located typically in C:\ProgramData directory) and reconstruct graph building sequence, including steps taken by DirectShow Intelligent Connect.

In order to activate the Intelligent Connect Trace property sheet, one needs to call exported function “DoGraphBuilderCallbackPropertySheetModal“, such as using runndll32 tool:

C:\DirectShowSpy>rundll32 DirectShowSpy.dll,DoGraphBuilderCallbackPropertySheetModal

The upper part of the property page displays recently created DirectShow fitler graphs, newest to older. For a selected graph, the lower part displays events in chronological order. The events include:

The latter two methods also show “Application Result” column and values, which are HRESULT values returned by IAMGraphBuilderCallback callback provided by the application. Typically, a failure HRESULT code indicates that the application rejected the filter.

The trace log is good to expose all DirectShow junk installed in the system. For example,

Continue reading →

How to use your own video transformation DirectShow filter as an effect while playing back a file

One of the popular questions asked in relation to DirectShow filters, and also a popular task is to modify video stream while in playback. There are various reasons one might need to do it, including keying video to replace color/range, or apply text or time code on top of video, including while re-compressing footage, adjust brightness or contrast.

DirectShow BaseClasses/SDK include samples (in most cases EzRGB24 sample is the best to start for a video effect, also demonstrates use of private interface on a filter) to quick-start with the task without getting too much into detail and once this part is done, next step is to integrate filter into playback, connect it with other filters.

As a result of DircetShowflexibility, there are ways to do things not so good, while still being under impression of keeping right track.

File playback is one of the basic tasks with DirectShow. To play a file, one creates a filter graph using powerful helpers provided by Filter Graph Manager object. It might be a actually a single call IGraphBuilder::RenderFile which takes all the complexity of finding matching filters, connecting them together, dealing with splitters and renderers, video and audio. A single call resolves the problems in a convenient way – easy.

Still a simple thing of inserting your own video transformation filters breaks the simplicity. One needs to build the graph partilly, insert the effect and complete building, or build the thing and break in with a new filter. How to find insertion point? Will the other filters like intrusion? Different file types and formats.

There is an easy and elegant solution to pre-add your own effect filter into graph and start rendering a file from there. Sounds reasonable and sometimes works. The problem is however that it does not work always, and you never know when it lets you down. The graph might be build and the effect filter is never taken and is left orphaned aside of playback pipeline.

Reliable graph building assumes you are in control over building steps and allow only the level of flexibility required to connect and build parts – and this is where Intelligent Connect is still a powerful helper. With an effect, the parts are “before the effect” and “after the effect”. RenderFile is no longer an option, and one has to dive deeper into graph building API.

First of all, the building starts with the file itself: unlike RenderFile, IGraphBuilder::AddSourceFilter method adds just the first filter for a given file/format. It stops there and lets caller continue building as required. At this point, it is the right time to manually add effect filter with IFilterGraph::AddFilter (IGraphBuilder is inherited from IFilterGraph and exposes the same method).

Having both ends in the graph for the “before the effect” part, intelligent connect can be used to connect and add filters required to make the connection. For an arbitrary file format, the task may be not trivial: depending on format and installed software components, the chain may look rather different. First, some filters combine stream splitting capability with immediate access to file (or another type of resource), others rely on joint operation of Async File Source filter with a format-dependent splitter filter. Some expose elementary stream pins immediately, some provide transport stream pin.

There may be a few approaches as for addressing pin connection task (see also General Graph-Building Techniques on MSDN). Straightforwardly, one might want to call IGraphBuilder::Connect and take advantage of intelligent connect. Before this can be done, however it takes caller to select a pin of the obtained source filter to start from. There might be a few pins, including those exposing video formats, non-video formats and pre-split formats where video is behind depacketizing (demultiplexing). Considering variety of formats and support, it might make sense to make a first attempt finding a video pin (by enumerating pins and their media types, looking and AM_MEDIA_TYPE::majortype and comparing to MEDIATYPE_Video) and, if not found, taking a first output pin of any type, or going through ping trying to connect first one which succeeds in connection.

An alternate approach is to take advantage of a helper object: Capture Graph Builder. While originally it is intended to help capture graph building, it contains useful methods for general building and connecting pins. It does not own a graph itself: it is designed to operate on top of existing graph, provided by its owner. So one need to provide its existing graph and call helper methods for easy graph building. One of the methods is ICaptureGraphBuilder2::RenderStream, which connects pins of given filters. Unlike API discussed earlier, it takes filter interfaces on input and will be responsible for finding proper pins itself, which might be a good idea if you don’t want to bother yourself doing it. To specify the requested task, it takes media type argument, which in this case might be video or, if fails, any type provided that video media type will still be anyway checked on input of effect filter.

Once the part “before the effect” is done, the other part may be completed as simple as calling IGraphBuilder::Render on the output pin of the effect. This will correspond to final step of original RenderFile execution.

A tiny Visual Studio 2010 C++ project illustrates discussed techniques and it available at SVN repository: RenderStreamTest01:

  • for a given media file in line 109 the project will start graph building
  • a suitable replacement for a video effect filter will be a Sample Grabber filter initialized with a video type (24-bit RGB, but the line 137 can be commented out)
  • switch in line 147 switches between base Connect approach and Capture Graph Builder helper
  • while message box is on the screen and also showing building status, the graph can be looked at using Graph Edit or similar tool, provided that DirectShow Spy is installed; alternatively you might want to put the graph onto ROT manually

The project also illustrates a solution for recent problems referencing Sample Grabber with new SDK. Sample Grabber was obsoleted and removed from Window SDK definition file (qedit.h). In order to resolved the problem without using an older version of SDK, the definitions might be imported from type the corresponding library and (apart from used as such) copied into the source code directly, as in lines 13-60.

See also on graph building:

DirectShow Spy: Memory Allocator Properties

A small update to the DirectShow Spy today: DirectShow Filter Graph Spy prints memory allocator properties as a part of graph topology trace on transition to running state. Why is that and what it is for? Filters normally agree on allocator properties (ALLOCATOR_PROPERTIES, obtained from IMemAllocator, obtained from IMemInputPin) themselves without interference from controlling application. Sometimes an undesired choice of buffers can cause sick runtime behavior, including but not limited to the following:

  1. live audio capture buffers are too long, and introduce significant latency, including from live video capture stream taking place in parallel; controlling application might need to take advantage of IAMBufferNegotiation::SuggestAllocatorProperties and request shorter buffers.
  2. a filter, such as for example DMO Wrapper Filter, may default to 1 buffer on allocator, which means that if a buffer reference is for some reason held for future reference (e.g. a video filter might be holding a reference to be able to re-push a video sample if an application is requesting video update), the entire streaming might lock.
  3. some filters are requesting unreasonably many/large buffers and consume vast spaces of RAM, such as MainConcept MXF Demultiplexer requesting 200 buffers x 640 kilobytes each (128 MB in total out of sudden)
  4. some filters are requesting unreasonably few/small buffers resulting in inability to pre-buffer data

In a chase for answers to questions “Where is my memory?”, “Why is it so choppy?”, “I would really appreciate a nice lipsync” and to troubleshoot the mentioned scenarios it is helpful to understand buffering configuration. DirectShow Filter Spy is here to deliver this information. Once the graph is put into running state, spy prints out topology data into log file (which is in most cases C:\ProgramData\DirectShowSpy.log):

Pin 2: Name "Input 01", Direction "Input", Peer "Tee 0x087A5AF0.Output2"
 Connection media type:
 majortype {73646976-0000-0010-8000-00AA00389B71}, subtype {31435641-0000-0010-8000-00AA00389B71}, pUnk 0x00000000
 bFixedSizeSamples 0, bTemporalCompression 0, lSampleSize 1
 formattype {E06D80E3-DB46-11CF-B4D1-00805F6CBBEA}, cbFormat 170, pbFormat 0x07c46fc0
 rcSource { 0, 0, 0, 0 ), rcTarget { 0, 0, 0, 0 }
 dwBitRate 0, dwBitErrorRate 0, AvgTimePerFrame 0
 dwInterlaceFlags 0x0, dwCopyProtectFlags 0x0, dwPictAspectRatioX 16, dwPictAspectRatioY 9, dwControlFlags 0x0
 bmiHeader.biSize 40, bmiHeader.biWidth 1280, bmiHeader.biHeight 720, bmiHeader.biPlanes 1, bmiHeader.biBitCount 24, bmiHeader.biCompression avc1
 bmiHeader.biSizeImage 0, bmiHeader.biXPelsPerMeter 1, bmiHeader.biYPelsPerMeter 1, bmiHeader.biClrUsed 0, bmiHeader.biClrImportant 0
 dwStartTimeCode 0x00000000, cbSequenceHeader 38, dwProfile 100, dwLevel 31, dwFlags 0x4
 [0x0000] 00 1D 67 64 00 1F AC 24 88 05 00 5B BF F0 00 10
 [0x0010] 00 11 00 00 03 03 E8 00 00 E9 BA 0F 18 32 A0 00
 [0x0020] 05 68 EE 32 C8 B0
 Memory Allocator: 1 buffers, 1,024 bytes each (1,024 bytes total), align 1, prefix 0

Partial Visual C++ .NET 2008 source code is available from SVN, release binary included (Win32, x64); installation instructions are in another post.

DirectShow Spy: IAMFilterData interface

This quick update for DirectShow Filter Graph Spy adds tracing for (deprecated) IAMFilterData interface, and also fixes problem with Media Player Classic – Home Cinema, when the player crashes in External Filters windows.

Partial Visual C++ .NET 2008 source code is available from SVN, release binary included (Win32, x64); installation instructions are in another post.

DirectShow Spy: Filter Mapper Spy

DirectShow Filter Graph Spy was updated to add new functionality: spying over another DirectShow object – Filter Mapper. Filter Mapper object is used for filter registration purposes and also internally by DirectShow’s Intelligent Connect. IFilterMapper2 interface traces provide detailed description on steps DirectShow takes in order to connect and/or render filter pins. For example when trying to render MEDIATYPE_Audio pin, Filter Mapper suggested “RDP DShow Redirection Filter” and, then, “Default DirectSound Device”:

FilterMapperSpy.h(186): CFilterMapperSpy::EnumMatchingFilters: nFlags 0x0, bExactMatch 0, nMinimalMerit 0x00200001, bInputNeeded 1, nInputTypeCount 1, pInputPinCategory NULL, bRender 1, bOutputNeeded 0, nOutputTypeCount 0, pOutputPinCategory NULL
FilterMapperSpy.h(191): CFilterMapperSpy::EnumMatchingFilters: nInputTypeIndex 0, MajorType {73647561-0000-0010-8000-00AA00389B71}, Subtype {00000000-0000-0000-0000-000000000000}
FilterMapperSpy.h(212): CFilterMapperSpy::EnumMatchingFilters: pMoniker @device:sw:{083863F1-70DE-11D0-BD40-00A0C911CE86}\{AB9D6472-752F-43F6-B29E-61207BDA8E06}
FilterMapperSpy.h(220): CFilterMapperSpy::EnumMatchingFilters: sFriendlyName "RDP DShow Redirection Filter", sDescription "", sDevicePath ""
FilterMapperSpy.h(212): CFilterMapperSpy::EnumMatchingFilters: pMoniker @device:cm:{E0F158E1-CB04-11D0-BD4E-00A0C911CE86}\Default DirectSound Device
FilterMapperSpy.h(220): CFilterMapperSpy::EnumMatchingFilters: sFriendlyName "Default DirectSound Device", sDescription "", sDevicePath ""

Important is that DirectShow’s internals are using the mapper through COM instantiation and this provides a safe method to override default behavior process wide in order to, for example, extend Intelligent Connect functionality onto custom, unregistered globally, private filters.

Also, the binary is renamed from FilterGraphSpy.dll to DirectShowSpy.dll.

Partial Visual C++ .NET 2008 source code is available from SVN, release binary included (Win32, x64); installation instructions are in another post.

DirectShow Filter Graph Spy: 64-bit version and hook API

Today’s update for DirectShow Filter Graph Spy introduces 64-bit version (mind the beta state) and a mini-API for an external module to be involved into graph building process.

Filter Graph Spy is offering three new interfaces that provide extensibility of the spy:

  • IFilterGraphAddRemoveHook
  • IFilterGraphConnectHook
  • IFilterGraphStateControlHook

The interfaces are contained in the type library and can be imported using #import directive. An implementation of one or more of these interfaces will receive hook style calls corresponding to respective Filter Graph Manager calls, system wide including in context of other applications.

A COM object may be registered as a hook object with Spy and NoThreadSpy COM classes under predefined registry keys:

Registering a DirectShow Filter Graph Spy Hook

Spy will instantiate the registered hook objects and forward them the calls it receive, before passing them to system Filter Graph Manager object. A hook object has an option to override default processing, including, for example, inserting its own filter in between. For example, IFilterGraph::Reconnect call is implemented the following way:

STDMETHOD(Reconnect)(IPin* pPin) throw()
    _Z4(atlTraceCOM, 4, _T("...\n"));
        OnReconnect(pT, pPin, &bDefault);
    return m_pInnerFilterGraph2->Reconnect(pPin);

Before passing the call to original Reconnect method, spy is iterating through associated hooks, passing them IFilterGraphConnectHook::OnReconnect call. Setting bDefault parameter to FALSE will prevent spy from passing the call to original method.

Included BdaHooks project shows a sample implementation of the hooking COM classes (note .rgs registration).

Filter Graph Spy is compatible with all current Windows operating systems, 32-bit and 64-bit (x64), in particular including:

  • Windows 7
  • Windows Server 2008
  • Windows Vista
  • Windows Server 2003
  • Windows XP
  • Windows 2000

NOTE: DirectShow Filter Graph Spy is NOT suitable for production environment, it is NOT licensed to be redistributed to be a part of production state software item.

Partial Visual C++ .NET 2008 source code is available from SVN, release binary included (Win32, x64); installation instructions are in another post.

DirectShow Filter Graph Spy: Log Filters and Connection Media Types

Having hard time to connect to remote filter graph on Windows 7 machine, I suspected that some of the filters do not like being connected through marshaling and freeze. Or, actually more likely, it is SDK’s proppage.dll which hosts proxy stub pairs behaves itself in a not quite expected way and freezes remote application (Graph Edit or similar) during attempt to connect to remote graph.

As the goal was to find out details about connection media types between the filters being spied over, the spy received a new feature of logging graph details each time the graph is going to running state:

Filter 7: Name "[1BDADev]", Class {17CCA71B-ECD7-11D0-B908-00A0C9223196}, State 2
 Pin 1: Name "MPEG2 Transport", Direction "Input", Peer "[0BDATnr].MPEG2 Transport"
 Connection media type:
 majortype {E436EB83-524F-11CE-9F53-0020AF0BA770}, subtype {E06D8023-DB46-11CF-B4D1-00805F6CBBEA}, pUnk 0x00000000
 bFixedSizeSamples 0, bTemporalCompression 0, lSampleSize 1
 formattype {8DEDA6FD-AC5F-4334-8ECF-A4BA8FA7D0F0}, cbFormat 24, pbFormat 0x00350868
 [0x0000] BC 00 00 00 38 9A 00 00 00 00 00 00 00 00 00 00
 [0x0010] 00 00 00 00 00 00 00 00
 Pin 2: Name "MPEG2 Transport", Direction "Output", Peer "Infinite Pin Tee Filter[BDATsIft].Input"
 Connection media type:
 majortype {E436EB83-524F-11CE-9F53-0020AF0BA770}, subtype {F4AEB342-0329-4FDD-A8FD-4AFF4926C978}, pUnk 0x00000000
 bFixedSizeSamples 1, bTemporalCompression 0, lSampleSize 39480
 formattype {0F6417D6-C318-11D0-A43F-00A0C9223196}, cbFormat 0, pbFormat 0x00000000
Filter 8: Name "[0BDATnr]", Class {17CCA71B-ECD7-11D0-B908-00A0C9223196}, State 2
 Pin 1: Name "Input0", Direction "Input", Peer "Microsoft Network Provider[NwPvd].Antenna Out"
 Connection media type:
 majortype {71985F41-1CA1-11D3-9CC8-00C04F7971E0}, subtype {E436EB8E-524F-11CE-9F53-0020AF0BA770}, pUnk 0x00000000
 bFixedSizeSamples 0, bTemporalCompression 0, lSampleSize 1
 formattype {0F6417D6-C318-11D0-A43F-00A0C9223196}, cbFormat 0, pbFormat 0x00000000
 Pin 2: Name "MPEG2 Transport", Direction "Output", Peer "[1BDADev].MPEG2 Transport"
 Connection media type:
 majortype {E436EB83-524F-11CE-9F53-0020AF0BA770}, subtype {E06D8023-DB46-11CF-B4D1-00805F6CBBEA}, pUnk 0x00000000
 bFixedSizeSamples 0, bTemporalCompression 0, lSampleSize 1
 formattype {8DEDA6FD-AC5F-4334-8ECF-A4BA8FA7D0F0}, cbFormat 24, pbFormat 0x00350868
 [0x0000] BC 00 00 00 38 9A 00 00 00 00 00 00 00 00 00 00
 [0x0010] 00 00 00 00 00 00 00 00

Partial Visual C++ .NET 2008 source code is available from SVN, release binary included; installation instructions are in another post.