[
    db_command("SELECT ServerData FROM Server WHERE Server = ?")
]
class CGetServerData
{
public:
    [ db_param(1) ] LONG m_nServer;
    [ db_column(1, length = "m_nDataLength") ] ISequentialStream* m_pDataStream; DBLENGTH m_nDataLength;
};

It worked great with Visual Studio .NET 2003 and it failed to work with later releases. There are questions on internet about the problem, but there few answers if any. As I recently had to convert a project from 2003 version of the compiler to Visual Studio 2008, the problem was finally to be resolved.

The first problem, and an easier one was that you have to put attribute values in quotes. ‘Db_column(1, …’ has to be “db_column(”1″, …’ and besides going through code and making changes, it appeared that if earlier implementation managed to distinguish between ordinal column numbers and field names by checking provided argument to be a number or a string, with Visual Studio 2008 one has to provide strings at any time. If the string is a valid integer, the attribute processor will treat it as an ordinal number.

So if you want your code to be both compiled with 2003 and 2008 versions of Visual Studio, you have a puzzle now: you have to use quotes, but version 2003 will no longer treat the argument as ordinal number. Note that version 2003 will keep compiling the code and the problem will only come up on runtime. As eventually in the project of interest support for version 2003 will be dropped, I was not interested in this problem anymore.

The main problem is related to use of ISequentialStream pointer variable as a data field. Again, it worked great with Visual Studio .NET 2003, it no longer did with Visual Studio .NET 2005. First of all, you cannot use this data type anymore, as templates don’t provide an automatic mapper to OLE DB Type.  Instead of ISequentialStream you have to use IUnknown and QueryInterface the provided pointer before use.

This solves compilation problem, but still the code does not work on runtime. An attempt to bind to the column using such accessor results in DB_E_ERRORSOCCURRED (0×80040E21) “Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.” A more thorough look gave DBBINDSTATUS_BADBINDINFO (3) for binding to the column of interest.

The cause of the problem is missing DBOBJECT object pointer in the corresponding DBBINDING::pObject field. Why? A general rule for troubleshooting this kind of problem is to enable expanding attributed source code to check what exactly is generated.

The problem is that unlike previous versions of Visual Studio, ATL attributed code provider is no longer generates correct OLEDB template map code for BLOB fields. And this results in incorrectly generated DBBINDING structure, inability to bind to column and as a final result in DB_E_ERRORSOCCURRED error code which does not give a sufficient clue on the root of the problem.

While the map needs a BLOB_* macro for the BLOB column (e.g. BLOB_ENTRY or BLOB_ENTRY_LENGTH), the generated code uses _COLUMN_ENTRY_CODE instead. This causes missing (null) DBBINDING::pObject field in the binding structure:

BEGIN_ACCESSOR_MAP(_CGetServerDataAccessor, 1)
    BEGIN_ACCESSOR(0, true)
        //_COLUMN_ENTRY_CODE(1, DBTYPE_IUNKNOWN, _SIZE_TYPE(m_pDataUnknown), 0, 0, offsetbuf(m_pDataUnknown), offsetbuf(m_nDataLength), 0)
        BLOB_ENTRY_LENGTH(1, __uuidof(ISequentialStream), STGM_READ, m_pDataUnknown, m_nDataLength)
    END_ACCESSOR()
END_ACCESSOR_MAP()

So it appears that to fix the problem, an attributed class needs to be replaced with a non-attributed update. To ease the conversion one can expand attributed source and use it as a base for the non-attributed class.

And example of such correction/replacement is provided below for a reference.

#if TRUE
class _CGetServerDataAccessor
{
public:

DEFINE_COMMAND_EX(_CGetServerDataAccessor, L"SELECT Data FROM Server WHERE Server = ?")

BEGIN_PARAM_MAP(_CGetServerDataAccessor)
    SET_PARAM_TYPE(DBPARAMIO_INPUT)
    //_COLUMN_ENTRY_CODE(1, _OLEDB_TYPE(m_nServer), _SIZE_TYPE(m_nServer), 0, 0, offsetbuf(m_nServer), 0, 0)
    COLUMN_ENTRY(1, m_nServer)
END_PARAM_MAP()

BEGIN_ACCESSOR_MAP(_CGetServerDataAccessor, 1)
    BEGIN_ACCESSOR(0, true)
        //_COLUMN_ENTRY_CODE(1, DBTYPE_IUNKNOWN, _SIZE_TYPE(m_pDataUnknown), 0, 0, offsetbuf(m_pDataUnknown), offsetbuf(m_nDataLength), 0)
        BLOB_ENTRY_LENGTH(1, __uuidof(ISequentialStream), STGM_READ, m_pDataUnknown, m_nDataLength)
    END_ACCESSOR()
END_ACCESSOR_MAP()

public:
    LONG m_nServer;
    IUnknown* m_pDataUnknown;
    DBLENGTH m_nDataLength;
};

class CGetServerData :
    public CCommand<CAccessor<_CGetServerDataAccessor> >
{
public:
// CGetServerData
    HRESULT OpenRowset(const CSession& Session, LPCWSTR pszCommand = NULL) throw()
    {
        if(!pszCommand)
            _V(_CGetServerDataAccessor::GetDefaultCommand(&pszCommand));
        return Open(Session, pszCommand, NULL);
    }
};
#else
[
    db_command("SELECT Data FROM Server WHERE Server = ?")
]
class CGetServerData
{
public:
    [ db_param("1") ] LONG m_nServer;
    [ db_column("1", length = "m_nDataLength") ] IUnknown* m_pDataUnknown; DBLENGTH m_nDataLength;
};
#endif

The filters behind that empower the sample are described in another post.

RenderHttpMjpegVideoIntoFiles01 sample takes an URL on the input to generate image/jpeg JPEG or multipart/x-mixed-replace M-JPEG stream. For example, it might be http://demo1.stardotcams.com/nph-mjpeg.cgi feed from a demo StarDot Technologies IP camera.

The application will create a new directory to write files into, and will save each new video frame received into new JPEG file.

Z:\MediaTools\Samples\RenderHttpMjpegVideoIntoFiles01\Release>RenderHttpMjpegVideoIntoFiles01.exe http://demo1.stardotcams.com/nph-mjpeg.cgi
URL: http://demo1.stardotcams.com/nph-mjpeg.cgi
Writing to directory: Z:\MediaTools\Samples\RenderHttpMjpegVideoIntoFiles01\Release\2010-05-12 22-07-37
Event: nCode EC_CLOCK_CHANGED 0xD, nParameter1 0x00000000, nParameter2 0x00000000
Event: nCode EC_PAUSED 0xE, nResult 0x00000000, nParameter2 0x00000000
[...]
^C

The application will generate the files and convert media sample time stamps into file time.

The DirectShow graph that implements the operation is the following:

The other sample RenderJpegFiles01 takes a directory path to look for JPEG files, e.g. generated by previous sample, and pushes them into DirectShow graph as a video feed. File times will be converted [back] to media sample times.

Z:\MediaTools\Samples\RenderJpegFiles01\Release>RenderJpegFiles01 "Z:\MediaTools\Samples\RenderHttpMjpegVideoIntoFiles01\Release\2010-05-12 22-07-37"
Directory: Z:\MediaTools\Samples\RenderHttpMjpegVideoIntoFiles01\Release\2010-05-12 22-07-37
Event: nCode EC_CLOCK_CHANGED 0xD, nParameter1 0x00000000, nParameter2 0x00000000
Event: nCode EC_PAUSED 0xE, nResult 0x00000000, nParameter2 0x00000000
Event: nCode EC_VMR_RECONNECTION_FAILED 0x55, nResult 0x80004005, nParameter2 0x00000000
Event: nCode EC_VMR_RENDERDEVICE_SET 0x53, RendererDeviceType 0x2, nParameter2 0x00000000
Event: nCode EC_VIDEO_SIZE_CHANGED 0xA, Extent 384x240, nParameter2 0x00000000
Event: nCode EC_COMPLETE 0x1, nResult 0x00000000, nParameter2 0x00000000
Event: nCode EC_ERRORABORT 0x3, nResult 0x80004005, nParameter2 0x00000000
Detected resolution change, new resolution is 1024x768, recreating filter graph...
Event: nCode EC_VMR_RECONNECTION_FAILED 0x55, nResult 0x80004005, nParameter2 0x00000000
Event: nCode EC_VMR_RENDERDEVICE_SET 0x53, RendererDeviceType 0x2, nParameter2 0x00000000
Event: nCode EC_VIDEO_SIZE_CHANGED 0xA, Extent 384x240, nParameter2 0x00000000
Event: nCode EC_CLOCK_CHANGED 0xD, nParameter1 0x00000000, nParameter2 0x00000000
Event: nCode EC_PAUSED 0xE, nResult 0x00000000, nParameter2 0x00000000

Download links:

• RenderHttpMjpegVideoIntoFiles01: source code, binary
• RenderJpegFiles01: source code, binary

Investigation details deserve a separate post, while this one briefly outlines the issues and also accompany the repository update with a version compatible with Vista OS.

First of all, COM registration of the DLL (which definitely requires privilege elevation) succeeded on Vista. This means the registration procedure did not encounter any errors on the way, or poorly written code ignored the problem. It appeared that the source of the problem was CoTreatAsClass API, which failed to do the requested action, however returned status code indicating successful operation. This definitely looks like a bug and further comments on this particular behavior are expected to appear on Windows Applications Security MSDN forum, where I opened a topic on the matter.

With a COM TreatAs feature activated, the class’s behavior to instantiate instead of DirectShow’s CLSID_FilterGraph is restored, and in particular the DLL generates FilterGraphSpy.log log file on filter graph activity. Note that log file location is OS dependent (due to Vista’s permissions and file system virtualization):

• pre-Vista OS: root of syste drive, typically C:\
• starting Vista, administrator with elevated privileges: CSIDL_COMMON_APPDATA, typically C:\ProgramData (note this directory is hidden by deafult)
• starting Vista, without elevated administrator privileges: CSIDL_LOCAL_APPDATA, typically C:\Users\$(UserName)\AppData\Local (note that AppData directory is hidden by deafult)

Still even with the log file generated and indicating activation of the spy, it was unable to connect to remote graph through the running object table (ROT). It appeared that the ROT entires are there where expected, it was OK to get an object from ROT and the problem came from QueryInterface code:

CComPtr<IUnknown> pFilterGraphUnknown;
ATLENSURE_SUCCEEDED(pRunningObjectTable->GetObject(pMoniker, &pFilterGraphUnknown));
CComQIPtr<IMyFilterGraph> pFilterGraph = pFilterGraphUnknown; // E_NOINTERFACE

The call reached the original object but COM subsystem was unable to marshal the interface through apartments to enable interprocess communication on it. The reason for this is absence of PSFactoryBuffer class (CLSID {92A3A302-DA7C-4A1F-BA7E-1802BB5D2D02}), which provides proxy/stub pairs for marshaling well known DirectShow interfaces in the Vista’s version of quartz.dll. As mentioned by Microsoft’s Mike Wasson, this class was moved from quartz.dll into Vista SDK’s proppage.dll, so in order to obtain connectivity to remote DirectShow graphs starting Vista, one needs to install this DLL with Windows SDK, or otherwise have it registered with the operating system.

Also note that DirectShow-enabled applications that have their filter graphs published on ROT will need an utility application such as GraphEdit started with the same permissions (elevated or not) in order to be able to access ROT entires.

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

• audio input device unrelated to physical device, to avoid conditions when device is already in use by someone else
• non-zero audio signal which is independent of certain speaker or broadcasting service, including one that makes capture, transmission or rendering issues easily perceptible by human
• configurable to provide different audio media types, including through well known DirectShow interfaces, such as IAMStreamConfig
• configurable to present in system as both regular DirectShow Filter, and as an Audio Capture Source which can be enumerated by applications
• to be able to check how exactly other applications are accessing and configuring audio capture source

I added a new filter to the Media Tools library, a Tone Source Filter. The filter generates PCM Audio data, either infinite sine waveform, or interrupted signal  of requested parameters:

The filter accepts connection on PCM media types with flexible sampling rates in range 8 KHz through 200 KHz, 8-bit audio and 1 channel. Media type is also configurable through or, if not configured this way, the first enumerated media type off the Output pin is also configurable through property page and/or persistent setting accessible via private IDispatch derived interface property.

Additionally, the filter may configured as a system-wide available audio input device, such as for example available to VLC Media Player or Media Player Classic:

In order to configure the filter as such device, the library exports function DoToneSourceFilterDevicePropertySheetModal, which can be called using rundll32 utility (“rundll32 Acquisition.dll,DoToneSourceFilterDevicePropertySheetModal” from command line):

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

File and Class Summary

Acqusition.dll

Acqusition.dll (download) hosts the following classes:

• DirectShow Filters
  • Tone Source Filter, to generate reference/debug audio data

Class Overview

Tone Source Filter

The filter generates uninterrupted or interrupted reference sine waveform in form of PCM audio data.

• Filter Executable: Acquisition.dll
• Filter CLSID: __uuidof(ToneSourceFilter) {8002935A-B2EC-40ef-968C-E0358E5DED10}
• Filter Property Page CLSID: __uuidof(ToneSourceFilterGeneralPropertyPage), {EBD8ABB2-6DD3-4c54-A7F1-9FE4DA283EDF}, ToneSourceFilterPreferedFormatPropertyPage, {EE224187-4FA3-4c3f-9D5D-492694CCFEB7}
• Filter Merit: MERIT_DO_NOT_USE
• Filter Category: CLSID_LegacyAmFilterCategory (also configurable as CLSID_AudioInputDeviceCategory)
• Filter Interfaces: IPersistStreamInit, IPersistStream, ISpecifyPropertyPages, IPersistPropertyBag, IBaseFilter, IMediaFilter, IPersist, IAMovieSetup, IAMFilterMiscFlags, IToneSourceFilter, IDispatch
• Filter Pins: single ouput pin (Ouput)
• Output Pin Media Types: MEDIATYPE_Audio/MEDIASUBTYPE_PCM
• Output Pin Interfaces: IPin, IMemInputPin, IAMPushSource, IAMLatency, IAMBufferNegotiation, IKsPropertySet, IAMStreamConfig

Remarks

The hosting library (DLL) exposes DoToneSourceFilterDevicePropertySheetModal function which provides GUI to configure the filter as one or more audio capture source device, which can be enumerated by System Device Enumerator.

While troubleshootting JPEG artefact issue with I used the following graph:

which started M-JPEG streaming from a camera and had entire stream written into Response-01586228-0001-0001-Headers.txt (Headers) and Response-01586228-0001-0001.dat (Data) files, while parsed JPEG frames were also written into individual files and were available for comparison.

Other changes include:

• new HTTP Stream File Renderer Filter (details are below)
• attachable media types on internally used memory allocators, which enable filters to effectively reagree media types in running state
• workaround for Panasonic BL-C140 camera (may be also helpful for other models), details here

There also seems to be a bug in Intel IPP 6.0 JPEG decoder that leaves bottom right image block incorrectly decoded, only when a sequence of images is decoded and the frame is not the first in the sequence.

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

File and Class Summary

Acqusition.dll

Acqusition.dll (download) hosts the following classes:

• DirectShow Filters
  • HTTP Stream File Renderer Filter, to write raw HTTP response headers and data into files

Class Overview

HTTP Stream File Renderer Filter

The filter renders input stream of one or more HTTP responses and write HTTP response headers and data into files, per HTTP response, into provided directory.

• Filter Executable: Acquisition.dll
• Filter CLSID: __uuidof(HttpStreamFileRendererFilter) {B02FDAEF-E851-4380-82D6-04772E33675D}
• Filter Property Page CLSID: __uuidof(HttpStreamFileRendererFilterDirectoryPropertyPage), {FAF31D20-842A-46b4-B6F8-E410FB0188A5}
• Filter Merit: MERIT_DO_NOT_USE
• Filter Category: CLSID_LegacyAmFilterCategory
• Filter Interfaces: IPersistStreamInit, IPersistStream, ISpecifyPropertyPages, IBaseFilter, IMediaFilter, IPersist, IAMovieSetup, IAMFilterMiscFlags, IHttpStreamFileRendererFilter, IDispatch, IFileSinkFilter2 (not available in graphedt.exe process), IFileSinkFilter (not available in graphedt.exe process)
• Filter Pins: single input pin (Input)
• Input Pin Media Types: MEDIATYPE_Stream/MEDIASUBTYPE_NULL
• Input Pin Interfaces: IPin, IMemInputPin

Remarks

None.

It was clear at that point that it is related to filter graph spy and the problem needs a fix. The problem definitely looked like the spy leave outstanding references to the filter graph and when the application releases graph and it is supposed to be automatically destroyed, it is still being alive and possibly, such as in case of Media Player Classic, even running.

The first clue was that being published to running object table (ROT), the object gets more than one additional reference on it and as such it never gets removed from ROT until application terminates as ROT keeps a reference to the graph and there is noone to remove it. By original design spy published filter graph on ROT and removed one reference immediately after that to compensate the ROT’s reference (one only) making it a weak reference. So when all references are released from the filter graph but ROT reference, the graph gets destroyed and in FinalRelease class on the COM object we re-add the reference back and automatically remove the object from ROT as well. The whole scheme works in assumption that ROT adds one reference only, if there are more then the object is referenced by the ROT forever.

Debugging however showed that ROT adds a single reference and there is no problem from this side. However I immediately noticed  another weird behavior in the same method FinalConstruct. An underlying (real) filter graph object, i.e. CFilterGraph from quartz.dll, is adding a reference on the outer COM object when being queried for a private interface from implicit (also known as main, raw) IUnknown when created as aggregated object. And it is not exactly the behavior one would expect from aggregated object because it basically break proper reference counting. I realize that aggregating is a rather rare technique and DirectShow COM base is one of the very first COM bases at all which then stopped being developed, so perhaps this problem was not identified on time and later left unfixed at all, so in present situation there should rather be a workaround.

It is also worth mentioning that there are two scenarios involved. My CSpy object creates DirectShow CFilterGraph as an aggregated object. And CSpy in its turn may be created as aggregated too. For example, Windows Media Player creates (as most applications in this universe do) filter graph object the regular way, as a standalone COM object. However it appears that Media Player Classic creates filter graph as aggregated, for whatever it needs it for. Because of the discovered problem inner CFilterGraph adds an outstanding reference on the top level “controlling IUnknown”, which is either CSpy’s private IUnkown in case of Windows Media Player, or higher level IUnknown in case of Media Player Classic.

So the proper workaround is to release unwanted references from controlling unknown rather than self:

CComPtr<IUnknown> pUnknown;
ATLENSURE_SUCCEEDED(pClassFactory->CreateInstance(pControllingUnknown, __uuidof(IUnknown), (VOID**) &pUnknown));
// NOTE: DirectShow FilterGraph is incorrectly implementing COM aggregation adding outer reference to interfaces queried from private IUnknown
CComQIPtr<IFilterGraph2> pFilterGraph2 = pUnknown;
ATLENSURE_THROW(pFilterGraph2, E_NOINTERFACE);
pFilterGraph2.p->Release(); // <<--------------------------

It looked like the problem is finally worked around but in aggregated creation scenario it brought another problem up. Since instantiation of inner CFilterGraph takes place in FinalConstruct and CSpy is also being instantiated as aggregated, it appears that outer COM object is sensible to manipulation with its reference counter. CFilterGraph added a reference it should not, CSpy released it in compensation but on this early stage the compensating Release might zero reference counter and might initiate unexpected object destruction. It is not necessarily this way as it depends on outer COM object base, but it seems that Media Player Classic does not do DECLARE_PROTECT_FINAL_CONSTRUCT or its equavalent and things go the worst scenario.

CSpy is an inner object and we don’t have any external instance to hold a temporary reference for us while outer COM object is being created, so a workround for this is to temporarily hold a circular reference to an outer object to let it complete its instanatiation and pass another reference higher up to the controlling application. At which point we are going to be safe to release the temporary reference and normalize reference counters.

if(m_bIsAggregated)
{
    pControllingUnknown.p->AddRef();
    const ULONG nReferenceCount = pControllingUnknown.p->Release();
    if(nReferenceCount == 1)
        m_pTemporaryUnknown = pControllingUnknown;
}

The only thing is left is to release the temporary reference early enough, but we have a filter graph, don’t we? What is typically done with a filter graph, adding a filter right? And it is a controlling application which has its own reference, who adds filters so we are safe to release temporary reference in IFilterGraph::AddFilter or IGraphBuilder::AddSourceFilter (actually it would not hurt to add this in other methods as well):

STDMETHOD(AddFilter)(IBaseFilter* pFilter, LPCWSTR pszName) throw()
{
    _Z4(atlTraceCOM, 4, _T("pszName \"%s\"\n"), CString(pszName));
    ReleaseTemporaryUnknown(); // <<--------------------------
    return m_pInnerFilterGraph2->AddFilter(pFilter, pszName);
}

Additionally, the spy also implements now IMediaControl interface and traces calls to log file, it is convenient.

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

reg-FilterGraphSpy.dll.bat and unreg-FilterGraphSpy.dll.bat files in SVN are convenient batch files to register and unregister the spy with the operating system.

CLSID_FilterGraphNoThread creates the Filter Graph Manager on the application’s thread. If you use this CLSID, the thread that calls CoCreateInstance must have a message loop that dispatches messages; otherwise, deadlocks can occur. Also, before the application thread exits, it must release the Filter Graph Manager and all graph objects (such as filters, pins, reference clocks, and so forth).

I have never needed to use this CLSID (by the way Windows SDK has one more CLSID – CLSID_FilterGraphPrivateThread – but it does not seem to be documented) and for this reason I left it for the time a need in it comes up. However it appeared that this CLSID was not created just for fun, there was a need in it. Filter graphs of Windows Media Player playing well known Windows’ clock.avi appeared to be not published on Running Object Table. Why?

Process Monitor showed clearly that Windows Media Player created filter graph through CLSID_FilterGraphNoThread and quite obviously it was not intercepted by Filter Graph Spy (still I wonder what made it different previous time when I could see WMP’s graph).

In order to extend spy’s support to CLSID_FilterGraphNoThread I moved most of the code from CSpy COM class into CSpyT template class and derived two COM classes from it, CSpy and CNoThreadSpy, in order to cover and “COM Treat As” both DirectShow/Quartz COM classes.

////////////////////////////////////////////////////////////
// CSpy

class ATL_NO_VTABLE CSpy :
    public CSpyT<CSpy, &CLSID_FilterGraph>,
    public CComCoClass<CSpy, &CLSID_Spy>
{
public:
    enum { IDR = IDR_SPY };

public:
// CSpy
};

OBJECT_ENTRY_AUTO(__uuidof(Spy), CSpy)

////////////////////////////////////////////////////////////
// CNoThreadSpy

class ATL_NO_VTABLE CNoThreadSpy :
    public CSpyT<CNoThreadSpy, &CLSID_FilterGraphNoThread>,
    public CComCoClass<CNoThreadSpy, &CLSID_NoThreadSpy>
{
public:
    enum { IDR = IDR_NOTHREADSPY };

public:
// CNoThreadSpy
};

OBJECT_ENTRY_AUTO(__uuidof(NoThreadSpy), CNoThreadSpy)

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

Who could imagine Windows Media Player to build such a filter graph for a video and audio Windows Media file? SRS Wow DMO, equalizer, analyzer and stuff?

There are many tricks to catch the created filter graph but the most legit is the mentioned above “Treat As” feature to emulate certain COM class identifier (CLSID) through another coclass. COM API offers CoTreatAsClass and CoGetTreatAsClass functions to manage the emulation. To emulate DirectShow’s CLSID_FilterGraph through my CSpy class, I am adding the following code to the default COM registration process for the utility DLL:

static HRESULT WINAPI UpdateRegistry(BOOL bRegister) throw()
{
    _Z2(atlTraceRegistrar, 2, _T("bRegister %d\n"), bRegister);
    _ATLTRY
    {
        CLSID TreatAsClassIdentifier;
        HRESULT nCoGetTreatAsClassResult = CoGetTreatAsClass(CLSID_FilterGraph, &TreatAsClassIdentifier);
        __C(nCoGetTreatAsClassResult);
        __D(!bRegister || nCoGetTreatAsClassResult != S_OK || TreatAsClassIdentifier == GetObjectCLSID(), E_UNNAMED);
        if(!bRegister && nCoGetTreatAsClassResult == S_OK)
            __C(CoTreatAsClass(CLSID_FilterGraph, CLSID_NULL));
        _A(_pAtlModule);
        UpdateRegistryFromResource<CSpy>(bRegister);
        if(bRegister)
            __C(CoTreatAsClass(CLSID_FilterGraph, GetObjectCLSID()));
    }
    _ATLCATCH(Exception)
    {
        _C(Exception);
    }
    return S_OK;
}

The idea is that the emulation is enabled in case it was not enabled before or it was already enabled through our coclass. Every time an application cocreates an instance of Filter Graph Manager (FGM), CSpy class will be created instead.

Still as we have no plans to reinvent one of the most important DirectShow objects, we have to embed the original FGM insde to forward the calls to. The proper place in the ATL C++ code is FinalConstruct and what we actually do there is instantiating an aggregated FGM from the original in-process server. The safest would be to query registry for the original host, but we know it’s hosted by quartz.dll and it seems to be safe to hardcode:

HINSTANCE hModule = CoLoadLibrary(_T("quartz.dll"), TRUE);
_ATLTRY
{
    typedef HRESULT (WINAPI *DLLGETCLASSOBJECT)(REFCLSID, REFIID, VOID**);
    DLLGETCLASSOBJECT DllGetClassObject = (DLLGETCLASSOBJECT) GetProcAddress(hModule, "DllGetClassObject");
    __E(DllGetClassObject);
    CComPtr<IClassFactory> pClassFactory;
    __C(DllGetClassObject(CLSID_FilterGraph, __uuidof(IClassFactory), (VOID**) &pClassFactory));
    _A(pClassFactory);
    CComPtr<IUnknown> pUnknown;
    __C(pClassFactory->CreateInstance(GetControllingUnknown(), __uuidof(IUnknown), (VOID**) &pUnknown));
    CComQIPtr<IFilterGraph2> pFilterGraph2 = pUnknown;
    __D(pFilterGraph2, E_NOINTERFACE);

What is remained at the point is to implement IFilterGraph2 (along with inherited IGraphBuilder and IFilterGraph) and forward all incoming calls to the inner aggregated object. It would also be helpful to trace calls to a log file C:\FilterGraphSpy.log:

2009-01-28 22:07:51	4456	5656	dllmain.h(29): CFilterGraphSpyModule::CFilterGraphSpyModule: this 0x01cd327c
2009-01-28 22:07:51	4456	5656	Spy.h(75): CSpy::CSpy: this 0x01ce297c
2009-01-28 22:07:51	4456	5656	Spy.h(132): CSpy::AddFilter: pszName "C:\Documents and Settings\John Doe\Desktop\Melissa Cherry Interview.wmv"
2009-01-28 22:07:51	4456	5656	Spy.h(142): CSpy::EnumFilters: ...
2009-01-28 22:07:51	4456	5656	Spy.h(132): CSpy::AddFilter: pszName "ffdshow Audio Decoder"
2009-01-28 22:07:51	4456	5656	Spy.h(152): CSpy::ConnectDirect: ...
2009-01-28 22:07:51	4456	5656	Spy.h(137): CSpy::RemoveFilter: ...
2009-01-28 22:07:51	4456	5656	Spy.h(132): CSpy::AddFilter: pszName "WMAudio Decoder DMO"
2009-01-28 22:07:51	4456	5656	Spy.h(152): CSpy::ConnectDirect: ...
2009-01-28 22:07:51	4456	5656	Spy.h(142): CSpy::EnumFilters: ...

And additionally, the filter graph is immediately published on the ROT and is accessible with GraphEdit. Feel free to spy over graphs system wide.

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

The key points are:

• manage NOTIFYICONDATA structure (obviously!), where non-zero .cbSize will be an indication of created icon
• create an icon in WM_SYSCOMMAND/SC_MINIMIZE handler and hide instead default minimization
• handle icon’s WM_LBUTTONDBLCLK to restore and WM_RBUTTONUP to pop up a menu
• use default dialog menu to avoid having private one, handle SC_RESTORE and SC_CLOSE system commands to restore and close from system tray icon popup menu

Relevant source code from application main window (dialog) class:

class CMainDialog :
...
{
public:

BEGIN_MSG_MAP_EX(CMainDialog)
...
    MSG_WM_INITDIALOG(OnInitDialog)
    MSG_WM_DESTROY(OnDestroy)
    MSG_WM_SYSCOMMAND(OnSysCommand)
    MESSAGE_HANDLER_EX(WM_SYSTEMTRAYICON, OnSystemTrayIcon)
    COMMAND_ID_HANDLER_EX(SC_RESTORE, OnScRestore)
    COMMAND_ID_HANDLER_EX(SC_CLOSE, OnScClose)
...
END_MSG_MAP()
...
    enum
    {
        WM_FIRST = WM_APP,
        WM_SYSTEMTRAYICON,
    };
...
private:
    NOTIFYICONDATA m_NotifyIconData;
...
public:
// CMainDialog
    CMainDialog() throw()
    {
        ZeroMemory(&m_NotifyIconData, sizeof m_NotifyIconData);
        ...
    }

// Window message handlers
    LRESULT OnInitDialog(HWND, LPARAM)
    {
        ZeroMemory(&m_NotifyIconData, sizeof m_NotifyIconData);
        ...
    }
    LRESULT OnDestroy()
    {
        if(m_NotifyIconData.cbSize)
        {
            Shell_NotifyIcon(NIM_DELETE, &m_NotifyIconData);
            ZeroMemory(&m_NotifyIconData, sizeof m_NotifyIconData);
        }
        ...
    }
    LRESULT OnSysCommand(UINT nCommand, CPoint)
    {
        switch(nCommand)
        {
        case SC_MINIMIZE:
            if(!m_NotifyIconData.cbSize)
            {
                m_NotifyIconData.cbSize = NOTIFYICONDATAA_V1_SIZE;
                m_NotifyIconData.hWnd = m_hWnd;
                m_NotifyIconData.uID = 1;
                m_NotifyIconData.uFlags = NIF_ICON | NIF_MESSAGE | NIF_TIP;
                m_NotifyIconData.uCallbackMessage = WM_SYSTEMTRAYICON;
                m_NotifyIconData.hIcon = AtlLoadIconImage(IDI_MODULE, LR_DEFAULTCOLOR, GetSystemMetrics(SM_CXSMICON), GetSystemMetrics(SM_CYSMICON));
                CString sWindowText;
                GetWindowText(sWindowText);
                _tcscpy_s(m_NotifyIconData.szTip, sWindowText);
                if(!Shell_NotifyIcon(NIM_ADD, &m_NotifyIconData))
                {
                    SetMsgHandled(FALSE);
                    return 0;
                }
            }
            ShowWindow(SW_HIDE);
            break;
        ...
        default:
            SetMsgHandled(FALSE);
        }
        return 0;
    }
    LRESULT OnSystemTrayIcon(UINT, WPARAM wParam, LPARAM lParam)
    {
        ATLASSERT(wParam == 1);
        switch(lParam)
        {
        case WM_LBUTTONDBLCLK:
            SendMessage(WM_COMMAND, SC_RESTORE);
            break;
        case WM_RBUTTONUP:
            {
                SetForegroundWindow(m_hWnd);
                CMenuHandle Menu = GetSystemMenu(FALSE);
                Menu.EnableMenuItem(SC_RESTORE, MF_BYCOMMAND | MF_ENABLED);
                Menu.EnableMenuItem(SC_MOVE, MF_BYCOMMAND | MF_GRAYED);
                Menu.EnableMenuItem(SC_SIZE, MF_BYCOMMAND | MF_GRAYED);
                Menu.EnableMenuItem(SC_MINIMIZE, MF_BYCOMMAND | MF_GRAYED);
                Menu.EnableMenuItem(SC_MAXIMIZE, MF_BYCOMMAND | MF_GRAYED);
                Menu.EnableMenuItem(SC_CLOSE, MF_BYCOMMAND | MF_ENABLED);
                CPoint Position;
                ATLVERIFY(GetCursorPos(&Position));
                Menu.TrackPopupMenu(TPM_LEFTALIGN | TPM_BOTTOMALIGN, Position.x, Position.y, m_hWnd);
            }
            break;
        }
        return 0;
    }
    LRESULT OnScRestore(UINT, INT, HWND)
    {
        if(m_NotifyIconData.cbSize)
        {
            Shell_NotifyIcon(NIM_DELETE, &m_NotifyIconData);
            ZeroMemory(&m_NotifyIconData, sizeof m_NotifyIconData);
        }
        ShowWindow(SW_SHOW);
        BringWindowToTop();
        return 0;
    }
    LRESULT OnScClose(UINT, INT, HWND)
    {
        PostMessage(WM_COMMAND, IDCANCEL);
        return 0;
    }

Implementation Details

The very first question is embedding of an existing filter into new filter. The two most common methods are:

• COM aggregation
• embedding a fully featured graph with a sink/renderer that intercepts media samples downstream and forwards to a higher level filter so that it streams them as a source filter

The COM aggregation methods is much easier in implementation but it is subject to a few constraints, the two most important of which are:

• embedded filter should support instantiation as an aggregated object
• it is the only underlying filter, not a chain of filters, which can produce required data

COM aggregation is quite fitting for the purpose, so the second embedding method is being left for another topic (with a certain luck to be appear very soon, a DirectShow video capture source filter for a real network/IP camera).

The next step is a check of sufficient implementation in an underlying filter. Obviously, a video source that pretends to be a live video capture source needs an endless stream of media samples, while original implementation streams JPEG files as media samples once only, we need an option to loop the streaming and automatically repeat the sequence.

Playback looping is added to the original JPEG Multi File Source Filter and its controlling private interface IJpegMultiFileSourceFilter received additional properties:

interface IJpegMultiFileSourceFilter : IDispatch
{
...
    [propget, id(2)] HRESULT AutoRepeat([out, retval] VARIANT_BOOL* pbAutoRepeat);
    [propput, id(2)] HRESULT AutoRepeat([in] VARIANT_BOOL bAutoRepeat);
    [propget, id(3)] HRESULT RepeatDelay([out, retval] LONG* pnRepeatDelay);
    [propput, id(3)] HRESULT RepeatDelay([in] LONG nRepeatDelay)

as well the new property page:

As soon as the underlying implementation is ready, it is the right time to get down to the immediate filter COM class. It will be require to implement certain interfaces, to expose certain interfaces of the aggregated object and possibly to hide certain interfaces of the aggregated object. However, the first thing to do is to instantiate aggregated object, for which the right place is ATLs’ FinalConstruct:

class ATL_NO_VTABLE CFilter :
...
{
...
DECLARE_PROTECT_FINAL_CONSTRUCT()

DECLARE_GET_CONTROLLING_UNKNOWN()
...
    CComPtr<IUnknown> m_pInnerUnknown;
    CComPtr<IBaseFilter> m_pInnerBaseFilter;
    CComPtr<IPersistStream> m_pInnerPersistStream;
...
    HRESULT FinalContruct()
    {
...
        __C(m_pInnerUnknown.CoCreateInstance(__uuidof(JpegMultiFileSourceFilter), GetControllingUnknown()));
        CComQIPtr<IJpegMultiFileSourceFilter> pJpegMultiFileSourceFilter = m_pInnerUnknown;
        __D(pJpegMultiFileSourceFilter, E_NOINTERFACE);
...
        __C(pJpegMultiFileSourceFilter->put_Directory(CComBSTR(sDirectory)));
        __C(pJpegMultiFileSourceFilter->put_AutoRepeat(ATL_VARIANT_TRUE));
        __C(pJpegMultiFileSourceFilter->put_RepeatDelay(nRepeatDelay));
...
        m_pInnerBaseFilter = CComQIPtr<IBaseFilter>(m_pInnerUnknown);
        m_pInnerPersistStream = CComQIPtr<IPersistStream>(m_pInnerUnknown);
        __D(m_pInnerBaseFilter, E_NOINTERFACE);

The aggregated (inner) object is created and its “main” IUnknown pointer is saved as lifetime reference (because for the COM aggregated object this is the only interface pointer which AddRef/Release has effect on internal reference counter – see COM Aggregation details). The object is pre-initialzed immediately after creation and the effective values will be taken from registry key “SOFTWARE\Alax.Info\Media Tools\JPEG MultiFile Video Capture Filter” (see below). It also makes sense to pre-query most used object’s interfaces to be able to use them directly without querying for them each time they are required.

Once the inner object is here, we need to sort out the list of interfaces to support. Video capture source filters have specific requirements (see Writing Capture Filters), including indication of pin category and implementation of IKsPropertySet interface; optional availability of preview pin, which we can omit from implementation and let Filter Graph Manager use Smart Tee Pin Filter to split stream into preview and capture.

Additionally, various software is expecting IAMStreamConfig interface on the output pin and would just fail to use the video capture device in case of its absence. We don’t have this interface implemented on the underlying filter, so we have to implement this on the wrapper. Since we need additional interfaces on the pin and the pin is an owned object of an inner aggregated object, it adds complexity to wrap original pin with a private pin class. An easy implementation of the latter point is not quite safe with COM aggregation, however a well written software that treats IPin interface as the “primary” pin’s interface and queries additional from it and holds the reference while using other interfaces will do just fine with a simple implementation on our side.

Another problem with the underlying filter which actually came up later but worth mentioning from the start. A JPEG Multi File Source Filter streams video media sample with an empty format structure attached to AM_MEDIA_TYPE media type. This assumes that an instance of JPEG Frame Decoder Filter will be added downstream and supply missing format by decoding JPEG data. While the filter is flexible enough to dynamically change media type along with JPEG data changes, this is not compatible with (not implemented by) most of the software. JPEG Frame Decoder Filter makes a first guess by settings resolution to 320×240. We need to set the right resolution immediately to avoid media type re-agreement.

To be able to do so we need to pass resolution information from original (inner) filter to the JPEG Frame Decoder Filter instance. It is also important that the filters might have other filters connected in between and in most cases there will be at least Smart Tee Pin Filter. To let the filters effectively communicate between each other and including in a reliable and documented way, available also to other/future filters, JPEG Frame Decoder Filter inroduces a new interface IJpegFrameDecoderFilterSource which, on its input pin connection, it looks for among the upstream filters. Iteratively walking through upstream connections up to developed video capture source filter it reaches our implementation and is capable of changing the default resolution to the effective resolution and exposed correct media type on its output pin right from the start.

So the interfaces on the filter:

• implemented on the filter:
  • IBaseFilter with IMediaFilter and IPersist to provide our own pin enumeration and supply a wrapper class for a pin object
  • IPersistStreamInit and IPersistStream to be friendly with persistence-enabled applications
  • IAMovieSetup to be able to be registered with DirectShow the way it is normally done
  • IFilter without IDispatch is the filters private interface with IDispatch not exposed directly to let the underlying filter expose his
  • IJpegFrameDecoderFilterSource to communicate to downstream JPEG Frame Decoder Filter and provide resolution information
• hidden implementation on the inner filter (other interfaces implemented by inner filter and not mentioned will be also available “outside”):
  • ISpecifyPropertyPages to hide inner property pages since the inner object is initialized internally
• implemented on the pin:
  • IPin to intercept method calls to possibly override media type enumeration and pin connection condition; while implementation of its methods (appeared to be not necessary, we still have to expose our private implementation of this interface since software will in most cases query additional interfaces from this interface pointer
  • IKsPropertySet per MSDN requirement
  • IAMStreamConfig to be compatible with wider range of software, specifically AMCap

Implementation details is available in reference implementation Filter.h in SVN.

An updated version of the dependent binaries Acquisition.dll and CodingI.dll is available from SVN.

A few points to mention explicitly:

• IKsPropertySet::Get marks the pin as a capture pin (PIN_CATEGORY_CAPTURE)
• IAMStreamConfig minimal implementation ignores media type in SetFormat; returns the only available media type in GetFormat using 300 fps frame rate value in order for it to be not zero and not very low to avoid collision of frames; returns one video capability in GetNumberOfCapabilities and GetStreamCaps – this should be sufficient provided that we actually don’t accept any cahnges/configuration on this interface
• IBaseFilter::EnumPins exposes a private wrapper pin class over original pin, so that we are capable of providing additional interfaces off the exposed output pin; we should have also provided an equal replacement through IBaseFilter::FindPin but this method is actually very rarely used and appears to be safe to omit, probably unless a compatibility issue arises later
• IJpegFrameDecoderFilterSource::GetDefaultExtent is the way JPEG Frame Decoder Filter obtains original video resolution.

On initialization the filter will query registry key “SOFTWARE\Alax.Info\Media Tools\JPEG MultiFile Video Capture Filter” for initialization values. Both HKEY_LOCAL_MACHINE and HKEY_CURRENT_USER might be used, HKEY_CURRENT_USER has the priority if present, unless HKEY_LOCAL_MACHINE has an additional non-zero REG_DWORD value named “Force”, in which case it takes priority over HKEY_CURRENT_USER. Initialization values are:

• “Directory” REG_SZ is the directory containing JPEG files to stream from
• “Repeat Delay” REG_DWORD is the pause in milliseconds between last and first frame when looping when playback
• “Video Width” and “Video Height” REG_DWORD is original video resolution and should match those of JPEG files

Another necessary implementation stroke is related to time stamps. The original inner JPEG Multi File Source Filter did not do any frame rate control at which the media samples are sent downstream. It was assumed that [video] renderer filter will enforce media sample presentation time and add necessary delay while playing video frames back. However, this approach does not work in a capture filter: when previewing video, a Smart Tee Pin Filter, which is inserted to make preview samples out of capture samples, is stripping time stamps and video renderer renders media samples at full possible rate. To address this issue, the JPEG Multi File Source Filter is received a new property Rate, which defaults to zero and implements uncontrolled sending of data downstream. With a Rate property value of 1.0, the filter does rate control and adds necessary delay as if the media samples are captured in real time.

Compatibility

Graph Edit and Graph Studio list, insert and are otherwise compatible with the new filter. They don’t have any assumptions on the filter implementation except the very basic, so they are quite capable of manipulating the new filter.

AMCap Sample recognizes the capture source and is rendering the filter through Capture Graph Builder, which automatically inserts Smart Tee Pin Filter. The application is properly showing the video from the new filter.

Media Player Classic’s “Open Device…” menu command is capable of connecting to the new filter and the application is showing video.

VideoLAN [unexpectedly] failed to receive video from the new filter. It appears that the application is enumerating supported media types and is only capable of accepting those handled directly by the application. JPEG video with FOURCC code ‘AIJ0′ is not on that list and the application does not make any attempt, that a proper application would do, to render the filter through intermediate codecs using Intelligent Connect. The application won’t be able to receive video from the filter before the latter implements decoding into a well known pixel format, such as for example RGB or YUY2.

Skype beta 4.0.0.176 failed to receive video from the filter, though the filter is seen by the software as a video source. The logs don’t indicate the reason and it seems that the application does not like video capabilities and stops trying to get video from the device.

Summary

• Virtual DirectShow camera implemented
• Supported by Graph Edit, Monogram Graph Studio (one has to re-enter file directory from GUI when adding a filter), AMCap, Media Player Classic
• Does not integrate into VideoLAN because of its [unexpectedly] limited capabilities
• Does not integrate into Skype for whatever reason, probably just another of a number of bugs in a beta version

A partial reference Visual C++ .NET 2008 source code is available from SVN, release binary included.