DirectShowSpy: Restore default system behavior

There was a problem reported for registered and relocated DirectShowSpy, which might be causing issues: Deleting faulty DirectShowSpy registry key.

Some users that use a 3rd party tool called DirectShowSpy may encounter errors when logging in to XSplit.

This can be caused by a fault registry key that is introduced when DirectShowSpy is registered to intercept Filter Graph initialization — Filter Graph is used by XSplit. The faulty DirectShowSpy registry key is usually caused by DirectShowSpy program begin relocated after registration.

To workaround this situation, XSplit1 detects the presence of HKEYCLASSESROOT\CLSID{E436EBB3-524F-11CE-9F53-0020AF0BA770}\TreatAs registry key2 when it fails to initialize Filter Graph and exits when it is found. In this case, user must manually correct the DirectShowSpy registration or delete3 the registry key. Only after either is done can XSplit be restarted.

The description of the problem is good, solution is good but incomplete.

DirectShowSpy intercepts a few COM classes, not just one, and removing single registry value is only a partial fix.

DirectShowSpy.dll exports UnregisterTreatAsClasses function to accurately restore operation of system classes. It does registry permission magic and updates all COM classes involved. Default unregistration (DllUnregisterServer, regsvr32 /u) behavior is to restore original classes only in case they are currently overridden by DirectShowSpy. That is, if the DLL is moved (deleted) the broken registrations are retained in the registry during unregistration process.

UnregisterTreatAsClasses resolved this problem by forcing recovery of original classes no matter who is overriding them at the moment.

C:\>rundll32 DirectShowSpy-Win32.dll,UnregisterTreatAsClasses
C:\>rundll32 DirectShowSpy-x64.dll,UnregisterTreatAsClasses

DirectShowSpy: REGDB_E_CLASSNOTREG with IMMDevice::Activate

A DirectShow developer complained on sudden failure of Core Audio IMMDevice::Activate call supposed to instantiate a DirectShow filter for a given device.

The problem appeared to be related to installed DirectShowSpy and its interference with the API calls. The symptom was of the following kind: when Activate was called for different types of objects, the calls all succeeded except interoperation with DirectShow (activation for IBaseFilter), e.g. EnumerateAudioDevices output:

    IAudioClient            0x00000000
    IAudioEndpointVolume    0x00000000
    IAudioMeterInformation  0x00000000
    IAudioSessionManager    0x00000000
    IAudioSessionManager2   0x00000000
    IBaseFilter             REGDB_E_CLASSNOTREG
    IDeviceTopology         0x00000000
    IMFTrustedOutput        0x00000000

When Core Audio is requested to do DirectShow activation, the API creates and instance of System Device Enumerator, which is forwarded the activation call to. DirectShowSpy intercepts these calls, however what it did not do was support for unknown COM interfaces, and support for undocumented IMMDeviceActivator interface which is used internally by the APIs to forward the activation call.

So, System Device Enumerator implements documented ICreateDevEnum and then it also implements undocumented internal IMMDeviceActivator. The entire sequence call is as follows:

// Top level code:

CComPtr<IMMDevice> pDevice = ...; // Audio endpoint interface
pDevice->Activate(..., __uuidof(IBaseFilter), ...)

// API:

STDMETHOD(Activate)(...)
{
    // ...
    if(requested is IBaseFilter)
    {
        CComPtr<IMMDeviceActivator> pDeviceActivator;
        pDeviceActivator.CoCreateInstace(CLSID_SystemDeviceEnum);
        return pDeviceActivator->Activate(pDevice, ...)
    }

DirectShowSpy’s failure to provide IMMDeviceActivator resulted in symptom in question and is fixed with version 1.0.0.2106 and on. The failure code is not so much descriptive, but of course the APIs did not expect external hook and failure is not actually a supposed possible behavior there.

System Device Enumerator matches the known devices to the provided Core Audio device and creates an instance of respective filter – this is how APIs work together. DirectShowSpy prints these calls out to its log.

roatlbase.h(1582): TraceModuleVersion: "D:\...\DirectShowSpy-Win32.dll" version is 1.0.0.2107, running in "D:\...\EnumerateAudioDevices-Win32.exe" at 0x63210000
dllmain.h(36): CDirectShowSpyModule::CDirectShowSpyModule: this 0x633963A4
SystemDeviceEnumeratorSpy.h(669): CSystemDeviceEnumeratorSpyT<...>::CSystemDeviceEnumeratorSpyT: this 0x02F1DA68
SystemDeviceEnumeratorSpy.h(681): CSystemDeviceEnumeratorSpyT<...>::FinalConstruct: pszPath "D:\...\EnumerateAudioDevices-Win32.exe", this 0x02F1DA68, m_dwRef 1
SystemDeviceEnumeratorSpy.h(49): CSystemDeviceEnumeratorSpyT<...>::InternalQueryInterface: 0x02F1DA68, Interface {3B0D0EA4-D0A9-4B0E-935B-09516746FAC0}, Result 0x00000000
SystemDeviceEnumeratorSpy.h(49): CSystemDeviceEnumeratorSpyT<...>::InternalQueryInterface: 0x02F1DA68, Interface {3B0D0EA4-D0A9-4B0E-935B-09516746FAC0}, Result 0x00000000
SystemDeviceEnumeratorSpy.h(808): CSystemDeviceEnumeratorSpyT<...>::Activate: this 0x02F1DA68, InterfaceIdentifier {56A86895-0AD4-11CE-B03A-0020AF0BA770}, pMmDevice 0x0054E7F8
SystemDeviceEnumeratorSpy.h(815): CSystemDeviceEnumeratorSpyT<...>::Activate: nActivateResult 0x00000000 
SystemDeviceEnumeratorSpy.h(673): CSystemDeviceEnumeratorSpyT<...>::~CSystemDeviceEnumeratorSpyT: this 0x02F1DA68

Download links

DirectShowSpy: Automated Media Sample Traces, Load Trace from File

Some time ago DirectShowSpy received a capability to record media sample traces (in specific points of the pipeline). DirectShowSpy UI visualized the chronology of the streaming which facilitated detection of various problems, especially out of order media samples, lost or late delivered samples, thread racing conditions.

A developer can easily integrate his filters with DirectShowSpy media sample tracing, and example of this is a fork of MP4 filters where this powerful option is integrated.

This time DirectShowSpy is improved to offer automated traces. Typical use scenario is automatic creation of media sample trace file so that it could be logged/attached to specific export process and be available for further review. More specifically, an application could run a transcoding session and generate a detailed media sample trace alogn with the output in order to troubleshoot specific issues (dropped frame etc).

DirectShowSpy’s IFilterGraphHelper (implemented by FilterGraphHelper class) object received new additional methods:

interface IFilterGraphHelper : IDispatch
{
    [...]
    [id(7)] HRESULT ResetMediaSampleTrace([in] VARIANT vProcessIdentifier);
    [id(8)] HRESULT LockMediaSampleTrace([out, retval] IUnknown** ppLockUnknown);
    [id(9)] HRESULT GetMediaSampleTrace([in] VARIANT vProcessIdentifier, [out, retval] BSTR* psText);

ResetMediaSampleTrace clears currently recorded in-memory trace. The method is similar to pressing Backspace key in media sample trace UI. Optional argument allows cleanup for specific process (not implemented, everything is reset).

LockMediaSampleTrace creates a COM object which internally references newly created traces so that they don’t get freed with the respective filter graph release. Traces are owned by respective filter graphs and even though are accessible across process boundaries they are disposed with the release of graphs-creators. Unless anyone else accesses and references the traces, in which case they are available for review even is originating graph is already gone. This is the behavior of media sample trace UI, which is automatically referencing new traces as long as UI is visible. Developer can review traces for graphs already terminated (even crashed!). LockMediaSampleTrace method returns an interface pointer which owns an internals thread monitoring and referencing new traces. Once the interface pointer is releases, all referenced traces are released as well.

GetMediaSampleTrace obtains a copy of media sample trace for all or for specific process, in multi-line TSV format similar to text created by user interactively when data is copied to clipboard or saved into file with the help of UI.

All together, a controlling application can create a trace automatically using the following code:

#import "libid:B9EC374B-834B-4DA9-BFB5-C1872CE736FF" raw_interfaces_only // AlaxInfoDirectShowSpy

// ...

CComPtr<AlaxInfoDirectShowSpy::IFilterGraphHelper> pFilterGraphHelper;
pFilterGraphHelper.CoCreateInstance(__uuidof(AlaxInfoDirectShowSpy::FilterGraphHelper));
ULONGLONG nAlaxInfoDirectShowSpyFileVersion = 0;
CComPtr<IUnknown> pMediaSampleTraceLock;
if(pFilterGraphHelper)
{
    const CComQIPtr<AlaxInfoDirectShowSpy::IModuleVersionInformation> pModuleVersionInformation = pFilterGraphHelper;
    if(pModuleVersionInformation)
        _V(pModuleVersionInformation->get_FileVersion((LONGLONG*) &nAlaxInfoDirectShowSpyFileVersion));
    if(nAlaxInfoDirectShowSpyFileVersion >= _VersionInfoHelper::MakeVersion(1, 0, 0, 2060)) // Available in 1.0.0.2060+
    {
        _V(pFilterGraphHelper->ResetMediaSampleTrace(CComVariant((LONG) GetCurrentProcessId())));
        _V(pFilterGraphHelper->LockMediaSampleTrace(&pMediaSampleTraceLock));
    }
}

// DirectShow filter graph runs go here

if(pFilterGraphHelper && nAlaxInfoDirectShowSpyFileVersion >= _VersionInfoHelper::MakeVersion(1, 0, 0, 2060))
    _ATLTRY
    {
        CComBSTR sText;
        __C(pFilterGraphHelper->GetMediaSampleTrace(CComVariant((LONG) GetCurrentProcessId()), &sText));
        if(sText && *sText)
        {
            CPath sPath = Combine(m_sOutputDirectory, _T("MediaSampleTrace.tsv"));
            CAtlFile File;
            __C(File.Create(sPath, GENERIC_WRITE, FILE_SHARE_READ, CREATE_ALWAYS));
            // SUGG: UTF-8 BOM
            CStringA sTextA = Utf8StringFromString(CStringW(sText));
            __C(File.Write(sTextA, (DWORD) sTextA.GetLength()));
        }
    }
    _ATLCATCHALL()
    {
        _Z_EXCEPTION();
    }

Additionally, interactive UI for reviewing media sampla traces (DoMediaSampleTracePropertySheetModal DLL export, see respective BAT file) reviewed a new command to load external TSV file (such as generated with the help of code above!) for review:

DirectShowSpy Media Sample Trace

Once data loaded from file, disabled Refresh link indicates use of non-live data. Context menu command offers option to switch back to live data.

Download links

Microsoft Developer Web Service Day

MSDN Forums bug today earlier was nothing. Here is a new one with Microsoft Connect. A StackOverflow poster found a bug and created a feedback item with MS Connect:

MS Connect before Sign In

Available to anonymous, however invisible to authenticated user after sign in:

MS Connect after Sign In

The content that you requested cannot be found or you do not have permission to view it.

It should be a permission thing, I suppose.

Video for Windows API and Multiple Cameras

A StackOverflow question (already deleted) asked about use of indices when referencing Video for Windows (VFW) capture devices such as in capGetDriverDescription API and other. Video capture with Video for Windows allowed use of up to 10 devices (did anyone have that many at that time?). The numbering was API specific and at the latest stage the documentation described it as:

Plug-and-Play capture drivers are enumerated first, followed by capture drivers listed in the registry, which are then followed by capture drivers listed in SYSTEM.INI.

Even though it is a legacy API, and the API was really really simple and limited in capabilities, it still exists in all Windows versions, there is still some code running, and due to complexity of modern APIs some people still use it in VB.NET and C# projects.

There is, however, a trap involved if someone attempts to use multiple cameras using VFW. VFW drivers are no longer developed since… Let us see what VirtualDub says about dates and how ancient they are:

The newer type of video capture driver in Windows uses the Windows Driver Model (WDM), which was introduced in Windows 98 and 2000. The Microsoft DirectShow API is the primary API to use these drivers. Because the DirectShow API supports a larger variety of commands and settings than VFW, the functionality set of a WDM driver is significantly improved.

DirectShow is a much more complex API than VFW, however, and WDM-model drivers historically have been a lot less stable than their VFW counterparts. It is not unusual to see problems such as capture applications that cannot be closed, because their program execution is stuck in the capture driver. WDM is the proscribed driver model going forward, however, so the situation should improve over time.

All new drivers were WDM drivers for 15+ years. In order to provide backward compatibility later between VFW and WDM, Microsoft came out with Microsoft WDM Image Capture (Win32) driver. Windows versions up to Windows 10 include it, and it is the only VFW driver in the system. In turn, it manages one of existing WDM-driver devices of choice and exposes video capture functionality to VFW applications. If there are two or more WDM drivers, the VFW driver offers to choose between the devices.

VFW Capture Source Dialog

The screenshot displays a long standing bug with this driver: it offers choices of all registered DirectShow video capture devices (it enumerates CLSID_VideoInputDeviceCategory category) and reality is that it can only work with WDM devices and not other (more on this below).

VirtualDub has a mention of this driver as well:

If you have a Windows Driver Model (WDM) driver installed, you may also have an entry in the device list called Microsoft WDM Image Capture (Win32) (VFW). This entry comes from a Microsoft driver called VFWWDM32 and is a wrapper that allows WDM-model drivers to be used through the older Video for Windows (VFW) API. The WDM driver that is adapted can be selected through the Video Source driver dialog.

There are unfortunately some quirks in the way this adapter works, and some video capture devices will work erratically or not at all through this wrapper. Device settings not accessible through VFW will also still not be available when using it. If possible, use the capture device directly in DirectShow mode rather than using the VFWWDM32 driver.

This is works pretty nice with VFW API and applications. Even though the are all ancient and deprecated years ago, the system still has bridge to newer devices and applications can leverage their functionality. The problem is that there is only one VFW driver, and its index is zero. If you need two cameras you’re busted.

VFWWDM32 itself does not use any system exclusive resources and there is no reason why its different instances could not be configured with different WDM devices. However, VFWWDM32 is a simple old wrapper, either thread unsafe or such as implemented as singleton. People complain the operation with two cameras is impossible or unstable. It is still possible to run two different processes (such as, for example, VirtualDub) with two completely different VFWWDM32’s which do not interfere because of process boundary and run fine. WDM device is selected using capDlgVideoSource interactively, developers had hard time to do selection programmatically.

The interesting part is how VFWWDM32 does video capture using WDM. It is a cut corner in development: instead of doing simple DriectShow graph with Source –> Renderer, or Source –> Sample Grabber -> Renderer topology, where the wrapper would easily support all DriectShow video devices, including virtual, they decided to implement it this way:

VFWWDM Filter Graph

One-filter graph, where the filter is the WDM Video Capture Filter for the device in question.

  • the graph is CLSID_FilterGraphPrivateThread type, *FINALLY* it is found what this undocumented flavor of DirectShow filter graph is used for
  • source filter output pins are not terminated, not connected to anything else
  • the graph is never run, produces VFW output in stopped state

Instead, VFWWDM32 uses some private undocumented communication to the WDM filter internals to run the device and receive frames.

Bottom line: VFW is a backward compatibility layer now on top DirectShow. DirectShow and Media Foundation both use WDM drivers to access video capture devices. Artificial constrain caused by simplistic implementation of VFWWDM driver is a limit of one video camera per process at a time.

“… you will never get the same high quality video experience that you find with DirectShow”

Microsoft’s James Daily wrote back in 2011 (and it’s an incredible response in the public forum from MS guy – provided that DirectShow branch of the same forum did not see anything close for 10+ years) about how technologies relate one to another:

Hey, I’m the team’s DShow expert. Trevor asked me to take a look at your post and give my two cents. From looking at the DShow code that you are using in your winforms application I just want you to be aware that by including quartz.dll as a dependency you are using the DirectShow 8 OLE automation objects. These objects have been deprecated for years and are certainly not recommended [this might perhaps be not accurate enough because generally stuff in quartz.dll is not deprecated, it’s rather orphaned and yet waits it deprecation like related stuff from qedit.dll; however the overall attitude is about right – RR]. At this time Microsoft does not have a supported solution for calling DirectShow code from C# (or any managed language). Please see the “note” at the top of the page at the link below for documented confirmation of this. Because the technology is not supported from the winforms environment it is not possible for us to suggest a supported workaround from managed code.

That said it should be possible to facilitate the functionality that you are looking for by creating a custom EVR presenter. By using a custom presenter you can get direct access to the D3D surface. You can then use the standard D3D constructs to draw directly to the same D3D surface that the EVR is using to blit the video. There are two things to keep in mind about this solution. First you must code this solution in unmanaged C++. Again this is due to the fact that DirectShow is not supported from managed code. Second, this solution is extremely complex and difficult to implement even for the most experienced DirectShow / D3D expert. Because of these two factors it is recommended that you take a serious look at the MediaElement in WPF.

As you know the WPF environment is constructed from the ground up to offer developers a very rich “graphics first” environment. The MediaElement in particular was designed to allow you to mix video with various other UI components seamlessly. This solution will give you the flicker free, “draw over video” solution that you are looking for. The best part is you can do all of this in C#. The bad part of this solution is that the MediaElement is not designed for displaying time sensitive media content. In other words, the MediaElement is notorious for dropping and delaying the display of video frames. There are ways to minimize this such as using SD rather than HD content, use a video accelerated codec, etc. However, you will never get the same high quality video experience that you find with DirectShow.

I hope this will help you understand the current shortcomings of the technologies that you have chosen and help you to focus your efforts on a fully supported and viable solution. If you need any additional clarification please let us know.

and then also:

Unfortunately you can’t really tell the WPF MediaElement to never drop frames. The term we use for this class of issues is “disparate clocks”. In this case WPF is updating the screen at a certain rate (clock 1). The MediaElement (based on WMP) is cranking out video frames at a slightly different rate (clock 2). Given the underlying technologies there is currently no way to synchronize the two clocks and force them to “tick” at the same rate. Since the display will only be updated according to the WPF clock, multiple frames of video may be sent from the MediaElement to WPF between clock ticks. Because of this the MediaElement may appear to drop frames. This is a very common problem in multimedia development and there is no simple solution.

So if you absolutely need frame accuracy in your application then using the MediaElement probably won’t work for you. That said, there are some things that you can do to improve the chances of your content dropping as few frames as possible. Modify your content so that it uses either the h.264 or VC1 codec. Require your users to have modern video HW capable of advanced video acceleration. Use the MPEG 4 or ASF file container. When encoding your content set your frame rate at or below 25 frames per second. Set the resolution of your content to something like 720×480. Set the bitrate to VBR constrained and set an upper limit of between 500 Kbps and 2.5 Mbps.

If you use the guidelines above you will minimize the number of frames that get dropped but you will never be able to completely eliminate them. Also keep in mind that the same frames may not be dropped. For example: if you play video1.asf the first time you might drop frames 200 and 375. On the next run of the same file you may drop frames 143, 678 and 901. This is due to the relatively nondeterministic nature of the Windows OS.

I hope this helps.

Another commenter responded rather angrily:

…fail to include any mention of the DirectShow.NET library. Why? And shame on them for failing to do so. This library helps you use DirectShow in a managed context. There are plenty of code samples to be found….

The answer to this, however, was given in the same thread above a couple of times and explains that the responses are limited by existing policy:

I cannot comment on 3rd party libraries.

Because the technology is not supported from the winforms environment it is not possible for us to suggest a supported workaround…