DirectShow Spy: Memory Allocator Properties

A small update to the DirectShow Spy today: DirectShow Filter Graph Spy prints memory allocator properties as a part of graph topology trace on transition to running state. Why is that and what it is for? Filters normally agree on allocator properties (ALLOCATOR_PROPERTIES, obtained from IMemAllocator, obtained from IMemInputPin) themselves without interference from controlling application. Sometimes an undesired choice of buffers can cause sick runtime behavior, including but not limited to the following:

  1. live audio capture buffers are too long, and introduce significant latency, including from live video capture stream taking place in parallel; controlling application might need to take advantage of IAMBufferNegotiation::SuggestAllocatorProperties and request shorter buffers.
  2. a filter, such as for example DMO Wrapper Filter, may default to 1 buffer on allocator, which means that if a buffer reference is for some reason held for future reference (e.g. a video filter might be holding a reference to be able to re-push a video sample if an application is requesting video update), the entire streaming might lock.
  3. some filters are requesting unreasonably many/large buffers and consume vast spaces of RAM, such as MainConcept MXF Demultiplexer requesting 200 buffers x 640 kilobytes each (128 MB in total out of sudden)
  4. some filters are requesting unreasonably few/small buffers resulting in inability to pre-buffer data

In a chase for answers to questions “Where is my memory?”, “Why is it so choppy?”, “I would really appreciate a nice lipsync” and to troubleshoot the mentioned scenarios it is helpful to understand buffering configuration. DirectShow Filter Spy is here to deliver this information. Once the graph is put into running state, spy prints out topology data into log file (which is in most cases C:\ProgramData\DirectShowSpy.log):

Pin 2: Name "Input 01", Direction "Input", Peer "Tee 0x087A5AF0.Output2"
 Connection media type:
 majortype {73646976-0000-0010-8000-00AA00389B71}, subtype {31435641-0000-0010-8000-00AA00389B71}, pUnk 0x00000000
 bFixedSizeSamples 0, bTemporalCompression 0, lSampleSize 1
 formattype {E06D80E3-DB46-11CF-B4D1-00805F6CBBEA}, cbFormat 170, pbFormat 0x07c46fc0
pbFormat as MPEG2VIDEOINFO:
 rcSource { 0, 0, 0, 0 ), rcTarget { 0, 0, 0, 0 }
 dwBitRate 0, dwBitErrorRate 0, AvgTimePerFrame 0
 dwInterlaceFlags 0x0, dwCopyProtectFlags 0x0, dwPictAspectRatioX 16, dwPictAspectRatioY 9, dwControlFlags 0x0
 bmiHeader.biSize 40, bmiHeader.biWidth 1280, bmiHeader.biHeight 720, bmiHeader.biPlanes 1, bmiHeader.biBitCount 24, bmiHeader.biCompression avc1
 bmiHeader.biSizeImage 0, bmiHeader.biXPelsPerMeter 1, bmiHeader.biYPelsPerMeter 1, bmiHeader.biClrUsed 0, bmiHeader.biClrImportant 0
 dwStartTimeCode 0x00000000, cbSequenceHeader 38, dwProfile 100, dwLevel 31, dwFlags 0x4
 [0x0000] 00 1D 67 64 00 1F AC 24 88 05 00 5B BF F0 00 10
 [0x0010] 00 11 00 00 03 03 E8 00 00 E9 BA 0F 18 32 A0 00
 [0x0020] 05 68 EE 32 C8 B0
 Memory Allocator: 1 buffers, 1,024 bytes each (1,024 bytes total), align 1, prefix 0

Partial Visual C++ .NET 2008 source code is available from SVN, release binary included (Win32, x64); installation instructions are in another post.

ATL ActiveX control hosting memory leak, _freea assertion failure and more

If you were adding and removing ActiveX controls programmatically with ATL, you might have noticed a nasty memory leak after final release of the control. Control added, removed and a leaky block is left om heap.

Tracking simple memory leaks is not a rocket science with Microsoft C++ runtime, by adding _CRTDBG_MAP_ALLOC define, as described in Memory Leak Detection Enabling on MSDN:

#if defined(_DEBUG)
#define _CRTDBG_MAP_ALLOC
#include <stdlib.h>
#include <crtdbg.h>
#endif // defined(_DEBUG)

Having that done, the problem was isolated to an extra instance of CAxHostWindow, which was excessively instantiated and never freed. Luckily, having this done you already have sufficient amount of keywords on hands to locate relevant articles on Microsoft knowledge base:

Basically, what is taking place is the behavior by design. That is, AtlAxCreateControlEx API is leaky by design, one gotta just be aware.

CComPtr<IUnknown> pContainerUnkonwn, pControlUnknown;
static LPCTSTR g_pszControlClass = _T("{8856F961-340A-11D0-A96B-00C04FD705A2}");
// NOTE: This results in two instances of CAxWindowHost being created, one of which appears to be leaked (see below), 
//       see http://support.microsoft.com/kb/218442
// 
// Detected memory leaks!
// Dumping objects ->
// {81} normal block at 0x00128928, 244 bytes long.
// Data: <x .       .     > 78 09 2E 01 01 00 00 00 0C 10 2E 01 00 00 00 00 
// Object dump complete.
ATLENSURE_SUCCEEDED(AtlAxCreateControlEx(CT2COLE(g_pszControlClass), m_hWnd, NULL, &pContainerUnkonwn, &pControlUnknown));

Continue reading →

Recursive SRW Locks

Windows Vista added new synchronization API, Slim Reader/Writer (SRW) Locks, which is a powerful alternative to critical sections. The detailed description is, as always on MSDN, and what makes it really cool is simple:

  • unlike critical sections, SRW Locks provide reader and writer access synchronization making it possible for 2 and more reader to not block one another
  • SRW Locks do not reference any resources and have size of a pointer, which is the simplest possible scenario; as a result, they don’t need a destructor and their initialization is simple zeroing of memory/variable (for which you however should use InitializeSRWLock API

Being lightweight they are efficient. To understand how at all they can work, one can imagine that a reader might be trying to InterlockedIncrement a synchronization variable. If result is positive, then it’s OK to go. Otherwise, reader should decrement it back, wait and retry. A writer, instead, does InterlockedAdd with an argument of -0x1000 and checks that result of the operation is exactly -0x1000.

This post is about a trap one cat enter into by neglecting one of the SRW lock warnings:

… so SRW locks cannot be acquired recursively. In addition, a thread that owns an SRW lock in shared mode cannot upgrade its ownership of the lock to exclusive mode.

SRW locks cannot be acquired recursively, but it is very easy to make a mistake. If you attempt to recursively acquire, you are likely to succeed, without a warning, error code, exception or assertion failure. You pass this point and you can write quite some code before you realize something is wrong.

It can be as simple as this:

Continue reading →

Website unavailability

Unfortunately, the hosting provider EMC Web Hosting decided to stop service without prior notice and the website was removed without a chance to take a backup and files. Google Cache shows that there was an announcement on the website, if they could at least send an email prior to going offline it would be great.

The website was restored from backup and lost all linked content such as [small amount of] downloads and embedded images. Sorry for inconvenience.

Nov 10, 2010

[…]

All EMC Webhosting servers & network equipment will be discontinued this Friday. All customers that are still hosted on the EMC Webhosting networks are advised to remove any and all accounts off network to another hosting environment prior to this: Friday, November 12, 6pm PST

[…]

Sorry for the late notice. Not our doing, however, we at least wanted to ensure you had some notice prior to the event, so you don’t lose your data.

How to get HMODULE of msvcr90.dll

Because Microsoft C++ Runtime library is distributed as a side-by-side assembly, GetModuleHandle and LoadLibrary is no longer available to locate the library within the process. Both API calls would return NULL handle.

HMODULE hModule = GetModuleHandle(_T("msvcr90.dll"));
HMODULE hModule = LoadLibrary(_T("msvcr90.dll"));

Still, how to locate the modules in order to, for example, be able to check DLL version? Process Status API (PSAPI) can help:

#include <psapi.h>
#pragma comment(lib, "psapi.lib")

...

DWORD nDataSize = 4 << 10; // 4K
EnumProcessModules(GetCurrentProcess(), NULL, 0, &nDataSize);
nDataSize += nDataSize / 2;
CTempBuffer<HMODULE> phModules;
ATLENSURE_THROW(phModules.AllocateBytes(nDataSize), E_OUTOFMEMORY);
if(EnumProcessModules(GetCurrentProcess(), phModules, nDataSize, &nDataSize))
{
    const SIZE_T nCount = nDataSize / sizeof *phModules;
    for(SIZE_T nIndex = 0; nIndex < nCount; nIndex++)
    {
        CPath sPath = _VersionInfoHelper::GetModulePath(phModules[nIndex]); // uses GetModuleFileName
        if(_tcsicmp(FindFileName(sPath), _T("msvcr90.dll")) == 0)
        {
            // NOTE: Here we found it
            ATLTRACE2(atlTraceGeneral, 2, _T("msvcr90.dll version is %s\n"), _VersionInfoHelper::GetVersionString(_VersionInfoHelper::GetFileVersion(sPath)));
            break;
        }
    }
}

Adobe Flash Media Live Encoder 3.1

It is the first time ever – because probably I am not as experienced as Geraint – I witness availability of IMediaSample interface without availability of IMediaSample2. One might be curious what kind of software could provide such a weirdo in 2010? It is latest and greatest Adobe Flash Media Live Encoder 3.1.

Additionally to this, they decided to provide garbage in AM_MEDIA_TYPE::formattype field of IAMStreamConfig::SetFormat. They perhaps learned that standard Audio Capture Filter will ignore it anyway, so why bother then?

Having fixed that, Tone Source Filter based virtual audio device is now compatible with Adobe Flash Media Live Encoder 3.1 and it is possible to send audio to remote Flash Media Server, e.g. such as Ustream.tv service (over RTMP protocol, as implemented by FMLE).

24-Image002

Attributed ATL: Accessing BLOB with ISequentialStream

Before attributed ATL was deprecated, it was a convenient way to access databases using attributed classes on top of OLEDB Consumer Templates. Does not it look nice?

[
    db_command("SELECT ServerData FROM Server WHERE Server = ?")
]
class CGetServerData
{
public:
    [ db_param(1) ] LONG m_nServer;
    [ db_column(1, length = "m_nDataLength") ] ISequentialStream* m_pDataStream; DBLENGTH m_nDataLength;
};

It worked great with Visual Studio .NET 2003 and it failed to work with later releases. There are questions on internet about the problem, but there few answers if any. As I recently had to convert a project from 2003 version of the compiler to Visual Studio 2008, the problem was finally to be resolved.

Continue reading →