Previous post on thread synchronization and context switches used number of thread context switches as one of the performance indicators. One might have hard times getting the number from operating system though.

The only well documented access to amount of context switches seems to be accessing corresponding performance counters. Thread performance counter will list available thread instances and counters “Thread(<process-name>/<thread-number>)/Context Switches/sec” will provide context switch rate per second.

While access to performance counters is far not the most convenient API, to access data programmatically one would really prefer absolute number of switches rather than rate per second (which is still good for interactive monitoring).

A gate into kernel world to grab the data of interest is provided with NtQuerySystemInformation function. Although mentioned in documentation, it is marked as unreliable for use, and Windows SDK static library is missing it so one has to obtain it using GetModuleHandle/GetProcAddress it explicity.

typedef NTSTATUS (WINAPI *NTQUERYSYSTEMINFORMATION)(SYSTEM_INFORMATION_CLASS SystemInformationClass, PVOID SystemInformation, ULONG SystemInformationLength, PULONG ReturnLength);
NTQUERYSYSTEMINFORMATION NtQuerySystemInformation = (NTQUERYSYSTEMINFORMATION) GetProcAddress(GetModuleHandle(_T("ntdll.dll")), "NtQuerySystemInformation");
ATLASSERT(NtQuerySystemInformation);
ATLVERIFY(NtQuerySystemInformation(...) == 0);

Having this done, the function is capable of providing SystemProcessInformation/SYSTEM_PROCESS_INFORMATION data about running processes.

MSDN documents only a part of the structure, and the rest of the data contains information of interest. Returned data contains a list of variable length SYSTEM_PROCESS_INFORMATION structures, each of which has a fixed part (partially documented on MSDN), and embeds list of variable length structures for every thread running within the process.

For 32-bit Win32 platform, layout of the structure is documented on Internet, for example, here: SYSTEM_THREAD on Undocumented functions of NTDLL.

For 64-bit x64 (amd64) platform the access method works too, however I had to fit the structure layout so that it matches actual data:

typedef struct _SYSTEM_THREAD_INFORMATION {
    ULONGLONG KernelTime;
    ULONGLONG UserTime;
    ULONGLONG CreateTime;
    ULONG WaitTime;
    ULONG Reserved1;
    PVOID StartAddress;
    CLIENT_ID ClientId;
    KPRIORITY Priority;
    LONG BasePriority;
    ULONG ContextSwitchCount;
    ULONG State;
    KWAIT_REASON WaitReason;
} SYSTEM_THREAD_INFORMATION, *PSYSTEM_THREAD_INFORMATION;

typedef struct _SYSTEM_PROCESS_INFORMATION {
    ULONG NextEntryOffset;
    ULONG NumberOfThreads;
    ULONGLONG Reserved[3];
    ULONGLONG CreateTime;
    ULONGLONG UserTime;
    ULONGLONG KernelTime;
    UNICODE_STRING ImageName;
    KPRIORITY BasePriority;
    HANDLE ProcessId;
    HANDLE InheritedFromProcessId;
    ULONG HandleCount;
    ULONG Reserved2[2];
    ULONG PrivatePageCount;  // Garbage
    VM_COUNTERS VirtualMemoryCounters;
    IO_COUNTERS IoCounters;
    SYSTEM_THREAD_INFORMATION Threads[1];
} SYSTEM_PROCESS_INFORMATION, *PSYSTEM_PROCESS_INFORMATION;

Each structure contains thread identifier and related absolute number of context switches.

Also, the same information is visually available from Process Explorer:

A more complete code snippet to access the data from x64 code is here in a small application/project, lines 18-144 within #pragma region.

A basic task in thread synchronization is putting something on one thread and getting it out on another thread for further processing. Two or more threads of execution are accessing certain data, and in order to keep data consistent and solid the access is split into atomic operations which are only allowed for one thread at a time. Before one thread completes its thing, another is not allowed to touch stuff, such as waiting for so called wait state. This is what synchronization objects and critical sections in particular for. Furthermore, a thread which is waiting for stuff to be available has nothing to do, so it uses one of the wait functions to not waste CPU time, and both threads are using event or similar objects to notify and receive notifications waking up from wait state.

Let us see what is the cost of doing things not quite right. Let  us take a send thread which is generating data/events which is locking shared resource and setting an event when something is done and requires another receive thread to wake up and take over. Send thread might be doing something like:

CComCritSecLock<CComAutoCriticalSection> DataLock(m_CriticalSection);
m_nSendCount++;
ATLVERIFY(m_AvailabilityEvent.Set());

And receive thread will wait and take over like this:

CComCritSecLock<CComAutoCriticalSection> DataLock(m_CriticalSection);
m_nReceiveCount++;
ATLVERIFY(m_AvailabilityEvent.Reset());

Let us have three send threads and one receive thread running in parallel:

The simplicity is tempting and having run this the result over 60 seconds is:

Send Count:        280,577,239
Receive Count:         703,238

Process Time:        User     14,539 ms, Kernel     60,746 ms
Send Thread 0 Time:  User      5,584 ms, Kernel     22,666 ms, Context Switches       5,449,926
Receive Thread Time: User        811 ms, Kernel      2,776 ms, Context Switches       8,836,263

One of the send threads performed 280M cycles (note the number is for one thread of the three, and also test code involves random delays so amount varies from execution to execution), while receive thread could only do 703K. The huge difference is definitely related to the behavior around critical section and thread wait time.

All four threads are locking critical section and making other threads to wait. However send threads are only setting the event, while receive thread is waiting for event and wake up on (that is, soon after) its setting. Once receive thread wakes up, its trying to enter critical section and acquire lock. Still, send threads are setting the event before leaving critical section, so if things happen too fast and receive thread tries to enter locked critical, it will fail up to eventually giving its time slice away doing a context switch.

This explains low number of iterations of the receive thread together with rather high amount of context switches.

To address this problem let us modify sending part to set event from outside of critical section lock scope.

{
    CComCritSecLock<CComAutoCriticalSection> DataLock(m_CriticalSection);
    m_nSendCount++;
}
ATLVERIFY(m_AvailabilityEvent.Set());

This makes sense too, event itself does not need to be signaled from inside protected fragment, sending thread might be doing this a bit later, with a small but acceptable chance that another send thread will set the event between leaving critical section and setting the event on original thread. From the point of view of receive thread this means that sometimes it might be entering protected area with result of work of 2+ threads and sometimes the even is set again after all work is done and there is nothing more to do.

Result of execution is:

Send Count:        233,620,244
Receive Count:      10,901,974

Process Time:        User     35,006 ms, Kernel     90,808 ms
Send Thread 0 Time:  User     10,654 ms, Kernel     27,190 ms, Context Switches       7,879,948
Receive Thread Time: User      3,338 ms, Kernel     12,012 ms, Context Switches       8,021,273

Apparently, the change had a positive effect on receive thread, which was able to make 15 times more iterations.

After all, how it compares to scenario when no critical section is involved at all? If synchronization is implemented using an alternate technique (such as, for example, interlocked variable access).

Removing critical section, the result of execution is:

Send Count:        246,133,642
Receive Count:      24,743,638

Process Time:        User     16,723 ms, Kernel    225,515 ms
Send Thread 0 Time:  User      4,461 ms, Kernel     56,519 ms, Context Switches           1,353
Receive Thread Time: User      3,354 ms, Kernel     56,019 ms, Context Switches          17,372

About the same number of send thread iterations and twice as many on receive thread. In the same time, amount of context switches dramatically dropped and indicates that this way the threads do not have to fall into wait state and wait one another. The threads do still wait for stop event (with zero timeout, that is they just check for it without any need to wait), and receive thread keep synchronizing to availability event.

Having done that, let us do one more test to check the effect of entering/leaving a critical section which is never owned by another thread. If send threads keep running without locking  critical section, and receive thread is back to enter and leave it (on its own only, without congestions with other threads involved), how much slower the overall execution will be?

The results show that without concurrent access to the critical section, no extra context switches take place and the execution is similar to the one in previous test 3:

Send Count:        213,333,676
Receive Count:      18,542,098

Process Time:        User     15,022 ms, Kernel    227,496 ms
Send Thread 0 Time:  User      3,322 ms, Kernel     57,626 ms, Context Switches           2,786
Receive Thread Time: User      3,510 ms, Kernel     56,316 ms, Context Switches          10,616

Some conclusions:

• setting event for shared resource while still holding a lock of it might be pretty expensive
• critical sections are quite low-weight as promised and add minimal overhead unless congestion takes place
• critical sections might still create a bottleneck in case of intensive use on multiple threads, in which case alternatives such as slim reader/writer locks and interlocked variable access
• amount of thread’s context switches is a good indicator of execution congestions

Good luck with further research on synchronization and performance and here is source code for the test, a Visual Studio 2010 C++ project.

Obtaining information on amount of context switches for x64 code deserves a separate post, and the code is here, lines 18-144 within #pragma region.

ProcessSnapshot is a utility to take a snapshot of process call stacks, and the snapshot taken is written into a human friendly text file.

Additionally to this, the utility has been given a capability to create process minidump files, on user request. The minidump files can be used with debugger to analyze the context of the process using feature rich debug environment, esp. Microsoft Visual Studio. To create a minidump for a process, check a corresponding box and press “Take a Dump” button. A file named “<process-image-name> – <date> <time>.dmp” will be created in the directory of the utility executable.

See also:

• Minidump Files (MSDN)
• How to read the small memory dump files that Windows creates for debugging
• Post-Mortem Debugging Your Application with Minidumps and Visual Studio .NET
• How to View Windows Minidump Files

A binary [Win32, x64] and partial Visual C++ .NET 2008 source code are available from SVN.

Sharing memory allocators between input and output pins is an important concept to keep performance of filter graph. Unlike more frequent scenario with different allocators, a filter (referred to as “middle filter” below) which has equal media types on input and output pins has an advantage to avoid memory-to-memory copy operation for every frame processed, by delivering downstream the buffer obtained from an upstream filter. With a high resolution video, at high rate, multiple streams running simultaneously this is the expense one would try to avoid for performance reasons.

Memory allocators are (or can be) shared by well known filters, such as Sample Grabber Filter, Infinite Tee Pin Filter and in-place transformation base filters (CTransInPlaceFilter Class).

Still handling Dynamic Format Changes (not only from video renderer filter) filters that share memory allocators may run into the problem of being notified of media type change. Because allocator are typically owned by another filter (e.g. Video Mixing Renderer Filter) and originally its buffer is queried by an upstream filter, the upstream filter obtains allocated buffer independently from the middle filter that shares memory allocators. If the upstream filter decides to never deliver this buffer, however the buffer has a media type attached (see AM_SAMPLE2_PROPERTIES::pMediaType), there is no way for the middle filter to learn about dynamic format change completed.

As a workaround for handling Format Changes from the Video Renderer, when resolution is not changed and it is only stride which might be extended, middle filter might be checking data size in lActual field and learn about the change from an increase in this value.

To be reliably notified on media type change the middle filter is to take extra measures while sharing the allocator. Instead using raw allocator obtained from one pin on another pin (typically output pin’s allocator to be used on an input pin), middle filter may be using an internal proxy object, which implements IMemAllocator interface and forward calls to internal IMemAllocator, obtained originally. Additionally to that, the proxy can check for attached media types on every buffer taken from the allocator, and once the change is noticed – at the moment upstream filter is requesting the buffer – the proxy has a timely chance to remember the new media type so that in the following IMemInputPin::Receive call this media type can be checked for the case upstream buffer decided to not deliver the buffer with attached media type.

if(IsSharingMemAllocators())
{
    // ...
    ATLASSERT((InputMediaSampleProperties.pMediaType != NULL) ^ !(InputMediaSampleProperties.dwSampleFlags & AM_SAMPLE_TYPECHANGED));
    {
        CRoCriticalSectionLock DataLock(GetDataCriticalSection());
        const CObjectPtr<CProxyMemAllocator>& pInputProxyMemAllocator = m_pInputPin->GetProxyMemAllocatorReference();
        CMediaType pMediaType;
        if(pInputProxyMemAllocator && pInputProxyMemAllocator->GetDynamicallyChangedMediaType(pMediaType, TRUE))
        {
            m_pInputPin->SetMediaType(pMediaType);
            m_pOutputPin->SetMediaType(pMediaType);
            // ...
        }
    }
    if(InputMediaSampleProperties.pMediaType)
    {
        m_pInputPin->SetMediaType(InputMediaSampleProperties.pMediaType);
        m_pOutputPin->SetMediaType(InputMediaSampleProperties.pMediaType);
        // ...
    }
    DeliverMediaSample(pMemInputPin, pInputMediaSample);
}

While troubleshooting released application on remote production site, it is very useful to grasp a state of the process for further analysis. There are several scenarios in which the following information about process state is helpful:

• modules (DLLs) loaded into process and their versions
• threads and their call stacks
• process and thread performance

An utility ProcessSnapshot takes advantage of Debugging Tools API (dbghelp.dll – note the dialog also displays DLL version in the right bottom corner) and writes this helpful information to text file and it can also take a sequence of the snapshots to compare thread performance and/or stacks and check the difference.

The generated file is in the directory of the utility application and looks like:

Snapshot
  System Time: 10/14/2008 8:46:33 PM
  Local Time: 10/14/2008 11:46:33 PM

Performance
  Creation System Time: 10/14/2008 8:46:28 PM
  Kernel Time: 0.094 s
  User Time: 0.031 s

Modules

  Module: ProcessSnapshot.exe @00400000
    Base Address: 0x00400000
    Base Size: 0x0005b000 (372736)
    Name: ProcessSnapshot.exe
    Path: D:\Projects\Utilities\ProcessSnapshot\Release\ProcessSnapshot.exe
    Product Version: 1.0.0.1
    File Version: 1.0.0.125

  Module: ntdll.dll @7c900000
    Base Address: 0x7c900000
    Base Size: 0x000af000 (716800)
    Name: ntdll.dll
    Path: C:\WINDOWS\system32\ntdll.dll
    Product Version: 5.1.2600.5512
    File Version: 5.1.2600.5512
[...]

Threads

  Thread: 3824
    Base Priority: 8
    Creation System Time: 10/14/2008 8:46:57 PM
    Kernel Time: 0.063 s
    User Time: 0.031 s
    Call Stack
      ntdll!7c90e4f4 KiFastSystemCallRet (+ 0) @7c900000
      USER32!7e4249c4 GetCursorFrameInfo (+ 460) @7e410000
      USER32!7e424a06 DialogBoxIndirectParamAorW (+ 54) @7e410000
      USER32!7e4247ea DialogBoxParamW (+ 63) @7e410000
      ProcessSnapshot!00403f45 ATL::CDialogImpl<CMainDialog,ATL::CWindow>::DoModal (+ 67) [c:\program files\microsoft visual studio 9.0\vc\atlmfc\include\atlwin.h, 3478] (+ 28) @00400000
      ProcessSnapshot!00403b6f CProcessSnapshotModule::RunMessageLoop (+ 74) [d:\projects\utilities\processsnapshot\processsnapshot.cpp, 67] (+ 0) @00400000
      ProcessSnapshot!004049b9 ATL::CAtlExeModuleT<CProcessSnapshotModule>::Run (+ 17) [c:\program files\microsoft visual studio 9.0\vc\atlmfc\include\atlbase.h, 3552] (+ 0) @00400000
      ProcessSnapshot!004041c3 ATL::CAtlExeModuleT<CProcessSnapshotModule>::WinMain (+ 48) [c:\program files\microsoft visual studio 9.0\vc\atlmfc\include\atlbase.h, 3364] (+ 5) @00400000
      ProcessSnapshot!00434477 wWinMain (+ 5) [*d:\projects\utilities\processsnapshot\release\processsnapshot.inj:5, 14] (+ 0) @00400000
      ProcessSnapshot!00415058 __tmainCRTStartup (+ 274) [f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c, 263] (+ 27) @00400000
      !00360033

How exactly this can facilitate troubleshooting problems with software. Here are several scenarios:

• the applications shows an unexpected error message and it is desired to find out the position and call stack
• the application deadlocks and call stacks are required for further troubleshooting
• the application maxes out CPU load on one of the cores and the thread needs to be identified
• the applciation runs slowly and bottleneck thread is to be find out
• the application loads undesired third party module (or otherwise has it mapped into process, esp. antivirus software, or a DLL hosting undesired DirectShow filter) or a module with improper version

In all mentioned above scenarios the snapshot is very helpful for troubleshooting, profiling, fixing.

Update 23-Dec-2008. The application auto-enables SeDebugPrivilege (SE_DEBUG_NAME) so that snapshot could be taken from processes such as service processes.

A binary [Win32, x64] and Visual C++ .NET 2008 source code are available from SVN.

> How do you test performance?

I don’t. I just believe in it.

This is actually what we have here but still we have managed to deliver software that gives more frames per second than rivals. Why? We hopefully knew what we did in first place. According to one of our partner hardware vendors, there are only two software packages which could render multiple megapixel video feeds at the rates cameras can provide, the ours one and another one with the track leading to sources in Eastern Europe…

This continues the topic raised by previous post. As fairly noticed by The March Hare, video renderer is using hardware overlay and the benchmark is incorrect if we are to extrapolate the performance to scenario with multiple video renderers.

So, an updated test application creates 16 video renderers with 16 threads pumping two meida samples through each of the 16 filter graphs.

The screen shot shows that there is only one video overlay in use (which image was not captured and blackness is shown instead), so results may be inaccurate for one of the graph among 16. In this simple test I disregard this.

Here go the results (in all tests CPU usage is maxed out):

• YUY2 Source -> VMR: 3,480 fps
• YUY2 Source -> AVI Decompressor (converts to 24-bit RGB) -> Sample Grabber (without processing) -> Color Space Converter (converts to 32-bit RGB) -> VMR: 560 fps
• YUY2 Source -> AVI Decompressor (converts to 32-bit RGB) -> Color Space Converter -> VMR: 390 fps

For a comprehensive study, let us also measure direct rendering of 24-bit and 32-bit RGB data the way we stream YUY2:

It is worth mentioning that Video Mixing Renderer Filter does not like 24-bit RGB on input and intelligent connect auto-inserts additional Color Space Converter Filter to convert 24-bit RGB into 32-bit RGB.This explains better performance with 32-bit RGB.

• 24-bit RGB Source -> Color Space Converter (converts to 32-bit RGB) -> VMR: 1,175 fps
• 32-bit RGB Source -> VMR: 1,660 fps

Another popular YUV pixel format is YV12, which is also very much important as used as original output of MPEG-4 and H.264/MPEG-4 AVC decoders. It is also normally supported natively by hardware:

Performance tests repeat YUY2 frame rates with the difference for higher YV12 rates because of more compact 12 bit per pixel format (as opposed to 16 bit YUY2):

• YV12 Source -> VMR: 4,780 fps
• YV12 Source -> AVI Decompressor (converts to 24-bit RGB) -> Sample Grabber (without processing) -> Color Space Converter (converts to 32-bit RGB) -> VMR: 900 fps
• YV12 Source -> AVI Decompressor (converts to 32-bit RGB) -> Color Space Converter -> VMR: 1,510 fps

The difference of YV12 from YUY2 is that scenario with Sample Grabber Filter removed shows [expectedly] higher frame rates than with 24-bit RGB sample grabber.

Reference source code (will require additional headers to compile): FrameRateSample02.03.zip (note that Release build binary is included)

Started here: How can I overlay timestamp on the image? on microsoft.public.win32.programmer.directx.video

Let us see if RGB conversion adds any noticeable effect on streaming YUY2 video, typical output of video decompressor.

As a reference I am taking a simple YUY2 source -> Video Mixing Render Filter (VMR) graph, where source filter streams the same pre-allocated and pre-initialized data in an infinite loop:

while(WaitForSingleObject(TerminationEvent, 0) == WAIT_TIMEOUT)
{
	ATLENSURE_SUCCEEDED(m_pSourceFilter->InjectMediaSample(m_pnData, m_nDataSize));
	CRoCriticalSectionLock DataLock(m_DataCriticalSection);
	m_pnInjectedFrameCounts[0]++;
}

Video resolution is 640×480 pixels.

What is actually consuming CPU resources here is data copy into VMR’s media sample buffer and actually streaming. VMR might be blocking control waiting on rendering completion, I am leaving this for default VMR to decide (it might be hardware dependent etc).

Running at full pace, the application is rendering 510 frames per second consuming virtually no CPU. That is VMR is waiting until meida sample is rendered, this only allows streaming mentioned number of media samples per second, however rendering process does not take CPU resource, just waiting for video hardware to complete.

I am inserting Sample Grabber Filter to the graph, initialized with 640×480 24-bit RGB media type, between the source and the renderer. No callback, just a filter insertion to insist on media type. “Before” and “after” code:

#if TRUE && FALSE
	CComPtr<IBaseFilter> pSampleGrabberBaseFilter;
	ATLENSURE_SUCCEEDED(pSampleGrabberBaseFilter.CoCreateInstance(CLSID_SampleGrabber));
	CComQIPtr<ISampleGrabber> pSampleGrabber = pSampleGrabberBaseFilter;
	CMediaType pRgbMediaType;
	pRgbMediaType.AllocateVideoInfo(640, 480, 24);
	pSampleGrabber->SetMediaType(pRgbMediaType);
	ATLENSURE_THROW(pSampleGrabber, E_NOINTERFACE);
	ATLENSURE_THROW(pGraphBuilder->AddFilter(pSampleGrabberBaseFilter, CStringW(_T("24-bit RGB Sample Grabber"))));
	ATLENSURE_SUCCEEDED(pGraphBuilder->Connect(m_pSourceFilter->GetOutputPin(), _FilterGraphHelper::GetFilterPin(pSampleGrabberBaseFilter, PINDIR_INPUT)));
	ATLENSURE_SUCCEEDED(pGraphBuilder->Render(_FilterGraphHelper::GetFilterPin(pSampleGrabberBaseFilter, PINDIR_OUTPUT)));
#else
	ATLENSURE_SUCCEEDED(pGraphBuilder->Render(m_pSourceFilter->GetOutputPin()));
#endif

DirectShow intelligent connect is inserting two additional filters to the graph: AVI Decompressor Filter to convert YUY2 to RGB and Color Space Converter Filter for the VMR to have necessary upstream flexibility to choose a media type with extended stride.

Running still at full pace the application is only rendering 210 frames per second while CPU consumption jumped to 30%. What is consuming CPU cycles in the changed filter graph? The conversion from YUY2 to RGB in AVI Decompressor Filter and possible additional data copy between buffers.

Hardware:

• CPU: Intel® Core™2 Duo Desktop Processor E6750
• Video Adapter: ATI Radeon HD 3470

Reference source code (will require additional headers to compile): FrameRateSample01.01.zip (note that Release build binaries are included)

UPDATE: Another test to bring more detail into performance impact. I am taking Sample Grabber Filter out and let the graph run without it but the the auto-inserted filters, in order to remove effect of the Sample Grabber Filter itself.

CComPtr<IPin> pSampleGrabberInputPeerPin = _FilterGraphHelper::GetPeerPin(_FilterGraphHelper::GetFilterPin(pSampleGrabberBaseFilter, PINDIR_INPUT));
CComPtr<IPin> pSampleGrabberOutputPeerPin = _FilterGraphHelper::GetPeerPin(_FilterGraphHelper::GetFilterPin(pSampleGrabberBaseFilter, PINDIR_OUTPUT));
ATLENSURE_SUCCEEDED(pGraphBuilder->RemoveFilter(pSampleGrabberBaseFilter));
ATLENSURE_SUCCEEDED(pGraphBuilder->Connect(pSampleGrabberInputPeerPin, pSampleGrabberOutputPeerPin));

Frame rate is slightly higher than in previous test with Sample Grabber Filter but it does not keep constant: it keeps jumping between 210 and 250 fps with CPU load jumping between 10% and 50% (note 50% is a 100% load on one of the CPU cores out of the two).

What makes it different in comparison with previous run where Sample Grabber Filter is present? A check of pins’ media types with GraphEdit shows that filters decided to not connect on 24-bit RGB media type. Instead they are connecting on 32-bit RGB between AVI Decompressor Filter and Color Space Converter Filter. Previously when Sample Grabber Filter insisted on 24-bit RGB, Color Space Converter Filter performed additional data conversion from 24-bit RGB to 32-bit RGB because VMR chose to use 32-bit RGB to accept data in.

Updated binary for the third test: FrameRateSample01-RGB32.zip