CaptureClock: Utility to Check Video/Audio Capture Rates

Someone discovered the utility while browsing my public repository (the app prompts to post data back to the website, and the anonymous user accepted the offer and posted the report from this unpublished application), so I have to drop a few lines about the tool.

The idea is basically straightforward: live capture involves attaching time stamps to media samples, and there is a chance that the time stamps slide away causing unwanted effects on captured clip. The application captures video and audio simultaneously and tracks media sample time stamps, and compares them against system clock as well. Having it simply run for a few minutes one can see how the capture is doing and if any of the timings drift away. Being stopped it puts report onto clipboard and optionally posts it back to me online (no actually specific intent about this data, however if you want to share data for a device that does drift away, you are to only click once to send me the details).

CaptureClock operation

The output is on clipboard in tab-separated values (TSV) format:

Computer Name   PSI
Windows Version 6.1.7601 Service Pack 1
Video Device    Conexant's BtPCI Capture    @device:pnp:\\?\pci#ven_109e&dev_036e&subsys_18511851&rev_02#4&39c3dd91&0&08f0#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global
Audio Device    Stereo Mix (Realtek High Defini @device:cm:{33D9A762-90C8-11D0-BD43-00A0C911CE86}\Stereo Mix (Realtek High Defini

System Time Video Sample Count  Video Sample Time   Relative Video Sample Time  Audio Sample Count  Audio Sample Time   Relative Audio Sample Time
30439   907 30381   -57 304 30291   -147

Or you might prefer pasting it onto Excel:

CapptureClock Output on Excel

By the way, this is also an easy way to ensure devices are operational and check effective video frame rate.

Download links:

DirectShow Spy: Easier Registration

Because DirectShow Spy is often a troubleshooting tool, one of its use scenarios is its being a drop-in module to quickly install on a system of interest in order to connect to graphs for troubleshooting purposes, such as to check topology and media types.

Its installation requires COM registration, and over time it changed gradually from simple to more and more complex step. Why? In Windows XP one had to open command prompt and regsvr32. With Vista’s UAC one needs a prompt, with privilege elevation, which opens typically in wrong directory, then UAC prompt. A relatively easy step became annoying multi-step operation. Then proxy/stub classes were moved into Windows SDK DLL…

Things are getting back to be easier with DirectShow Spy. It is given a special property sheet right there withing the DLL, to take care of all the important things:

  • checks registration status
  • buttons to Unregister/Register
  • per-user registration (not recommended though due to system wide class hooking)
  • automatically takes care of UAC prompt
  • place Windows SDK proppage.dll into the same directory near spy, and additional property page will help you to register this additional dependency

To invoke registration UI, start:

rundll32 DirectShowSpy.dll,DoRegistrationPropertySheetModal

from command line, or just have a .BAT file ready to do it for you.

Download links

Log Process Exceptions: Filters and Email Notification

Moving on with LogProcessExceptions tool which externally monitors (debugs, to be more specific) an application of interest and captures its exceptions writing minidump files for further analysis.

This updates adds two features:

  1. Ability to filter out the exceptions of interest
  2. An email notification on exception, with or without minidump file

Filter

A new property page in the wizard provides one with an option to specify filters/rules to identify exceptions of interest. One rule per line, once exception occurred its codes are compared against rules top to bottom. The first matching rule results in decision whether to log the minidump (and issue a notification) or not. If no filter/rule applicable found, or the entire filter is empty, the exception is logged (positive).

The rule syntax is the following:

  • empty lines are ignored
  • anything to the right from // is considered to be a comment
  • meaningful line has one or more items separated by spaces
  • first item on the line is either - or + to indicate whether the rule will result in skipping the exception (minus sign) or logging in (plus sign)
  • second item is exception code, e.g. 0xC0000005 for memory access violation, 0xE06D7363 for native C++ exception (this constant has a dedicated alias “C++”
  • third item applies to C++ exceptions and matches ATL CAtlException codes, and this allows to filter out specific HRESULTs; it is also possible to specify a range of codes as shown on the screenshot above

Email Notification

Why? Some issues are long to wait, so you don’t want to have it in front of you until the issue takes place. This is when a friendly email from the application would be so much appreciated.

The application gets you that by queuing an email once exception of interest takes place. The email settings are basically those of class described on an earlier EmailTools.dll post. The whole SMTP email class is embedded into this application, and it actually deserves a dedicated post on explaining how to embed ATL COM class with persistence based on COM map, which in turn depends on type library expected to be registered, into another application which additionally runs without registration.

There is a default preset tuned for Google Mail, but there are other options as well. Note that both filter and email settings are saved into registry under HKCU to be reused in next runs of the application.

Small minidump files are attached right to the emails. Larger ones (typically those with Full Memory option checked) are only mentioned. There is a threshold setting that defines how many megabytes is OK to attach the most.

Download links:

LogProcessExceptions: Log Service Process Exceptions

One of the nasty issues with LogProcessExceptions utility was that it was unable to attach to service processes and track them to catch their exceptions.

The actual problem was that the processes were not listed in first place, so there was nothing to attach to. Access and security requirements necessary for a process to debug another process are listed in MSDN DebugActiveProcess article:

The debugger must have appropriate access to the target process, and it must be able to open the process for PROCESS_ALL_ACCESS. DebugActiveProcess can fail if the target process is created with a security descriptor that grants the debugger anything less than full access. If the debugging process has the SE_DEBUG_NAME privilege granted and enabled, it can debug any process.

The utility did enable the SE_DEBUG_NAME privilege, however it was doing it prior to starting debugging session and after the process of interest was already pointed to by user.

This was insufficient because EnumProcesses only lists service processes (not actually exactly services, but processes running in different security context) in case debug privilege is already enable by the time of the API call. The utility now enabled the privilege well in advance and list the services, so can be effectively applied to those.

Download links:

File Mappings: Virtual Memory and Virtual Address Space

More and more applications hit the Windows limit of available address space for 32-bit applications, and the whole concept becomes more important for understanding due to necessity to work things around.

A thing, which is more or less easy to understand, is that a user mode 32-bit application can address 2^32 addresses. The addresses are not directly physical RAM and the operating system is responsible for management of the mapping addresses into RAM as a part of virtual memory manager operation. Paged memory organization is well documented on MSDN, and the questions has been raised numerous times. An interesting question is whether a 32-bit application can effectively manage memory amounts exceeding address space limits.

Back in 80386 times, the systems could address megabytes of RAM in 16-bit code through XMS and EMS services. The application could access “high” memory addresses by requesting mapping portions of RAM into lower megabyte address space. In some way similar technique is also here for 32-bit applications in Windows through use of file mappings.

A regular memory backed file mapping requests Windows to reserve a memory block which becomes available for mapping into address space of one or more processes. Creating file mapping itself does not imply mapping and this leaves a great option for the owner to allocate more data than it can actually map into address space: if 32-bit process virtual address space is fundamentally constrained, the file mapping allocation space is more loosely limited by amount of physical memory and paging file. The application can allocate 2, 3, 4 and more gigabytes of memory – it just cannot still map it all together into address space and make it available simultaneously.

The FileMappingVirtualAddress utility does a simple thing:

  • on startup it allocates (CreateFileMapping) as many 256 MB file mappings as operating system would allow, and shows it in a list
  • each time a user checks a box, the application maps (MapViewOfFile) corresponding file mapping into address space; unchecking a box unmaps the view
  • the caption shows currently used and maximal available virtual address space

A plain 32-bit version of the application allocated 51 blocks for me (which totals in 13 GB of memory, with 8 GB physical RAM installed in the system). The allocation takes place immediately because the operating system does not actually make all this memory prepared for use – the actual pages would be allocated and ready to use on demand when the application requires them.

The most important part made so obvious is that the 32-bit application succeeds in allocating well over 4 GB, which is maximal virtual address space it can ever get.

The virtual address space in use is only 1641 MB and another request to map an additional section with MapViewOfFile would fail (the default address space limit is 2 GB) – space fragmentation make mapping unavailable earlier than we actually use the whole space, since the API would need to allocate contiguous range of addresses to satisfy the request.

32-bit application built with /LARGEADDRESSAWARE parameter might manage to do more allocations: 64-bit versions of Windows provide 4 GB of addresses to 32-bit processes. 32-bit operating systems might also be extending the limit in case of 4GB RAM Tuning (which would typically be 3 GB of space for a process).

Finally, 64-bit build of the application is free from virtual address space limit as the limit is 8 terabytes. The mapping is again instantaneous because actual RAM will be supplied on first request to mapped pages only.

A binary [Win32, Win32 with /LARGEADDRESSAWARE, x64] and partial Visual C++ .NET 2010 partial source code are available from SVN.

GPS Location/Coordinate Converter: Multiple Locations at Once

Today’s update lets you convert multiple locations at once with a single click. Here is the story behind the update and use case scenario.

In rally raid sport events (so called baja), a team gets a road book for the next competition day in a few hours before actual start. The GPS coordinates are printed on one of the pages of the roadbook and are not available in any electronic format.

There were just a few times when the organizer also uploaded a copy of a file with the coordinates and shared a link to download from, but this was rather an exception. Another alternate option was a dedicated person to upload the coordinates (they were earlier full tracks, but at some point tracks were no longer available at all) to pilots’ hardware, but in a state of pre-start рфыеу and variety of GPS hardware, formats, cable etc. this created lines of people. The most one can rely on is a sheet of paper with GPS coordinates. The mistery does not end even here as you don’t know whether you are to get Degrees only, or Degrees and Minutes, or Degrees, Minutes and Seconds. Everything depends on software the organizer uses.

As soon as you get a hard copy of this, the idea is to upload it into device as quickly as possible because there are other things to do and the time is normally 11 PM when the race is to start 7 AM next day tens of miles away from you. The time interval will be shared by uploading data, sleeping and transfer to start location.

The utility is here to grant extra sleep time. Since it is capable to accept various separators on the input, a convenient way is to quickly type in the text in Microsoft Excel, check the data against the hardcopy, and copy into clipboard to transfer to the utility.

A hotkey with conversion transfers data into format of interest, and single “Find and Replace” operation creates a good OziExplorer waypoint file which is good for upload onto portable navigation device.

The whole thing take a few minutes to do with minimal routine typing in.

A binary [Win32] and partial Visual C++ .NET 2010 partial source code are available from SVN.

Bonus picture, rally raid Suzuki is on the way to score the victory and the rally promotional teaser:

Rally Raid Suzuki Samurai on the Way

Common Controls: Versions, Compatibility, WTL

An application appears to be not working in Windows XP in a weird way: rebar control appeared to fail showing up.

Where the application is expected to look much nicer with rebar control as a container for menu (implemented as command bar WTL control) and toolbar with buttons:

A WTL sample project generated by a Visual Studio wizard would never give such effect, and the bug was a combination of factors:

  1. An application built with a newer version of Windows SDK, which includes support for features (Windows Vista+ Common Controls) that are more recent than production environment (Windows XP); the application targets to Windows Vista+ environment too (_WIN32_WINNT >= 0x0600)
  2. Compatibility issues of Common Controls library
  3. WTL version (7.5), which did not yet include a workaround for the problem

The problem, which caused the bug directly was the REBARBANDINFO structure and its use as an argument with Common Controls API. As MSDN shows, the structure was amended twice with additional fields.

One of the way to support multiple versions of the structure definition, and to resolve compatibility issues, is to embed structure size into structure payload. In fact, REBARBANDINFO::cbSize member is there exactly for this reason.

The application is normally filling cbSize with the maximal known structure size and fills the rest of the fields respectively. The API is expected to be checking cbSize member and be detecting API version compatibility scenarios:

  1. cbSize holds exactly the value the API expects (that is, the maximal value known/defined to the API) – the simplest scenario where the API and the application are on the same page, both are using the same versions of the “protocol”/interface.
  2. cbSize is smaller than API can support – the API sees that it is dealing with a sort of legacy application which cannot utilize all available features, and the API acts respectively supporting the older part of the protocol, and keeping defaults or “old look” for the rest of implementation. This addresses backward compatibility: the newer API works with apps designed for older version of the API
  3. cbSize is greater then API can support – the API sees that the application is already aware of newer version API and is possibly requesting some of the missing features. The API might be ignoring the unsupported part in assumption that API evolution tool place keeping some compatibility in mind, and still do the best it can with the existing implementation. Or, the API might just fail to work.

The latter item #3 is the scenario here with rebar control. The application is using Windows Vista version of REBARBANDINFO structure and Windows XP implementation choses to completely fail.

While it does not seem to be directly a bug, this attitude is definitely not developer friendly: there is no reason for the control to not work in its best and default way. Having API acting this way, each developer using the API needs to take care of the situation explicitly: whenever Windows Vista enabled application needs to be able to run in Windows XP system, the code around REBARBANDINFO would look like this:

REBARBANDINFO BandInformation = { sizeof BandInformation, RBBIM_LPARAM };
#if _WIN32_WINNT >= 0x0600
if(GetOsVersion() < 0x00060000 || GetCommCtrlVersion() < 0x00060000) // pre-Vista, Common Controls pre-6.0
    BandInformation.cbSize = REBARBANDINFO_V6_SIZE;
#endif// _WIN32_WINNT >= 0x0600
const BOOL bGetBandInfoResult = Rebar.GetBandInfo(0, &BandInformation);

If the API was nicer to developers, the code would be plain and simple:

REBARBANDINFO BandInformation = { sizeof BandInformation, RBBIM_LPARAM };
const BOOL bGetBandInfoResult = Rebar.GetBandInfo(0, &BandInformation);

To address this problem, WTL 8.0 comes up with RunTimeHelper namespace and its SizeOf_REBARBANDINFO function. It takes care of details for the developer choosing the proper size of the structure on runtime. The code is being taken back to a simpler shape:

REBARBANDINFO BandInformation = { RunTimeHelper::SizeOf_REBARBANDINFO(), RBBIM_LPARAM };
const BOOL bGetBandInfoResult = Rebar.GetBandInfo(0, &BandInformation);

All in all:

  • be aware of compatibility issues (same scenario exists with other SDK structures: LVGROUP, LVTILEINFO, MCHITTESTINFO, NONCLIENTMETRICS and other).
  • use latest version of WTL to have things worked around for you where Microsoft developers were not kid enough to provide perfect API
  • be aware and take advantage of WTL’s RunTimeHelper class