The small application is a goos sample and useful tool in the same time. It enumerates PE binary resources and counts languages used. Why? Normally you want single language of resources, however Visual Studio IDE does not show you languages in a convenient way and it is so easy to make a language mess which does not bite you immediately but might bring you troubles later. Apart from this there was a suspicion that language mess might cause runtime bugs on UpdateResource API.

The application gets you a summary of languages used and returns with exit code (checkable using e.g. errorlevel) indicating number of languages.

Output is TSV: LANGID, Language Name, Resource Count:

C:\>ListResourceLanguages.exe C:\Windows\syswow64\shell32.dll
1049    Russian 545
1033    English 3318

Download links:

• Visual C++ .NET 2010 source code: Trac, Subversion

The original intent is pretty clear, as MSDN states:

In release builds, ATLTRACE2 compiles to (void) 0.

As simple as this, but once in a while you are in a situation where release build fails to work for unknown reason and you need additional information for troubleshooting, and then you remember that you had debug tracing code still nicely available in the source, it is just being stripped out by definition of ATLTRACE/ATLTRACE2 macros for release builds.

To avoid reinvention of the wheel and putting new tracing, it might make sense to just re-enable existing tracing (certainly, if putting debug build binary is out of question, which might be the case in production environment and/or to avoid the hassle of installing additional runtime).

The macros need to be #undef’ined and redefined appropriately with or without limiting scope by push_macro/pop_macro #pragma’s. The trick with macro has to reach two goals, to pick file name, line and current symbol name using __FILE__ and friend macros, and also accept variable number of arguments.

The trick ATL does and we can use too is to define a helper class, with constructor taking file name, line and symbol name values, and cast operator () taking actual tracing parameters and arguments. Internally the output can be mapped to OutputDebugString API so that output could be seen using external tool such as DebugView.

When everything is well set, new the macros in question can be defined as follows:

#pragma push_macro("ATLTRACE")
#pragma push_macro("ATLTRACE2")

#undef ATLTRACE
#undef ATLTRACE2

#define ATLTRACE2 CAtlTrace(__FILE__, __LINE__, __FUNCTION__)
#define ATLTRACE ATLTRACE2

int _tmain(int argc, _TCHAR* argv[])
{
    ATLTRACE("First: %d\n", __LINE__);
    ATLTRACE(L"Second: %d\n", __LINE__);
    ATLTRACE2(atlTraceGeneral, 2, "Third: %d\n", __LINE__);
    ATLTRACE2(atlTraceGeneral, 2, L"Fourth: %d\n", __LINE__);
    return 0;
}

#pragma pop_macro("ATLTRACE2")
#pragma pop_macro("ATLTRACE")

And the Release configuration output will be:

Visual C++ .NET 2010 source code is available from SVN; in particular CAtlTrace class is here.

Bonus reading:

• Debugging and Error Reporting Macros
• Debugging with Visual Studio 2005/2008: Logging and Tracing

Having no creativity to build problems myself yet trying to train 9 yo buddy in arithmetics, a small application is here to help generate problem to solve to get familiar with operator priorities and basic math.

Tweaking code back and forth can make it more or less complex. Regular expression replace by pattern erases all answers from end of lines and the text is ready to be printed for the one being trained.

180 - 70 - (6460 + 91 - 6) : 77 + (152 - 74 + 2733 - 67) : (4606 : 47) = ?
(1747 - (153 - 66)) : (43 + 125 - (117 - 32)) - 13 * 1 = 7
21 + 145 - (155 - 79) - 120 : 10 - (99 + 560 : 56 - 96) = 65
2 * (270 + 40) : 62 + (139 - 60 + 8753) : (40 + 29) - (180 - (149 - 36 - (91 - 63))) = 43
(4071 - 12 + 2640 : 80) : (98 - (46 + 35 + 7) + 34) = 93
151 + 39 - 45 - (59 + 62 - 33 - (61 - (70 - 46))) = 94
1 * 11 * 4 - (185 - 73 - (118 + 54 + 14 - 89)) = 29
122 - 83 + 81 - (1 + 8) - (101 - 42 - 5 * 2) = 62
254 - 82 - 65 - ((3342 + 13) : 55 + 118 - 96) + (41 * 2 - 38) * 0 * 95 = 24
(1 * 35 + 204 - 87 - 79 + 4 + 5 - (16 + 2758) : (81 - 43)) * 2133 : 79 : 27 = 9

A binary [Win32] and partial Visual C++ .NET 2010 source code are available from SVN.

If you just need an infinite source of things to crack, it’s here also: 01.txt.

More and more applications hit the Windows limit of available address space for 32-bit applications, and the whole concept becomes more important for understanding due to necessity to work things around.

A thing, which is more or less easy to understand, is that a user mode 32-bit application can address 2^32 addresses. The addresses are not directly physical RAM and the operating system is responsible for management of the mapping addresses into RAM as a part of virtual memory manager operation. Paged memory organization is well documented on MSDN, and the questions has been raised numerous times. An interesting question is whether a 32-bit application can effectively manage memory amounts exceeding address space limits.

Back in 80386 times, the systems could address megabytes of RAM in 16-bit code through XMS and EMS services. The application could access “high” memory addresses by requesting mapping portions of RAM into lower megabyte address space. In some way similar technique is also here for 32-bit applications in Windows through use of file mappings.

A regular memory backed file mapping requests Windows to reserve a memory block which becomes available for mapping into address space of one or more processes. Creating file mapping itself does not imply mapping and this leaves a great option for the owner to allocate more data than it can actually map into address space: if 32-bit process virtual address space is fundamentally constrained, the file mapping allocation space is more loosely limited by amount of physical memory and paging file. The application can allocate 2, 3, 4 and more gigabytes of memory – it just cannot still map it all together into address space and make it available simultaneously.

The FileMappingVirtualAddress utility does a simple thing:

• on startup it allocates (CreateFileMapping) as many 256 MB file mappings as operating system would allow, and shows it in a list
• each time a user checks a box, the application maps (MapViewOfFile) corresponding file mapping into address space; unchecking a box unmaps the view
• the caption shows currently used and maximal available virtual address space

A plain 32-bit version of the application allocated 51 blocks for me (which totals in 13 GB of memory, with 8 GB physical RAM installed in the system). The allocation takes place immediately because the operating system does not actually make all this memory prepared for use – the actual pages would be allocated and ready to use on demand when the application requires them.

The most important part made so obvious is that the 32-bit application succeeds in allocating well over 4 GB, which is maximal virtual address space it can ever get.

The virtual address space in use is only 1641 MB and another request to map an additional section with MapViewOfFile would fail (the default address space limit is 2 GB) – space fragmentation make mapping unavailable earlier than we actually use the whole space, since the API would need to allocate contiguous range of addresses to satisfy the request.

32-bit application built with /LARGEADDRESSAWARE parameter might manage to do more allocations: 64-bit versions of Windows provide 4 GB of addresses to 32-bit processes. 32-bit operating systems might also be extending the limit in case of 4GB RAM Tuning (which would typically be 3 GB of space for a process).

Finally, 64-bit build of the application is free from virtual address space limit as the limit is 8 terabytes. The mapping is again instantaneous because actual RAM will be supplied on first request to mapped pages only.

A binary [Win32, Win32 with /LARGEADDRESSAWARE, x64] and partial Visual C++ .NET 2010 partial source code are available from SVN.

So you got a memory corruption issue with a piece of software. It comes in a unique scenario along the line of having a huge pile of weird code running well most of the time and then, right out of the blue, a corruption takes place followed by unexpected code execution and unstable software state in general.

The biggest problem with memory corruption is that a fragment of code is modifying a memory block which it does not own, and it has no idea who actually is the owner of the block, while the real owner has no timely way to detect the modification. You only face the consequences being unable to capture the modification moment in first place.

To get back to the original cause, an engineer has to drop into a time machine, turn back time and step back to where the trouble took originally place. As developers are not actually given state-of-the-art time machines, the time turning step is speculative.

CVirtualHeapPtr Class: Memory with Exception-on-Write access mode

At the same time a Windows platform developer is or might be aware of virtual memory API which among other things provides user mode application with capabilities to define memory protection modes. Having this on hands opens unique opportunity to apply read-only protection (PAGE_READONLY) onto a memory block and have exception raised at the very moment of unexpected memory modification, having call stack showing up a source of the problem. I refer to this mode of operation as “hardware assisted” because the access violation exception/condition would be generated purely in hardware without any need to additionally do any address comparison in code.

Needless to say that this way is completely convenient for the developer as he does not need to patch the monstrous application all around in order to compare access addresses against read-only fragment. Instead, a block defined as read-only will be immediately available as such for the whole process almost without any performance overhead.

As ATL provides a set of memory allocator templates (CHeapPtr for heap backed memory blocks, allocated with CCRTAllocator, alternate options include CComHeapPtr with CComAllocator wrapping CoTaskMemAlloc/CoTaskMemFree API), let us make an alternate allocator option that mimic well-known class interface and would facilitate corruption detection.

Because virtual memory allocation unit is a page, and protection mode is defined for the whole page, this would be the allocation granularity. For a single allocated byte we would need to request SYSTEM_INFO::dwPageSize bytes of virtual memory. Unlike normal memory heap manager, we have no way to share pages between allocations as we would be unable to effectively apply protection modes. This would definitely increase application pressure onto virtual memory, but is still acceptable for the sacred task of troubleshooting.

We define a CVirtualAllocator class to be compatible with ATL’s CCRTAllocator, however based on VirtualAlloc/VirtualFree API. The smart pointer class over memory pointer would be defined as follows:

template <typename T>
class CVirtualHeapPtr :
    public CHeapPtr<T, CVirtualAllocator>
{
public:
// CVirtualHeapPtr
    CVirtualHeapPtr() throw();
    explicit CVirtualHeapPtr(_In_ T* pData) throw();
    VOID SetProtection(DWORD nProtection)
    {
        // TODO: ...
    }
};

The SetProtection method is to define memory protection for the memory block. Full code for the classes is available on Trac here (lines 9-132):

• CGlobalVirtualAllocator class is a singleton querying operating system for virtual memory page size, and provides alignment method
• CVirtualAllocator class is a CCRTAllocator-compatible allocator class
• CVirtualHeapPtr class is smart template class wrapping a pointer to allocated memory

Use case code will be as follows. “SetProtection(PAGE_READONLY)” enables protection on memory block and turns on exception generation at the moment memory block modification attempt. “SetProtection(PAGE_READWRITE)” would restore normal mode of memory operation.

CVirtualHeapPtr<BYTE> p;
p.Allocate(2);
p[1] = 0x01;
p.SetProtection(PAGE_READONLY);
// NOTE: Compile with /EHa on order to catch the exception
_ATLTRY
{
    p[1] = 0x02;
    // NOTE: We never reach here due to exception
}
_ATLCATCHALL()
{
    // NOTE: Catching the access violation for now to be able to continue execution
}
p.SetProtection(PAGE_READWRITE);
p[1] = 0x03;

Given the information what data gets corrupt, the pointer allocator provides an efficient opportunity to detect the violation attempt. The only thing remained is to keep memory read-only, and temporarily revert to write access when the “legal” memory modification code is about to be executed.

One-shot Read/Write Protection with Guard Pages

Another option granted by memory protection modes is brought by PAGE_GUARD flag. MSDN says:

A guard page provides a one-shot alarm for memory page access. This can be useful for an application that needs to monitor the growth of large dynamic data structures. For example, there are operating systems that use guard pages to implement automatic stack checking.

Setting a guard page mode provides an additional option to trigger an exception with even read access to a protected memory block.

p.SetProtection(PAGE_READWRITE | PAGE_GUARD);
BYTE n = p[0];

CDebugHeapPtr Class: More Options to Catch Memory Corruption Conditions

While setting memory protection attributes on a memory block of interest provides unique troubleshooting opportunities, it still does not cover important typical problems with memory misuse scenarios. Those are writing immediately before the allocated block, and writing immediately after. Having array of N items, this would be writing to indices -1 and N respectively.

To address this scenarios of misuse we can extend CVirtualHeapPtr class so that it could additionally provide “sanity pages” with PAGE_NOACCESS protection at the boundary of allocation. Because virtual memory allocation is granular, we will have to have padding bytes that extend our block to the page boundary, however we have an option to put the padding bytes before or after the payload data block in order to capture after or before memory block writes respectively.

The figure below shows memory layout for the data:

Source code for the CDebugHeapPtr class is available on Trac (lines 155-). The sanity pages create a block of inaccessible addresses which immediately cause access violation exception on either read of write access attempt. Under debugger, those are shows with question marks:

The padding space is pre-initialized with hardcoded value 0×77, and the space is checked for integrity at release of memory block call.

Catching the Exceptions

Having the exceptions generated on run-time, they immediately alter application execution code path and are easy to track and catch. There is no need to bring the feature rich debugger, such as Visual Studio to the production site in order to catch the exception and environment, instead a way simpler tool such as LogProcessExceptions would be able to create a minidump file and write the state of the application. The minidump can be transferred into debugger-enabled environment for detailed check.

Visual C++ .NET 2010 source code is available from SVN.

Today’s update lets you convert multiple locations at once with a single click. Here is the story behind the update and use case scenario.

In rally raid sport events (so called baja), a team gets a road book for the next competition day in a few hours before actual start. The GPS coordinates are printed on one of the pages of the roadbook and are not available in any electronic format.

There were just a few times when the organizer also uploaded a copy of a file with the coordinates and shared a link to download from, but this was rather an exception. Another alternate option was a dedicated person to upload the coordinates (they were earlier full tracks, but at some point tracks were no longer available at all) to pilots’ hardware, but in a state of pre-start рфыеу and variety of GPS hardware, formats, cable etc. this created lines of people. The most one can rely on is a sheet of paper with GPS coordinates. The mistery does not end even here as you don’t know whether you are to get Degrees only, or Degrees and Minutes, or Degrees, Minutes and Seconds. Everything depends on software the organizer uses.

As soon as you get a hard copy of this, the idea is to upload it into device as quickly as possible because there are other things to do and the time is normally 11 PM when the race is to start 7 AM next day tens of miles away from you. The time interval will be shared by uploading data, sleeping and transfer to start location.

The utility is here to grant extra sleep time. Since it is capable to accept various separators on the input, a convenient way is to quickly type in the text in Microsoft Excel, check the data against the hardcopy, and copy into clipboard to transfer to the utility.

A hotkey with conversion transfers data into format of interest, and single “Find and Replace” operation creates a good OziExplorer waypoint file which is good for upload onto portable navigation device.

The whole thing take a few minutes to do with minimal routine typing in.

A binary [Win32] and partial Visual C++ .NET 2010 partial source code are available from SVN.

Bonus picture, rally raid Suzuki is on the way to score the victory and the rally promotional teaser:

This adds a small update to the recently published GPS Location/Coordinate Converter utility:

• Seconds in Degrees, Minutes & Seconds notation are shown and are accepted as floating point numbers
• More shortcuts to popular online map services (note that only Google Maps and Yandex Maps are still accepted as input via clipboard):
• Bing Maps
• Yahoo Maps
• Open Street Map
• WikiMapia

The latter makes the tool an easy to use converted between the services for a GPS POI.

A binary [Win32] and partial Visual C++ .NET 2010 partial source code are available from SVN.

This tool came up as a result of mess around GPS coordinate formatting and variety of popular formats and notations. Some use latitude and longitude degrees, some prefer degrees and minutes with fractional part. this does not take into account choices for datum and file formats.

For instance, both Google Maps and Yandex Maps accept latitude and longitude as ll= URL parameter with the value of latitude and longitude in degrees, however Google uses latitude coming first, while Yandex prefers the opposite.

Recently, a rally road book contained the following formatting of track points:

Having to put a stop to the madness, this utility comes up as a handy assistant to convert and format GPS coordinates into reasonable presentation and separators.

The utility runs on background and monitors clipboard, and once it recognizes one of the following:

• GPS Point in Degrees
• GPS Point in Degrees and Minutes
• GPS Point in Degrees, Minutes and Seconds
• Google Maps URL
• Yandex Maps URL

it flashes and updates its UI providing the choices of track point formatting:

The utility is trying hard to accept various separators and formats, just one thing is important to keep in mind: if you are using minutes without seconds, the fractional part of minutes should be separated by decimal point (.).

The utility also responds to hotkeys Ctrl+Shift+F1, Ctrl+Shift+F2 etc. and copies the formatted point location back into clipboard so that one could quickly re-format editable text in another application through clipboard updates. Alternatively, it is possible to click the formatted value and place it into clipboard.

The utility also provides clickable hyperlinks to open Google Maps and Yandex Maps into the point of interest.

A binary [Win32] and partial Visual C++ .NET 2010 partial source code are available from SVN.

Note: you might need to run the tool “As Administrator” elevating UAC privileges in Vista/7 operating systems.

Bonus links on the topic:

• GPS Coordinate Converter, Maps and Info
• ITN Converter – converts for route file formats

The minimalistic sample demonstrates support of [deprecated] Waveform Audio API for multiple playback streams.

Depending on command line parameters, the application starts threads to open audio hardware using separate waveOutOpen call and stream one or more generated sine waves:

• 1,000 Hz sine wave as 22,050 Hz, Mono, 16-bit PCM (command line parameter “a”)
• 5,000 Hz sine wave as 32,000 Hz, Mono, 16-bit PCM (command line parameter “b”)
• 15,000 Hz sine wave as 44,100 Hz, Mono, 16-bit PCM (command line parameter “c”)

Check(waveOutOpen(&hWaveOut, WAVE_MAPPER, &WaveFormatEx, NULL, NULL, CALLBACK_NULL));
ATLASSERT(hWaveOut);
WAVEHDR* pWaveHeader;
HGLOBAL hWaveHeader = (WAVEHDR*) GlobalAlloc(GMEM_MOVEABLE | GMEM_SHARE, sizeof *pWaveHeader + WaveFormatEx.nAvgBytesPerSec * 10);
pWaveHeader = (WAVEHDR*) GlobalLock(hWaveHeader);
ATLENSURE_THROW(pWaveHeader, E_OUTOFMEMORY);
pWaveHeader->lpData = (LPSTR) (BYTE*) (pWaveHeader + 1);
pWaveHeader->dwBufferLength = WaveFormatEx.nAvgBytesPerSec * 10;
//pWaveHeader->dwUser = 
pWaveHeader->dwFlags = 0;
pWaveHeader->dwLoops = 0;
#pragma region Generate Actual Data
{
    SHORT* pnData = (SHORT*) pWaveHeader->lpData;
    SIZE_T nDataCount = pWaveHeader->dwBufferLength / sizeof *pnData;
    for(SIZE_T nIndex = 0; nIndex < nDataCount; nIndex++)
    pnData[nIndex] = (SHORT) (32000 * sin(1.0 * nIndex / WaveFormatEx.nSamplesPerSec * nFrequency * 2 * M_PI));
}
#pragma endregion 
Check(waveOutPrepareHeader(hWaveOut, pWaveHeader, sizeof *pWaveHeader));
Check(waveOutWrite(hWaveOut, pWaveHeader, sizeof *pWaveHeader));
GlobalUnlock(hWaveHeader);

The operating system is supposed to mix the waves, which can be easily perceived taking place. It is possible to run the application with multiple waveforms within a process, e.g. “abc” command line parameter, and/or start multiple instances of the application.

A binary [Win32] and partial Visual C++ .NET 2010 source code are available from SVN.

The utility and code does straightforward enumeration of MMDevices (Vista+, check MSDN for MMDevice API availability), which correspond to MMDevice API, WASAPI, Core Audio API. The code itself is straightforward, with a ready to use binary to quickly lookup data of interest:

The data is detailed well and in Excel-friendly format (via Copy/Paste):

The code also automatically looks up for named Windows SDK constants, such as PKEY_Device_FriendlyName:

    Identifier    {0.0.1.00000000}.{4c1a7642-3f91-43e5-8fcf-b4b1e803d3f9}
    State    DEVICE_STATE_DISABLED    0x02
    Properties:
        {B3F8FA53-0004-438E-9003-51A46E139BFC}, 15    16 bytes of BLOB, DA 07 03 00 02 00 09 00 0E 00 39 00 16 00 C5 02    65
        PKEY_Device_DeviceDesc    Stereo Mix    31
        {B3F8FA53-0004-438E-9003-51A46E139BFC}, 6    Realtek High Definition Audio    31
        {B3F8FA53-0004-438E-9003-51A46E139BFC}, 2    {1}.HDAUDIO\FUNC_01&VEN_10EC&DEV_0888&SUBSYS_80860034&REV_1002\4&37D44F2F&0&0201    31
        {83DA6326-97A6-4088-9453-A1923F573B29}, 3    oem29.inf:AzaliaManufacturerID.NTamd64.6.0:IntcAzAudModel:6.0.1.5964:hdaudio\func_01&ven_10ec&dev_0888    31
        PKEY_Device_BaseContainerId    {00000000-0000-0000-FFFF-FFFFFFFFFFFF}    72
        PKEY_Device_ContainerId    {00000000-0000-0000-FFFF-FFFFFFFFFFFF}    72
        PKEY_Device_EnumeratorName    HDAUDIO    31
        PKEY_AudioEndpoint_FormFactor    10    19
        PKEY_AudioEndpoint_JackSubType    {DFF21FE1-F70F-11D0-B917-00A0C9223196}    31
        PKEY_DeviceClass_IconPath    %windir%\system32\mmres.dll,-3018    31
        {840B8171-B0AD-410F-8581-CCCC0382CFEF}, 0    316 bytes of BLOB, 01 00 00 00 38 01 00 00 ... 00 00 00 00    65
        PKEY_AudioEndpoint_Association    {00000000-0000-0000-0000-000000000000}    31
        PKEY_AudioEndpoint_Supports_EventDriven_Mode    1    19
        {24DBB0FC-9311-4B3D-9CF0-18FF155639D4}, 3    0    11
        {24DBB0FC-9311-4B3D-9CF0-18FF155639D4}, 4    -1    11
        {9A82A7DB-3EBB-41B4-83BA-18B7311718FC}, 1    65536    19
        {233164C8-1B2C-4C7D-BC68-B671687A2567}, 1    {2}.\\?\hdaudio#func_01&ven_10ec&dev_0888&subsys_80860034&rev_1002#4&37d44f2f&0&0201#{6994ad04-93ef-11d0-a3cc-00a0c9223196}\rtstereomixwave    31
        {5A9125B7-F367-4924-ACE2-0803A4A3A471}, 0    1610612916    19
        {B3F8FA53-0004-438E-9003-51A46E139BFC}, 0    3    19
        PKEY_Device_FriendlyName    Stereo Mix (Realtek High Definition Audio)    31
        PKEY_DeviceInterface_FriendlyName    Realtek High Definition Audio    31
        PKEY_AudioEndpoint_GUID    {4C1A7642-3F91-43E5-8FCF-B4B1E803D3F9}    31

A binary [Win32, x64] and partial Visual C++ .NET 2010 source code are available from SVN.

See also:

• A similar tool with source code
• IMMDeviceEnumerator::EnumAudioEndpoints on MSDN