More and more applications hit the Windows limit of available address space for 32-bit applications, and the whole concept becomes more important for understanding due to necessity to work things around.

A thing, which is more or less easy to understand, is that a user mode 32-bit application can address 2^32 addresses. The addresses are not directly physical RAM and the operating system is responsible for management of the mapping addresses into RAM as a part of virtual memory manager operation. Paged memory organization is well documented on MSDN, and the questions has been raised numerous times. An interesting question is whether a 32-bit application can effectively manage memory amounts exceeding address space limits.

Back in 80386 times, the systems could address megabytes of RAM in 16-bit code through XMS and EMS services. The application could access “high” memory addresses by requesting mapping portions of RAM into lower megabyte address space. In some way similar technique is also here for 32-bit applications in Windows through use of file mappings.

A regular memory backed file mapping requests Windows to reserve a memory block which becomes available for mapping into address space of one or more processes. Creating file mapping itself does not imply mapping and this leaves a great option for the owner to allocate more data than it can actually map into address space: if 32-bit process virtual address space is fundamentally constrained, the file mapping allocation space is more loosely limited by amount of physical memory and paging file. The application can allocate 2, 3, 4 and more gigabytes of memory – it just cannot still map it all together into address space and make it available simultaneously.

The FileMappingVirtualAddress utility does a simple thing:

• on startup it allocates (CreateFileMapping) as many 256 MB file mappings as operating system would allow, and shows it in a list
• each time a user checks a box, the application maps (MapViewOfFile) corresponding file mapping into address space; unchecking a box unmaps the view
• the caption shows currently used and maximal available virtual address space

A plain 32-bit version of the application allocated 51 blocks for me (which totals in 13 GB of memory, with 8 GB physical RAM installed in the system). The allocation takes place immediately because the operating system does not actually make all this memory prepared for use – the actual pages would be allocated and ready to use on demand when the application requires them.

The most important part made so obvious is that the 32-bit application succeeds in allocating well over 4 GB, which is maximal virtual address space it can ever get.

The virtual address space in use is only 1641 MB and another request to map an additional section with MapViewOfFile would fail (the default address space limit is 2 GB) – space fragmentation make mapping unavailable earlier than we actually use the whole space, since the API would need to allocate contiguous range of addresses to satisfy the request.

32-bit application built with /LARGEADDRESSAWARE parameter might manage to do more allocations: 64-bit versions of Windows provide 4 GB of addresses to 32-bit processes. 32-bit operating systems might also be extending the limit in case of 4GB RAM Tuning (which would typically be 3 GB of space for a process).

Finally, 64-bit build of the application is free from virtual address space limit as the limit is 8 terabytes. The mapping is again instantaneous because actual RAM will be supplied on first request to mapped pages only.

A binary [Win32, Win32 with /LARGEADDRESSAWARE, x64] and partial Visual C++ .NET 2010 partial source code are available from SVN.

Today’s update lets you convert multiple locations at once with a single click. Here is the story behind the update and use case scenario.

In rally raid sport events (so called baja), a team gets a road book for the next competition day in a few hours before actual start. The GPS coordinates are printed on one of the pages of the roadbook and are not available in any electronic format.

There were just a few times when the organizer also uploaded a copy of a file with the coordinates and shared a link to download from, but this was rather an exception. Another alternate option was a dedicated person to upload the coordinates (they were earlier full tracks, but at some point tracks were no longer available at all) to pilots’ hardware, but in a state of pre-start рфыеу and variety of GPS hardware, formats, cable etc. this created lines of people. The most one can rely on is a sheet of paper with GPS coordinates. The mistery does not end even here as you don’t know whether you are to get Degrees only, or Degrees and Minutes, or Degrees, Minutes and Seconds. Everything depends on software the organizer uses.

As soon as you get a hard copy of this, the idea is to upload it into device as quickly as possible because there are other things to do and the time is normally 11 PM when the race is to start 7 AM next day tens of miles away from you. The time interval will be shared by uploading data, sleeping and transfer to start location.

The utility is here to grant extra sleep time. Since it is capable to accept various separators on the input, a convenient way is to quickly type in the text in Microsoft Excel, check the data against the hardcopy, and copy into clipboard to transfer to the utility.

A hotkey with conversion transfers data into format of interest, and single “Find and Replace” operation creates a good OziExplorer waypoint file which is good for upload onto portable navigation device.

The whole thing take a few minutes to do with minimal routine typing in.

A binary [Win32] and partial Visual C++ .NET 2010 partial source code are available from SVN.

Bonus picture, rally raid Suzuki is on the way to score the victory and the rally promotional teaser:

An application appears to be not working in Windows XP in a weird way: rebar control appeared to fail showing up.

Where the application is expected to look much nicer with rebar control as a container for menu (implemented as command bar WTL control) and toolbar with buttons:

A WTL sample project generated by a Visual Studio wizard would never give such effect, and the bug was a combination of factors:

1. An application built with a newer version of Windows SDK, which includes support for features (Windows Vista+ Common Controls) that are more recent than production environment (Windows XP); the application targets to Windows Vista+ environment too (_WIN32_WINNT >= 0×0600)
2. Compatibility issues of Common Controls library
3. WTL version (7.5), which did not yet include a workaround for the problem

The problem, which caused the bug directly was the REBARBANDINFO structure and its use as an argument with Common Controls API. As MSDN shows, the structure was amended twice with additional fields.

One of the way to support multiple versions of the structure definition, and to resolve compatibility issues, is to embed structure size into structure payload. In fact, REBARBANDINFO::cbSize member is there exactly for this reason.

The application is normally filling cbSize with the maximal known structure size and fills the rest of the fields respectively. The API is expected to be checking cbSize member and be detecting API version compatibility scenarios:

1. cbSize holds exactly the value the API expects (that is, the maximal value known/defined to the API) – the simplest scenario where the API and the application are on the same page, both are using the same versions of the “protocol”/interface.
2. cbSize is smaller than API can support – the API sees that it is dealing with a sort of legacy application which cannot utilize all available features, and the API acts respectively supporting the older part of the protocol, and keeping defaults or “old look” for the rest of implementation. This addresses backward compatibility: the newer API works with apps designed for older version of the API
3. cbSize is greater then API can support – the API sees that the application is already aware of newer version API and is possibly requesting some of the missing features. The API might be ignoring the unsupported part in assumption that API evolution tool place keeping some compatibility in mind, and still do the best it can with the existing implementation. Or, the API might just fail to work.

The latter item #3 is the scenario here with rebar control. The application is using Windows Vista version of REBARBANDINFO structure and Windows XP implementation choses to completely fail.

While it does not seem to be directly a bug, this attitude is definitely not developer friendly: there is no reason for the control to not work in its best and default way. Having API acting this way, each developer using the API needs to take care of the situation explicitly: whenever Windows Vista enabled application needs to be able to run in Windows XP system, the code around REBARBANDINFO would look like this:

REBARBANDINFO BandInformation = { sizeof BandInformation, RBBIM_LPARAM };
#if _WIN32_WINNT >= 0x0600
if(GetOsVersion() < 0x00060000 || GetCommCtrlVersion() < 0x00060000) // pre-Vista, Common Controls pre-6.0
    BandInformation.cbSize = REBARBANDINFO_V6_SIZE;
#endif// _WIN32_WINNT >= 0x0600
const BOOL bGetBandInfoResult = Rebar.GetBandInfo(0, &BandInformation);

If the API was nicer to developers, the code would be plain and simple:

REBARBANDINFO BandInformation = { sizeof BandInformation, RBBIM_LPARAM };
const BOOL bGetBandInfoResult = Rebar.GetBandInfo(0, &BandInformation);

To address this problem, WTL 8.0 comes up with RunTimeHelper namespace and its SizeOf_REBARBANDINFO function. It takes care of details for the developer choosing the proper size of the structure on runtime. The code is being taken back to a simpler shape:

REBARBANDINFO BandInformation = { RunTimeHelper::SizeOf_REBARBANDINFO(), RBBIM_LPARAM };
const BOOL bGetBandInfoResult = Rebar.GetBandInfo(0, &BandInformation);

All in all:

• be aware of compatibility issues (same scenario exists with other SDK structures: LVGROUP, LVTILEINFO, MCHITTESTINFO, NONCLIENTMETRICS and other).
• use latest version of WTL to have things worked around for you where Microsoft developers were not kid enough to provide perfect API
• be aware and take advantage of WTL’s RunTimeHelper class

This adds a small update to the recently published GPS Location/Coordinate Converter utility:

• Seconds in Degrees, Minutes & Seconds notation are shown and are accepted as floating point numbers
• More shortcuts to popular online map services (note that only Google Maps and Yandex Maps are still accepted as input via clipboard):
• Bing Maps
• Yahoo Maps
• Open Street Map
• WikiMapia

The latter makes the tool an easy to use converted between the services for a GPS POI.

A binary [Win32] and partial Visual C++ .NET 2010 partial source code are available from SVN.

This tool came up as a result of mess around GPS coordinate formatting and variety of popular formats and notations. Some use latitude and longitude degrees, some prefer degrees and minutes with fractional part. this does not take into account choices for datum and file formats.

For instance, both Google Maps and Yandex Maps accept latitude and longitude as ll= URL parameter with the value of latitude and longitude in degrees, however Google uses latitude coming first, while Yandex prefers the opposite.

Recently, a rally road book contained the following formatting of track points:

Having to put a stop to the madness, this utility comes up as a handy assistant to convert and format GPS coordinates into reasonable presentation and separators.

The utility runs on background and monitors clipboard, and once it recognizes one of the following:

• GPS Point in Degrees
• GPS Point in Degrees and Minutes
• GPS Point in Degrees, Minutes and Seconds
• Google Maps URL
• Yandex Maps URL

it flashes and updates its UI providing the choices of track point formatting:

The utility is trying hard to accept various separators and formats, just one thing is important to keep in mind: if you are using minutes without seconds, the fractional part of minutes should be separated by decimal point (.).

The utility also responds to hotkeys Ctrl+Shift+F1, Ctrl+Shift+F2 etc. and copies the formatted point location back into clipboard so that one could quickly re-format editable text in another application through clipboard updates. Alternatively, it is possible to click the formatted value and place it into clipboard.

The utility also provides clickable hyperlinks to open Google Maps and Yandex Maps into the point of interest.

A binary [Win32] and partial Visual C++ .NET 2010 partial source code are available from SVN.

Note: you might need to run the tool “As Administrator” elevating UAC privileges in Vista/7 operating systems.

Bonus links on the topic:

• GPS Coordinate Converter, Maps and Info
• ITN Converter – converts for route file formats

The utility and code does straightforward enumeration of MMDevices (Vista+, check MSDN for MMDevice API availability), which correspond to MMDevice API, WASAPI, Core Audio API. The code itself is straightforward, with a ready to use binary to quickly lookup data of interest:

The data is detailed well and in Excel-friendly format (via Copy/Paste):

The code also automatically looks up for named Windows SDK constants, such as PKEY_Device_FriendlyName:

    Identifier    {0.0.1.00000000}.{4c1a7642-3f91-43e5-8fcf-b4b1e803d3f9}
    State    DEVICE_STATE_DISABLED    0x02
    Properties:
        {B3F8FA53-0004-438E-9003-51A46E139BFC}, 15    16 bytes of BLOB, DA 07 03 00 02 00 09 00 0E 00 39 00 16 00 C5 02    65
        PKEY_Device_DeviceDesc    Stereo Mix    31
        {B3F8FA53-0004-438E-9003-51A46E139BFC}, 6    Realtek High Definition Audio    31
        {B3F8FA53-0004-438E-9003-51A46E139BFC}, 2    {1}.HDAUDIO\FUNC_01&VEN_10EC&DEV_0888&SUBSYS_80860034&REV_1002\4&37D44F2F&0&0201    31
        {83DA6326-97A6-4088-9453-A1923F573B29}, 3    oem29.inf:AzaliaManufacturerID.NTamd64.6.0:IntcAzAudModel:6.0.1.5964:hdaudio\func_01&ven_10ec&dev_0888    31
        PKEY_Device_BaseContainerId    {00000000-0000-0000-FFFF-FFFFFFFFFFFF}    72
        PKEY_Device_ContainerId    {00000000-0000-0000-FFFF-FFFFFFFFFFFF}    72
        PKEY_Device_EnumeratorName    HDAUDIO    31
        PKEY_AudioEndpoint_FormFactor    10    19
        PKEY_AudioEndpoint_JackSubType    {DFF21FE1-F70F-11D0-B917-00A0C9223196}    31
        PKEY_DeviceClass_IconPath    %windir%\system32\mmres.dll,-3018    31
        {840B8171-B0AD-410F-8581-CCCC0382CFEF}, 0    316 bytes of BLOB, 01 00 00 00 38 01 00 00 ... 00 00 00 00    65
        PKEY_AudioEndpoint_Association    {00000000-0000-0000-0000-000000000000}    31
        PKEY_AudioEndpoint_Supports_EventDriven_Mode    1    19
        {24DBB0FC-9311-4B3D-9CF0-18FF155639D4}, 3    0    11
        {24DBB0FC-9311-4B3D-9CF0-18FF155639D4}, 4    -1    11
        {9A82A7DB-3EBB-41B4-83BA-18B7311718FC}, 1    65536    19
        {233164C8-1B2C-4C7D-BC68-B671687A2567}, 1    {2}.\\?\hdaudio#func_01&ven_10ec&dev_0888&subsys_80860034&rev_1002#4&37d44f2f&0&0201#{6994ad04-93ef-11d0-a3cc-00a0c9223196}\rtstereomixwave    31
        {5A9125B7-F367-4924-ACE2-0803A4A3A471}, 0    1610612916    19
        {B3F8FA53-0004-438E-9003-51A46E139BFC}, 0    3    19
        PKEY_Device_FriendlyName    Stereo Mix (Realtek High Definition Audio)    31
        PKEY_DeviceInterface_FriendlyName    Realtek High Definition Audio    31
        PKEY_AudioEndpoint_GUID    {4C1A7642-3F91-43E5-8FCF-B4B1E803D3F9}    31

A binary [Win32, x64] and partial Visual C++ .NET 2010 source code are available from SVN.

See also:

• A similar tool with source code
• IMMDeviceEnumerator::EnumAudioEndpoints on MSDN

The LoadCpu tool creates one or more threads to waste CPU cycles and emulate slower environment, such as for testing and troubleshooting. There has been a great deal of similar tools out there, and this one in particular is convenient/special as it takes process affinity mask as a parameter.

That is, /a:NN defines affinity mask, e.g. 5 for 1st and 3rd CPU (two threads), and /r:MM defines [roughly] CPU load time in percent.

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\>loadcpu /a:5 /r:25
System Affinity Mask: 0xff
Process Affinity Mask: 0x05
Rate: 25%

A binary [Win32] and partial Visual C++ .NET 2010 source code are available from SVN.

Automatic Snapshots runs as a background application monitors clipboard. Should a new image be copied there, the utility automatically writes the image into a file. The process does not require any user interaction and is completely automatic so that – per Mike Westen‘s request – user could quickly and easily create snapshots of a multitude of windows (Alt+Print Screen) or full screen images (Print Screen) using standard hotkeys.

Features:

• Automatic PNG compression (can be disabled via registry and enable BMP mode)
• Reduction of color depth to create smaller PNG files
• Asynchronous processing, the compression and file writes are on background thread to not slow foreground application
• Application window flashes on application bar while busy and indicates the status via window caption
• Application appends current foreground window caption to the image file name

A binary [Win32, x64] and Visual C++ .NET 2010 source code are available from SVN.

It is as stupid as it sounds: you can add RBBS_NOGRIPPER flag to a rebar band to implement “lock controls” feature, but as soon as you start removing it – you miserably fail.

A related thread on WTL group is dated year 2006. You can find the same dated 2003. Now it’s 2011 and it’s still here.

OK, it might be some bug in controlling code etc, but here is Control Spy 2.0. Here you go:

• rebar
• RB_GETBANDINFO, RBBIM_STYLE
• add RBBS_NOGRIPPER, Send
• remove RBBS_NOGRIPPER, Send
• ouch!

WTL does a cool trick to fool the bastard:

void LockBands(bool bLock)
{
    int nBandCount = GetBandCount();
    for(int i =0; i < nBandCount; i++)
    {
        REBARBANDINFO rbbi = { RunTimeHelper::SizeOf_REBARBANDINFO() };
        rbbi.fMask = RBBIM_STYLE;
        BOOL bRet = GetBandInfo(i, &rbbi);
        ATLASSERT(bRet);

        if((rbbi.fStyle & RBBS_GRIPPERALWAYS) == 0)
        {
            rbbi.fStyle |= RBBS_GRIPPERALWAYS;
            bRet = SetBandInfo(i, &rbbi);
            ATLASSERT(bRet);
            rbbi.fStyle &= ~RBBS_GRIPPERALWAYS;
        }
        if(bLock)
            rbbi.fStyle |= RBBS_NOGRIPPER;
        else
            rbbi.fStyle &= ~RBBS_NOGRIPPER;

        bRet = SetBandInfo(i, &rbbi);
        ATLASSERT(bRet);
    }
}

Pretty cool and works… if you have one rebar only. If you have another one in the same window (e.g. mine is at bottom) – you fail again.

A nasty workaround is to nudge hosting window by resizing it, followed by a full cycle of internal layout updates:

CRect Position;
ATLVERIFY(GetWindowRect(Position));
Position.bottom++;
ATLVERIFY(MoveWindow(Position));
Position.bottom--;
ATLVERIFY(MoveWindow(Position));

The question is being asked from time to time. Everyone knows what is full screen video. Video renderers implement full screen capability since long ago through their IVideoWindow::put_FullScreenMode property, and even Filter Graph Manager exposes its own IVideoWindow interface to forward calls to filter’s implementation of IVideoWindow interface.

However, for Video Mixing Renderers, version 7 and 9, the preferred and recommended mode is windowless, where no IVideoWindow interface is available.

Note The IVMRWindowlessControl or IVMRWindowlessControl9 interface is now preferred over IVideoWindow. For more information, see Using Windowless Mode.

So in order to implement full screen mode it takes the application to attach Video Mixing Renderer filter to a private frameless window, to its entire client area and expand the window to entire monitor area.

The sample project FullScreenWindowlessVmrSample01 is illustrating this technique.

Main application dialog owns a special window to host renderer, implemented by CRendererDialog class.

class CRendererDialog :
    public CDialogImpl<CRendererDialog>
{
public:

    enum { IDD = IDD_RENDERER };

BEGIN_MSG_MAP_EX(CRendererDialog)
    MSG_WM_INITDIALOG(OnInitDialog)
    MSG_WM_DESTROY(OnDestroy)
    MSG_WM_ERASEBKGND(OnEraseBkgnd)
    MSG_WM_PAINT(OnPaint)
    MSG_WM_DISPLAYCHANGE(OnDisplayChange)
    MSG_WM_KEYDOWN(OnKeyDown)
    COMMAND_ID_HANDLER_EX(IDCANCEL, OnCancel)
END_MSG_MAP()

private:
    CMainDialog& m_Owner;
//...

The dialog template does not have a caption, it has no frame and has its window rectangle equal to client area. As a result, spanning the Video Mixing Renderer to the entire client area and proper positioning of the dialog window is showing video on the entire monitor.

ATLVERIFY(m_RendererDialog.Create(m_hWnd));
ATLVERIFY(m_RendererDialog.MoveWindow(MonitorData.m_Position));
m_RendererDialog.ShowWindow(SW_SHOWNORMAL);
// ...
ATLENSURE_SUCCEEDED(pVmrWindowlessControl->SetVideoClippingWindow(m_RendererDialog));
ATLENSURE_SUCCEEDED(pVmrWindowlessControl->SetVideoPosition(NULL, m_RendererDialog.GetVideoPosition()));

Visual C++ .NET 2008 source code is available from SVN, release binary included. The project is using ATL and WTL libraries.