Range-based for with ATL Collections

ATL collection classes did not receive an update related to new C++11 range-based fors and other fancy features. Well it’s a pity because writing

for(auto&& pPin: AudioPinList)
  pPin-> //...

compared to

for(POSITION Position = AudioPinList.GetHeadPosition(); Position; AudioPinList.GetNext(Position))
{
  IPin* pPin = AudioPinList.GetAt(Position);
  pPin-> //...

is indeed a relief. Namespace std is, of course, good but ATL is still here as well. Here is a take on adding range-based for loops to ATL collection classes.

  • CRoIterativeCollectionT class is a traits-based uniform base for lists and arrays, adding (again, uniform) helpers to iterate through collection:
    • GetCountThat, GetThat, RemoveThat, RemoveFirst, RemoveLast
    • Find, FindThat, FindFirst, FindLast, FindFirstThat, FindLastThat
    • ForEach
    • begin, end implementing for range based for loops
  • CRoArrayT is a class extending CAtlArray, adding mentioned above through inheriting from CRoIterativeCollectionT
  • CRoListT is a class extending CAtlList, adding mentioned above through inheriting from CRoIterativeCollectionT
  • CRoAssignableListT, CRoAssignableArrayT, CRoAssignableMapT to inherit collections and allow assignment operators on those (through duplication of elements), esp. to be able to easily put the collections as members of classes, eligible for assignment operator copies
  • CRoFixedArrayT, CRoFixedMapT are compatible collection classes backed by stack memory only, no allocations (old stuff as is, was used somewhere in handler where immediate response assumed no memory allocation operations)
  • CRoMapT is essentially an thin extension of CAtlMap, however also adds GetPositions() and GetValues() methods, which can be used for range-based for loops

Download

Three ways to implement VBScript (VB6, VBA) callback from C++/ATL class

Suppose you have an automation object that needs to implement a callback into caller Visual Basic environment, which can be Scripting Host, Visual Basic for Applications, ASP etc. With all the late binding in VB scripting and so much different C++ code – how to put everything together? There are great choices, let us have three.

On all three samples we add numbers on the way:

  • 300 is the initial argument of the VB caller
  • 20 more in the VB callback
  • 1 adds the C++ COM class

The three .VBS files will output 321 is everything goes well. The C++ method name is OuterDo, while the supposed callback method is InnerDo.

There are two basic problems on the way:

  1. We have to somehow pass the callback to COM class
  2. We have to define function in a way that all they are compatible one with another

Callback in VB Class Function

Straightforwardly, we can pass VB class to have a method called back. While it might be a good idea with VBA where it might be possible to add a reference to type library and have the named callback interface available in scripting environment (VB’s Implements keyword), this does not work directly in VB Scripting Host. To work this around in .VBS sample we don’t reference IFirstSite interface, and the automation object First will access the method by its hardcoded name.

The source code has commented parts which can be used to connect the parts more strictly using IFirstSite interface.

Class FirstSite 
  'Implements IFirstSite
  Public Function IFirstSite_InnerDo(ByVal A)
    IFirstSite_InnerDo = 20 + A
  End Function
End Class

Dim First
Set First = WScript.CreateObject("AlaxInfo.VbsCallback.First")
Result = First.OuterDo(300, new FirstSite)
WScript.Echo Result

C++ implementation accepts the call using IDL syntax:

interface IFirst : IDispatch
{
    //[id(1)] HRESULT OuterDo([in] LONG nA, [in] IFirstSite* pSite, [out, retval] LONG* pnB);
    [id(1)] HRESULT OuterDo([in] LONG nA, [in] IDispatch* pSite, [out, retval] LONG* pnB);
};

In C++ we receive a COM interface of the class, and now we are to locate the callback method by its name using IDispatchEx interface. Once we succeed with this, we invoke a dispatch interface call.

Callback in VB Function

An alternate option is to pass separate function IDispatch interface and have it called from C++. compared to using strictly defined interface IFirstSite this might look like an inferior way, however considering the workarounds we have to put in method above to stay compatible with Scripting Host, this method might look even a bit simpler.

Function InnerDo(ByVal A)
    InnerDo = 20 + A
End Function

Dim Second
Set Second = WScript.CreateObject("AlaxInfo.VbsCallback.Second")
Result = Second.OuterDo(300, GetRef("InnerDo"))
WScript.Echo Result

On the caller side, the key to success is GetRef method that creates a IDispatch-enabled object from a separate function. C++ will use IDispatch::Invoke on DISPID of zero in order for the call to reach the implementation.

Callback through Connection Points

Connection points are standard and well known mechanism to deliver outgoing calls from an automation object, however they are subject to certain constraints:

  • late binding is taking place and we have to use IDispatch::Invoke to deliver calls, luckily Visual Studio is capable of generating proxy classes for that (no IFirstSite-like strictly defined and callable interfaces!)
  • connection points assume that there might be several parties connected to the points/events, and the interface should nicely supports this (no return values!)

Most of the environments have support for connection points on caller side, so this methods is nicely applicable.

Sub Third_InnerDo(ByRef C)
  C = C + 20
End Sub

Dim Third
Set Third = WScript.CreateObject("AlaxInfo.VbsCallback.Third", "Third_")
Result = Third.OuterDo(300)
WScript.Echo Result

In C++ there is a proxy class to deliver the event, so implementation is as simple as this:

// IThird
    STDMETHOD(OuterDo)(LONG nA, LONG* pnB) throw()
    {
        ATLASSERT(pnB);
        CComVariant vB(nA + 1);
        ATLVERIFY(SUCCEEDED(Fire_InnerDo(&vB)));
        ATLASSERT(vB.vt == VT_I4);
        *pnB = vB.lVal;
        return S_OK;
    }

Note that [out] parameters need to be VARIANTs or otherwise the returned values might get lost on the way.

So we are ready for a test run:

D:\Projects\Alax.Info\Repository-Public\Utilities\VbsCallback\Scripts>cscript First.vbs
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

321

D:\Projects\Alax.Info\Repository-Public\Utilities\VbsCallback\Scripts>cscript Second.vbs
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

321

D:\Projects\Alax.Info\Repository-Public\Utilities\VbsCallback\Scripts>cscript Third.vbs
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

321

Good news, all three methods work well!

Visual C++ .NET 2010 source code [Trac, Subversion] is available from SVN. .VBS scripts are included.

LogProcessExceptions: Log Service Process Exceptions

One of the nasty issues with LogProcessExceptions utility was that it was unable to attach to service processes and track them to catch their exceptions.

The actual problem was that the processes were not listed in first place, so there was nothing to attach to. Access and security requirements necessary for a process to debug another process are listed in MSDN DebugActiveProcess article:

The debugger must have appropriate access to the target process, and it must be able to open the process for PROCESS_ALL_ACCESS. DebugActiveProcess can fail if the target process is created with a security descriptor that grants the debugger anything less than full access. If the debugging process has the SE_DEBUG_NAME privilege granted and enabled, it can debug any process.

The utility did enable the SE_DEBUG_NAME privilege, however it was doing it prior to starting debugging session and after the process of interest was already pointed to by user.

This was insufficient because EnumProcesses only lists service processes (not actually exactly services, but processes running in different security context) in case debug privilege is already enable by the time of the API call. The utility now enabled the privilege well in advance and list the services, so can be effectively applied to those.

Download links:

Enabling ATLTRACE output in Release configuration builds

The original intent is pretty clear, as MSDN states:

In release builds, ATLTRACE2 compiles to (void) 0.

As simple as this, but once in a while you are in a situation where release build fails to work for unknown reason and you need additional information for troubleshooting, and then you remember that you had debug tracing code still nicely available in the source, it is just being stripped out by definition of ATLTRACE/ATLTRACE2 macros for release builds.

To avoid reinvention of the wheel and putting new tracing, it might make sense to just re-enable existing tracing (certainly, if putting debug build binary is out of question, which might be the case in production environment and/or to avoid the hassle of installing additional runtime).

The macros need to be #undef’ined and redefined appropriately with or without limiting scope by push_macro/pop_macro #pragma’s. The trick with macro has to reach two goals, to pick file name, line and current symbol name using __FILE__ and friend macros, and also accept variable number of arguments.

The trick ATL does and we can use too is to define a helper class, with constructor taking file name, line and symbol name values, and cast operator () taking actual tracing parameters and arguments. Internally the output can be mapped to OutputDebugString API so that output could be seen using external tool such as DebugView.

When everything is well set, new the macros in question can be defined as follows:

#pragma push_macro("ATLTRACE")
#pragma push_macro("ATLTRACE2")

#undef ATLTRACE
#undef ATLTRACE2

#define ATLTRACE2 CAtlTrace(__FILE__, __LINE__, __FUNCTION__)
#define ATLTRACE ATLTRACE2

int _tmain(int argc, _TCHAR* argv[])
{
    ATLTRACE("First: %d\n", __LINE__);
    ATLTRACE(L"Second: %d\n", __LINE__);
    ATLTRACE2(atlTraceGeneral, 2, "Third: %d\n", __LINE__);
    ATLTRACE2(atlTraceGeneral, 2, L"Fourth: %d\n", __LINE__);
    return 0;
}

#pragma pop_macro("ATLTRACE2")
#pragma pop_macro("ATLTRACE")

And the Release configuration output will be:

Visual C++ .NET 2010 source code is available from SVN; in particular CAtlTrace class is here.

Bonus reading:

Hardware assisted memory corruption detection

So you got a memory corruption issue with a piece of software. It comes in a unique scenario along the line of having a huge pile of weird code running well most of the time and then, right out of the blue, a corruption takes place followed by unexpected code execution and unstable software state in general.

The biggest problem with memory corruption is that a fragment of code is modifying a memory block which it does not own, and it has no idea who actually is the owner of the block, while the real owner has no timely way to detect the modification. You only face the consequences being unable to capture the modification moment in first place.

To get back to the original cause, an engineer has to drop into a time machine, turn back time and step back to where the trouble took originally place. As developers are not actually given state-of-the-art time machines, the time turning step is speculative.

CVirtualHeapPtr Class: Memory with Exception-on-Write access mode

At the same time a Windows platform developer is or might be aware of virtual memory API which among other things provides user mode application with capabilities to define memory protection modes. Having this on hands opens unique opportunity to apply read-only protection (PAGE_READONLY) onto a memory block and have exception raised at the very moment of unexpected memory modification, having call stack showing up a source of the problem. I refer to this mode of operation as “hardware assisted” because the access violation exception/condition would be generated purely in hardware without any need to additionally do any address comparison in code.

Needless to say that this way is completely convenient for the developer as he does not need to patch the monstrous application all around in order to compare access addresses against read-only fragment. Instead, a block defined as read-only will be immediately available as such for the whole process almost without any performance overhead.

As ATL provides a set of memory allocator templates (CHeapPtr for heap backed memory blocks, allocated with CCRTAllocator, alternate options include CComHeapPtr with CComAllocator wrapping CoTaskMemAlloc/CoTaskMemFree API), let us make an alternate allocator option that mimic well-known class interface and would facilitate corruption detection.

Because virtual memory allocation unit is a page, and protection mode is defined for the whole page, this would be the allocation granularity. For a single allocated byte we would need to request SYSTEM_INFO::dwPageSize bytes of virtual memory. Unlike normal memory heap manager, we have no way to share pages between allocations as we would be unable to effectively apply protection modes. This would definitely increase application pressure onto virtual memory, but is still acceptable for the sacred task of troubleshooting.

We define a CVirtualAllocator class to be compatible with ATL’s CCRTAllocator, however based on VirtualAlloc/VirtualFree API. The smart pointer class over memory pointer would be defined as follows:

template <typename T>
class CVirtualHeapPtr :
    public CHeapPtr<T, CVirtualAllocator>
{
public:
// CVirtualHeapPtr
    CVirtualHeapPtr() throw();
    explicit CVirtualHeapPtr(_In_ T* pData) throw();
    VOID SetProtection(DWORD nProtection)
    {
        // TODO: ...
    }
};

The SetProtection method is to define memory protection for the memory block. Full code for the classes is available on Trac here (lines 9-132):

  • CGlobalVirtualAllocator class is a singleton querying operating system for virtual memory page size, and provides alignment method
  • CVirtualAllocator class is a CCRTAllocator-compatible allocator class
  • CVirtualHeapPtr class is smart template class wrapping a pointer to allocated memory

Use case code will be as follows. “SetProtection(PAGE_READONLY)” enables protection on memory block and turns on exception generation at the moment memory block modification attempt. “SetProtection(PAGE_READWRITE)” would restore normal mode of memory operation.

CVirtualHeapPtr<BYTE> p;
p.Allocate(2);
p[1] = 0x01;
p.SetProtection(PAGE_READONLY);
// NOTE: Compile with /EHa on order to catch the exception
_ATLTRY
{
    p[1] = 0x02;
    // NOTE: We never reach here due to exception
}
_ATLCATCHALL()
{
    // NOTE: Catching the access violation for now to be able to continue execution
}
p.SetProtection(PAGE_READWRITE);
p[1] = 0x03;

Given the information what data gets corrupt, the pointer allocator provides an efficient opportunity to detect the violation attempt. The only thing remained is to keep memory read-only, and temporarily revert to write access when the “legal” memory modification code is about to be executed.

Continue reading →

GPS Location/Coordinate Converter: Fractional Seconds, More Shortcuts

This adds a small update to the recently published GPS Location/Coordinate Converter utility:

  • Seconds in Degrees, Minutes & Seconds notation are shown and are accepted as floating point numbers
  • More shortcuts to popular online map services (note that only Google Maps and Yandex Maps are still accepted as input via clipboard):
    • Bing Maps
    • Yahoo Maps
    • Open Street Map
    • WikiMapia

The latter makes the tool an easy to use converted between the services for a GPS POI.

A binary [Win32] and partial Visual C++ .NET 2010 partial source code are available from SVN.

ATLENSURE_SUCCEEDED double failure

A colleague pointed out that code snippet in previous post is misusing ATL’s ATLENSURE_SUCCEEDED macro making it [possibly] evaluate its argument twice in case of failure, that is evaluating into failure HRESULT code. As it is defined like this:

#define ATLENSURE_SUCCEEDED(hr) ATLENSURE_THROW(SUCCEEDED(hr), hr)

It does things in a straightforward way, for a code line

ATLENSURE_SUCCEEDED(pFilterGraph.CoCreateInstance(CLSID_FilterGraph));

It is doing “let’s CoCreateInstance the thing and if it fails, let’s CoCreateInstance it again to find out error code”. Disassembly shows this clearly:

This is exactly another spin of the story previously happened with HRESULT_FROM_WIN32 macro and possibly a number of others. With it being originally a macro, SDK offered an option to override the definition by pre-defining INLINE_HRESULT_FROM_WIN32. This way a user might be explicitly requesting a safer definition while still leaving legacy code live with macro. See more detailed story on this in Matthew’s blog.

A tricky thing is that with successful execution the problem does not come up. In case of failure, it depends on the functions called, some with just repeat the error code, some will return a different code on second run, some might create less desired and expected consequences. So you can find yourself having written quite some code before you even suspect a problem.

Having identified the issue, there are a few solutions.

1. First of all, the original ATLENSURE_SUCCEEDED macro can still be used, provided that you don’t put expressions as arguments.

This is going to do just fine:

const HRESULT nCoCreateInstanceResult = pFilterGraph.CoCreateInstance(CLSID_FilterGraph);
ATLENSURE_SUCCEEDED(nCoCreateInstanceResult);

2. Second straightforward way is to replace the original ATL definition in ATL code (boo, woodenly)

3. As ATL code is checking for the macros to be already defined, and skipping its own definition in such case, it is possible to inject a safer private definition before including ATL headers (which would typically need one to do the define in stdafx.h):

#define ATLENSURE_SUCCEEDED(x){ const HRESULT nResult =(x); ATLENSURE_THROW(SUCCEEDED(nResult), nResult); }

#include <atlbase.h>
#include <atlstr.h>

Pre-evaluating the argument into local variable is going to resolve the original multi-evaluation problem.

4. There might be a new inline function defined on top of the original macro, which will be used instead and which is free from the problem:

inline VOID ATLENSURE_INLINE_SUCCEEDED(HRESULT nResult)
{
    ATLENSURE_SUCCEEDED(nResult);
}

Either way, the correct code compiles into single argument evaluation and throws an exception with failure code immediately:

Also, vote for the suggestion on Microsoft Connect. The issue is marked as fixed in future version of Visual Studio.