Fooling Around

If you dare to convert your C++ ATL Service project created with an earlier version of Visual Studio to version 2010, as I recently did, you might find yourself surprised with why the hell the bloody thing does not work anymore as a regular executable.

After passing compiler/linker and SDK update issues, which you possibly might have too, the started executable will stumble on ATL error/exception with CO_E_NOTINITIALIZED (0x800401F0 “CoInitialize has not been called.”). Luckily the error code is good enough for quickly locating the problem and the reason is that the one you trusted, that is ATL, introduced an small improvement which is good for running as service but it not initializing COM anymore if you run your .EXE in application mode.

The bug is on MS Connect since August 3, 2010 closed as if it is going to be fixed some time in future when the fix is propagated to end developers. If you are in a rush and would like to write some code before the event, here are the details.

Previously, COM was initialized right in module constructor, in CAtlExeModuleT. CAtlServiceModuleT class just inherited from there. Later on, someone smart decided that it was not so cool and moved initialization to a later point into CAtlExeModuleT::WinMain. Well, this makes sense as you might (a) end up not needing COM at all, or (b) you want to do some important things before even initializing COM.

Unfortunately, the fact that CAtlServiceModuleT is inherited and relies on base class was not paid too much attention. CAtlServiceModuleT is not getting COM initialization from constructor any longer, CAtlServiceModuleT::WinMain is overridden in full and does not receive initialization from new location either. So well, it does not receive it at all unless run as service, which code execution branch looks still heatlhy here and exhibits another issue later soon.

To resolve the problem, the fragment in CAtlServiceModuleT::Start needs the correction as shown below (within #pragma region):

        if (::StartServiceCtrlDispatcher(st) == 0)
                m_status.dwWin32ExitCode = GetLastError();
            return m_status.dwWin32ExitCode;
        }

        // local server - call Run() directly, rather than
        // from ServiceMain()
        #pragma region Run wrapped by InitializeCom/UninitializeCom
        // FIX: See http://connect.microsoft.com/VisualStudio/feedback/details/582774/catlservicemodulet-winmain-coinitialize-not-called-800401f0
#ifndef _ATL_NO_COM_SUPPORT
        HRESULT hr = E_FAIL;
        hr = T::InitializeCom();
        if (FAILED(hr))
        {
            // Ignore RPC_E_CHANGED_MODE if CLR is loaded. Error is due to CLR initializing
            // COM and InitializeCOM trying to initialize COM with different flags.
            if (hr != RPC_E_CHANGED_MODE || GetModuleHandle(_T("Mscoree.dll")) == NULL)
                return hr;
        } else
            m_bComInitialized = true;
        m_status.dwWin32ExitCode = pT->Run(nShowCmd);
        if (m_bComInitialized)
            T::UninitializeCom();
#else
        m_status.dwWin32ExitCode = pT->Run(nShowCmd);
#endif
        #pragma endregion 

        return m_status.dwWin32ExitCode;
    }

Going further from there, the introduced optimization also removed COM initialization from main process thread module Run function. Provided there earlier too through module constructor it is not longer there. So if you are doing something in application’s run when the application is set to run as service and is executed in application (where you might want to start application as a sort of a helper, or otherwise in specific mode), you need COM initialization there too.

Continue reading →