It was a sort of ridiculous problem: an attempt to instantiate a Microsoft.Jet.OLEDB.4.0 object failed with error. Still some applications are still running without problems connecting to Jet databases, how comes?

There has been a number of posts on Internet, but none of the top ones appeared to be relevant.

The problem is reproduced extremely simple:

int _tmain(int argc, _TCHAR* argv[])
{
    ATLVERIFY(SUCCEEDED(CoInitialize(NULL)));
    {
        //CoLoadLibrary(L"C:\\Program Files (x86)\\Common Files\\System\\Ole DB\\oledb32.dll", TRUE);
        CComPtr<IDBInitialize> pDbInitialize;
        const HRESULT nResult = pDbInitialize.CoCreateInstance(L"Microsoft.Jet.OLEDB.4.0", NULL, CLSCTX_INPROC_SERVER);
        _tprintf(_T("nResult 0x%08x\n"), nResult);
    }
    CoUninitialize();
    return 0;
}

Oops, the code gives error REGDB_E_CLASSNOTREG 0×80040154 “Class not registered”. It looked like system was unable to locate one of the internally used DLLs – oledb32.dll, and if we help by uncommenting the line commented in the code snippet above, the error changes to ERROR_MOD_NOT_FOUND 0x8007007e “The specified module could not be found”.

The problem appears to be that one of the system components, which is involved, “Microsoft OLE DB Data Conversion Library” is registered with the system using a REG_EXPAND_SZ value, to be located using path “%CommonProgramFiles(x86)%\System\Ole DB\oledb32.dll”. It is obvious that CommonProgramFiles(x86) is placeholder to be expanded, but does the expansion succeed?

On my system there was no such environment string, so expansion failed. There however was CommonProgramFiles, and it pointed to correct location. So, this hot fix makes it work right:

BOOL UpdateEnvironmentVariables()
{
    CString sCommonProgramFilesX86;
    sCommonProgramFilesX86.GetEnvironmentVariable(_T("CommonProgramFiles(x86)"));
    if(sCommonProgramFilesX86.IsEmpty())
    {
        CString sCommonProgramFiles;
        sCommonProgramFiles.GetEnvironmentVariable(_T("CommonProgramFiles"));
        if(!sCommonProgramFiles.IsEmpty())
        {
            ATLVERIFY(SetEnvironmentVariable(_T("CommonProgramFiles(x86)"), sCommonProgramFiles));
            return TRUE;
        }
    }
    return FALSE;
}

int _tmain(int argc, _TCHAR* argv[])
{
    ATLVERIFY(SUCCEEDED(CoInitialize(NULL)));
    {
        UpdateEnvironmentVariables();
        CComPtr<IDBInitialize> pDbInitialize;
        const HRESULT nResult = pDbInitialize.CoCreateInstance(L"Microsoft.Jet.OLEDB.4.0", NULL, CLSCTX_INPROC_SERVER);
        _tprintf(_T("nResult 0x%08x\n"), nResult);
    }
    CoUninitialize();
    return 0;
} 

I am under impression that the problem is not specific to Windows 7. I remember similar symptoms on Windows XP system, but they were not critical and not investigated to solution. It rather seems that OLE DB component is registering itself using a reference to environment variable which is not present on a number of systems.

One thing is worth special mentioning in connection with previous post on DirectShow Filter Graph Spy on Microsoft Vista system: ATL’s CSecurityDesc class caused to waste some time.

CRegKey Key;
ATLENSURE_SUCCEEDED(HRESULT_FROM_WIN32(Key.Open(HKEY_CLASSES_ROOT, pszKeyName, READ_CONTROL | WRITE_OWNER)));
CSecurityDesc AdministratorsOwnerSecurityDescriptor;
AdministratorsOwnerSecurityDescriptor.SetOwner(Sids::Admins());
ATLENSURE_SUCCEEDED(HRESULT_FROM_WIN32(Key.SetKeySecurity(OWNER_SECURITY_INFORMATION, &AdministratorsOwnerSecurityDescriptor)));

The code compiles fine, but on runtime it gives error 87 (ERROR_INVALID_PARAMETER, E_INVALIDARG) in the last line, returned from RegSetKeySecurity API call. My first guess was that ATL’s CSecurityDesc class for some reason prepared wrong descriptor which resulted in rejecting it as an argument. From the first glance it looks (not sure) that this class deals, to some extent, with structures itself rather than using API functions, so it could be that it results in something looking differently from expected by API calls.

Still the problem is in class itself and its cast from CSecurityDesc& to required SECURITY_DESCRIPTOR* type. The class only implements operator to automatically cast to const SECURITY_DESCRIPTOR* type, so the following line would not be passed by compiler:

Key.SetKeySecurity(OWNER_SECURITY_INFORMATION, AdministratorsOwnerSecurityDescriptor)

However &AdministratorsOwnerSecurityDescriptor is another level of indirection and hence SECURITY_DESCRIPTOR** type, which is passed by compiler, but results in indeed invalid argument.

So in order to correctly convert CSecurityDesc& to SECURITY_DESCRIPTOR* it can be done this way:

CRegKey Key;
ATLENSURE_SUCCEEDED(HRESULT_FROM_WIN32(Key.Open(HKEY_CLASSES_ROOT, pszKeyName, READ_CONTROL | WRITE_OWNER)));
CSecurityDesc AdministratorsOwnerSecurityDescriptor;
AdministratorsOwnerSecurityDescriptor.SetOwner(Sids::Admins());
ATLENSURE_SUCCEEDED(HRESULT_FROM_WIN32(Key.SetKeySecurity(OWNER_SECURITY_INFORMATION, const_cast<SECURITY_DESCRIPTOR*>((const SECURITY_DESCRIPTOR*) AdministratorsOwnerSecurityDescriptor))));

I have been receiving comments that Filter Graph Spy tool does not work with Microsoft Vista operating system. I never had a moment to check until recently, and this time I realized that it really does not work. I am definitely aware of dramatic changes introduced with this operating system, and in particular UAC feature, virtualization and changes in security. No wonder this was the first guess that security was the cause, however the investigation showed there was a trail of issues underneath…

Investigation details deserve a separate post, while this one briefly outlines the issues and also accompany the repository update with a version compatible with Vista OS.

First of all, COM registration of the DLL (which definitely requires privilege elevation) succeeded on Vista. This means the registration procedure did not encounter any errors on the way, or poorly written code ignored the problem. It appeared that the source of the problem was CoTreatAsClass API, which failed to do the requested action, however returned status code indicating successful operation. This definitely looks like a bug and further comments on this particular behavior are expected to appear on Windows Applications Security MSDN forum, where I opened a topic on the matter.

With a COM TreatAs feature activated, the class’s behavior to instantiate instead of DirectShow’s CLSID_FilterGraph is restored, and in particular the DLL generates FilterGraphSpy.log log file on filter graph activity. Note that log file location is OS dependent (due to Vista’s permissions and file system virtualization):

• pre-Vista OS: root of syste drive, typically C:\
• starting Vista, administrator with elevated privileges: CSIDL_COMMON_APPDATA, typically C:\ProgramData (note this directory is hidden by deafult)
• starting Vista, without elevated administrator privileges: CSIDL_LOCAL_APPDATA, typically C:\Users\$(UserName)\AppData\Local (note that AppData directory is hidden by deafult)

Still even with the log file generated and indicating activation of the spy, it was unable to connect to remote graph through the running object table (ROT). It appeared that the ROT entires are there where expected, it was OK to get an object from ROT and the problem came from QueryInterface code:

CComPtr<IUnknown> pFilterGraphUnknown;
ATLENSURE_SUCCEEDED(pRunningObjectTable->GetObject(pMoniker, &pFilterGraphUnknown));
CComQIPtr<IMyFilterGraph> pFilterGraph = pFilterGraphUnknown; // E_NOINTERFACE

The call reached the original object but COM subsystem was unable to marshal the interface through apartments to enable interprocess communication on it. The reason for this is absence of PSFactoryBuffer class (CLSID {92A3A302-DA7C-4A1F-BA7E-1802BB5D2D02}), which provides proxy/stub pairs for marshaling well known DirectShow interfaces in the Vista’s version of quartz.dll. As mentioned by Microsoft’s Mike Wasson, this class was moved from quartz.dll into Vista SDK’s proppage.dll, so in order to obtain connectivity to remote DirectShow graphs starting Vista, one needs to install this DLL with Windows SDK, or otherwise have it registered with the operating system.

Also note that DirectShow-enabled applications that have their filter graphs published on ROT will need an utility application such as GraphEdit started with the same permissions (elevated or not) in order to be able to access ROT entires.

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

Since ATL version 3, and actually it probably existed even since even earlier time, I have seen Microsoft Visual Studio (and its predecessor Visual C++) generating .RGS registration script with hardcoded CLSID value, ProgID values and other strings entered through IDE wizard. While it seems to work nice at the moment of generation, it still gives the problem to lose sync between identifiers if code is copied between projects or a different way. It might be an oversight originally, but was not it the obvious thing to do later to avoid possible typos of inaccurate user? Microsoft Visua Studio 2008 still generates the same code.

Previous versions of development environment offered attributed code option which in fact took care of this problem very well. Although there has been a lot of negative feedback written about C++ attributes, this was a kind of problem attributes left no chances to break: CLSID was specified once only in C++ COM class attribute and the rest of the deal was generated automatically, including IDL and registration sequence.

It is especially unclear why it was left this way because internal ATL classes already have all necessary capabilities to process wildcards. For example, default .RGS script contains an entry for COM class type library, HKCR\CLSID\{…} key, “TypeLib” value. Why would hardcode it if it can be specified through a replacement:

HKCR
{
	NoRemove CLSID
	{
		ForceRemove %CLSID% = s '...'
		{
			'TypeLib' = s '%LIBID%'

And the replacement is so easily added overriding CAtlModule’s virtual AddCommonRGSReplacements:

class CFooModule :
	public CAtlDllModuleT<CFooModule>
{
...
// CAtlModule
	HRESULT AddCommonRGSReplacements(IRegistrarBase* pRegistrar) throw()
	{
		_ATLTRY
		{
			ATLENSURE_SUCCEEDED(__super::AddCommonRGSReplacements(pRegistrar));
			ATLASSERT(!IsEqualGUID(m_libid, GUID_NULL));
			ATLENSURE_SUCCEEDED(pRegistrar->AddReplacement(L"LIBID", StringFromIdentifier(m_libid)));
		}
		_ATLCATCH(Exception)
		{
			return Exception;
		}
		return S_OK;
	}

I thought it would be implemented natively in ATL long ago, along with per-coclass replacements like %CLSID%.

This is how I like my registration script:

HKCR
{
	%PROGID% = s '%DESCRIPTION%'
	{
		CLSID = s '%CLSID%'
	}
	%VIPROGID% = s '%DESCRIPTION%'
	{
		CLSID = s '%CLSID%'
		CurVer = s '%PROGID%'
	}
	NoRemove CLSID
	{
		ForceRemove %CLSID% = s '%DESCRIPTION%'
		{
			ProgID = s '%PROGID%'
			VersionIndependentProgID = s '%VIPROGID%'
			ForceRemove 'Programmable'
			InprocServer32 = s '%MODULE%'
			{
				val ThreadingModel = s 'Both'
			}
			TypeLib = s '%LIBID%'
		}
	}
}