Fooling Around // Software Production Line

Microsoft C++ compiler offers a buffer security check option since quite a long ago. It adds so called “security cookies” onto stack and checks them to be not modified by code in order to early-detect stack buffer overflow conditions. The compiler option /GS is enabled by default in Release configurations and is not applicable to debug code (which has optimizations disabled).

Here comes the trouble: release code might show unexpected message box “Buffer overrun detected!”. I have never seen such after years of development and tens of thousands of lines of code written, but luckily there are colleagues who can contribute a challenging issue to investigate.

The message is shown through C runtime’s __crtMessageBoxA function and is not only annoying and angering the customer, it also stops application operation and terminates it:

By default, an application that fails a security check displays a dialog box that states “Buffer overrun detected!” When the dialog box is closed, the application terminates.

However it is possible to override this behavior:

The CRT library offers the developer an option to use a different handler that would react to the buffer overrun in a manner more sensible to the application. The function, __set_security_error_handler, is used to install the user handler by storing the user-defined handler in the user_handler variable

In my case we would be overriding this to trace call stack at the issue point. However, let us see first what other conditions might cause a message box. CRT source has the following occurrences for__crtMessageBoxA:

• assert.c - assertion failure, which normally only applies to debug code
• crt0msg.c - _NMSG_WRITE function, typically for a fatal run time error messages
• crtlib.c - _CRTDLL_INIT fails to start on a Win32s system (VER_PLATFORM_WIN32s)
• crtmbox.c - implementation of __crtMessageBoxA (which is, among other things binds to API functions on runtime instead of static linking)
• dbgrpt.c - CrtMessageWindow function, applies to debug code only
• secfail.c - __security_error_handler function, the /GS thing

So it appears that hopefully /GS failure is the only surprising condition to display an unexpected message box.

OK, let us just see what we got finally here. Using the MSDN sample code and overriding default check failure handler:

int main()
{
	_set_security_error_handler(report_failure); // <<---------------- Override
	// declare buffer that is bigger than expected
	char large_buffer[] = "This string is longer than 10 characters!!";
	vulnerable(large_buffer);
}

Private handler is:

void __cdecl report_failure(int code, void*)
{
	ATLTRACE2(atlTraceGeneral, 0, _T("Security Error Handler: Code %d\n"), code);
	CONTEXT Context = { CONTEXT_CONTROL };
	if(GetCurrentThreadContext(&Context))
		CDebugTraceCallStack::TraceCallStack(Context);
}

and finally output:

BufferSecurityCheckSample.cpp(20): report_failure: Security Error Handler: Code 1
  BufferSecurityCheckSample!0x00402a2d report_failure (+ 125) [c:\buffersecuritychecksample\buffersecuritychecksample.cpp, 22] (+ 10) @00400000
  BufferSecurityCheckSample!0x00403d56 __security_error_handler (+ 48) [f:\vs70builds\6030\vc\crtbld\crt\src\secfail.c, 79] (+ 6) @00400000
  BufferSecurityCheckSample!0x00402e1d report_failure (+ 25) [f:\vs70builds\6030\vc\crtbld\crt\src\secchk.c, 117] (+ 9) @00400000
  BufferSecurityCheckSample!0x0040142c vulnerable (+ 44) [c:\buffersecuritychecksample\buffersecuritychecksample.cpp, 36] (+ 11) @00400000
  BufferSecurityCheckSample!0x00404019 mainCRTStartup (+ 371) [f:\vs70builds\6030\vc\crtbld\crt\src\crt0.c, 259] (+ 18) @00400000
  kernel32!0x7c817067 RegisterWaitForInputIdle (+ 73) @7c800000

The log file would immediately show the source of the run time problem.

See Also:

• /GS (Buffer Security Check)
• Compiler Security Checks In Depth

A simple question being asked all over again. Given IBasicVideo, IBasicVideo2, IVMRWindowlessControl or IVMRWindowlessControl9, how to save image to file? It is easy. It is a bit easier with IBasicVideo because it is possible to query this interface directly from graph’s interface, such as  IGraphBuilder, and the call will be forwarded to video renderer. This code assumes internal bitmap format is non-paletted, which I believe is always the case.

LONG nBufferSize = 0;
ATLENSURE_SUCCEEDED(pBasicVideo->GetCurrentImage(&nBufferSize, NULL));
CHeapPtr<BITMAPINFO> pBitmapInfo;
ATLENSURE_THROW(pBitmapInfo.AllocateBytes(nBufferSize), E_OUTOFMEMORY);
ATLENSURE_SUCCEEDED(pBasicVideo->GetCurrentImage(&nBufferSize, (LONG*) (BITMAPINFO*) pBitmapInfo));
const BYTE* pnData = (const BYTE*) (&pBitmapInfo->bmiHeader + 1);
// NOTE: You might wish to handle <=8 bpp bitmaps here
ATLASSERT(pBitmapInfo->bmiHeader.biBitCount > 8);
ATLASSERT(pBitmapInfo->bmiHeader.biCompression == BI_RGB);
ATLASSERT(pBitmapInfo->bmiHeader.biSizeImage);
BITMAPFILEHEADER BitmapFileHeader;
ZeroMemory(&BitmapFileHeader, sizeof BitmapFileHeader);
BitmapFileHeader.bfType = 'MB';
BitmapFileHeader.bfSize = (DWORD) (sizeof (BITMAPFILEHEADER) + pBitmapInfo->bmiHeader.biSize + pBitmapInfo->bmiHeader.biSizeImage);
BitmapFileHeader.bfOffBits = (DWORD) (sizeof (BITMAPFILEHEADER) + pBitmapInfo->bmiHeader.biSize);
ATLENSURE_SUCCEEDED(File.Write(&BitmapFileHeader, sizeof BitmapFileHeader));
ATLENSURE_SUCCEEDED(File.Write(&pBitmapInfo->bmiHeader, pBitmapInfo->bmiHeader.biSize));
ATLENSURE_SUCCEEDED(File.Write(pnData, (DWORD) pBitmapInfo->bmiHeader.biSizeImage));

This seems to be undocumented (Google does not know anything on this): ATL server executables may be run with /RegServerPerUser and /UnregServerPerUser switches to hopefully (this what would expect this to do without checking) register contained COM classes under HKEY_CURRENT_USER rather than HKEY_LOCAL_MACHINE and thus allow COM server run under credentials of non-administrative user. See Also: Merged View of HKEY_CLASSES_ROOT.

Good bye _ATL_MIN_CRT: this mode is retired with Visual Studio .NET 2008.

PROP_ENTRY and PROP_ENTRY_EX are deprecated and replaced with PROP_ENTRY_TYPE and PROP_ENTRY_TYPE_EX. Note that property pages have to be mentioned in property map explicitly using PROP_PAGE as opposed to old behavior when they were referenced as a part of property declaration.

ATL Server is taken out of the product and moved to open source project at CodePlex.

Many small fixes, including for example for CenterWindow.

I am compiling C++ project which is using IJG JPEG library using Visual Studio .NET 2005 (yes, 2008 is yet to come) and it came as a surprise that I stumbled upon struct data alignment issue. I am pretty sure this was not a problem with Visual Studio .NET 2003.

The project includes my C++ files and C source from the library. C++ code includes:

extern "C"
{

};

Running this, I receive an error from the library (thanks for pointing this out early anyway!):

JPEG parameter struct mismatch: library thinks size is 360, caller expects 347.

How comes? Project settings are default and set to use default alignment /Zp. It seems as if compiler uses default value of 1 for my C++ code and uses default value of 8 for C code… with a result of break on runtime.

The workaround is in referencing C code with alignment override using #pragma pack:

extern "C"
{

};


Another weird bug from Visual C++ .NET/ATL internals (Visual Studio .NET 2003 Service Pack 1). IDispatch implementation (IModuleCommunicationTcp on the screenshot) supplied by Visual C++  compiler that converts “by DISPID” IDispatch::Invoke call into virtual method call (IModuleCommunicationTcp::get_HttpHost) does not initialize local BSTR variable i1 on stack. In case of method failure (in which case there is no guarantee i1 was properly initialized/cleared) the code still uses the variable to initialize VARIANT.

In my case this caused resulting VARIANT (CComVariant) contain .vt = VT_BSTR, .bstrVal = 0×00000002, which is obviously incorrect and caused an access violation exception later in VariantClear call.

A friend of mine recently told that BrainBench has a promotion with a free C++ test (to be requested for free before March, 31) and I decided to spend an hour to take it. Here is the test description:

Our C++ test measures your knowledge of programming in C++. Designed for experienced programmers, this test covers the following topics: Construction and Destruction, Encapsulation and Classes, Exceptions, I/O, Inheritance, Non-core Features, Overloading, Standard Template Library, Templates, and Types.

I have been knowing about BrainBench since long ago, if my memory serves me right (since soon after their start, I suppose) and when I took Delphi test. I previously heard that some employers require that candidates pass this kind of test for the employer to have an evaluation of skills, for example from my email archive:

We will ask you to solve a small but fairly difficult low-level API problem in Delphi. If that task is completed in a timely manner, we will then arrange for a personal interview. Part of the personal interview may include passing a test paid for by iolo technologies, LLC from Brain Bench (www.brainbench.com). This test can be taken before or after the interview.

Not my case and seriously I think it makes more sense to evaluate a junior or lower intermediate this way. The test contained a lot of STL and C++ language specific questions which I, having been a C++ engineer for years, have never dealt with in real life. No wonder about STL because I have been ATL/WTL guy but this was true even non-STL questions. And I am pretty sure that many of those would be as surprising and unexpected for the folks I am nip and tuck on a daily basis…

What came to me as surprise was “Percentile”, higher than 99% of those who took a free test:

Score: 4.59
Percentile: Scored higher than 99% of previous examinees

Demonstrates understanding of most advanced concepts within the subject area. Appears capable of mentoring others on the most complex projects.

Strong Areas

• Exceptions
• Construction and Destruction
• Standard Template Library
• Types

Weak Areas

• None noted

There has been an unexpected problem localizing application resources when STRING resources failed to load while there were no problems with other resource types. The source of the problem appears to be within ATL core. As long as CP_THREAD_ACP is involved in ATL’s conversion the loaded string is screwed.
The problem is illustrated by the sample. The same string resource is loaded with LoadString API and CString::LoadString ATL function. The difference is emphasized with bold below. As soon as Russian string is loaded from a thread with non-Russian thread locale, the string is screwed but only when loaded through ATL class (warning: Russian characters seems to be forged by WordPress in quoted text below):

C:\>ThreadLocaleSample01.exe
Default Thread Locale: 1033 (0×409), Primary Language 9 (0×9), SubLanguage 1 (0×1), Sort 0 (0×0)
String 101: ABCDEFGHIJKLMNOPQRSTUVWXYZ
String 102: ????????????????????????????????
ATL String 101: ABCDEFGHIJKLMNOPQRSTUVWXYZ
ATL String 102: ????????????????????????????????
Russian Locale: 1049 (0×419), Primary Language 25 (0×19), SubLanguage 1 (0×1), Sort 0 (0×0)
String 101: ABCDEFGHIJKLMNOPQRSTUVWXYZ
String 102: ????????????????????????????????
ATL String 101: ABCDEFGHIJKLMNOPQRSTUVWXYZ
ATL String 102: ????????????????????????????????
English Locale: 1033 (0×409), Primary Language 9 (0×9), SubLanguage 1 (0×1), Sort 0 (0×0)
String 101: ABCDEFGHIJKLMNOPQRSTUVWXYZ
String 102: ????????????????????????????????
ATL String 101: ABCDEFGHIJKLMNOPQRSTUVWXYZ
ATL String 102: ????????????????????????????????

Read more »