Fooling Around // Software Production Line

Microsoft C++ compiler offers a buffer security check option since quite a long ago. It adds so called “security cookies” onto stack and checks them to be not modified by code in order to early-detect stack buffer overflow conditions. The compiler option /GS is enabled by default in Release configurations and is not applicable to debug code (which has optimizations disabled).

Here comes the trouble: release code might show unexpected message box “Buffer overrun detected!”. I have never seen such after years of development and tens of thousands of lines of code written, but luckily there are colleagues who can contribute a challenging issue to investigate.

The message is shown through C runtime’s __crtMessageBoxA function and is not only annoying and angering the customer, it also stops application operation and terminates it:

By default, an application that fails a security check displays a dialog box that states “Buffer overrun detected!” When the dialog box is closed, the application terminates.

However it is possible to override this behavior:

The CRT library offers the developer an option to use a different handler that would react to the buffer overrun in a manner more sensible to the application. The function, __set_security_error_handler, is used to install the user handler by storing the user-defined handler in the user_handler variable

In my case we would be overriding this to trace call stack at the issue point. However, let us see first what other conditions might cause a message box. CRT source has the following occurrences for__crtMessageBoxA:

• assert.c - assertion failure, which normally only applies to debug code
• crt0msg.c - _NMSG_WRITE function, typically for a fatal run time error messages
• crtlib.c - _CRTDLL_INIT fails to start on a Win32s system (VER_PLATFORM_WIN32s)
• crtmbox.c - implementation of __crtMessageBoxA (which is, among other things binds to API functions on runtime instead of static linking)
• dbgrpt.c - CrtMessageWindow function, applies to debug code only
• secfail.c - __security_error_handler function, the /GS thing

So it appears that hopefully /GS failure is the only surprising condition to display an unexpected message box.

OK, let us just see what we got finally here. Using the MSDN sample code and overriding default check failure handler:

int main()
{
	_set_security_error_handler(report_failure); // <<---------------- Override
	// declare buffer that is bigger than expected
	char large_buffer[] = "This string is longer than 10 characters!!";
	vulnerable(large_buffer);
}

Private handler is:

void __cdecl report_failure(int code, void*)
{
	ATLTRACE2(atlTraceGeneral, 0, _T("Security Error Handler: Code %d\n"), code);
	CONTEXT Context = { CONTEXT_CONTROL };
	if(GetCurrentThreadContext(&Context))
		CDebugTraceCallStack::TraceCallStack(Context);
}

and finally output:

BufferSecurityCheckSample.cpp(20): report_failure: Security Error Handler: Code 1
  BufferSecurityCheckSample!0x00402a2d report_failure (+ 125) [c:\buffersecuritychecksample\buffersecuritychecksample.cpp, 22] (+ 10) @00400000
  BufferSecurityCheckSample!0x00403d56 __security_error_handler (+ 48) [f:\vs70builds\6030\vc\crtbld\crt\src\secfail.c, 79] (+ 6) @00400000
  BufferSecurityCheckSample!0x00402e1d report_failure (+ 25) [f:\vs70builds\6030\vc\crtbld\crt\src\secchk.c, 117] (+ 9) @00400000
  BufferSecurityCheckSample!0x0040142c vulnerable (+ 44) [c:\buffersecuritychecksample\buffersecuritychecksample.cpp, 36] (+ 11) @00400000
  BufferSecurityCheckSample!0x00404019 mainCRTStartup (+ 371) [f:\vs70builds\6030\vc\crtbld\crt\src\crt0.c, 259] (+ 18) @00400000
  kernel32!0x7c817067 RegisterWaitForInputIdle (+ 73) @7c800000

The log file would immediately show the source of the run time problem.

See Also:

• /GS (Buffer Security Check)
• Compiler Security Checks In Depth

It is amazing how bugs go through versions and years. Visual Studio’s IDE dockable windows framework is cool, no doubt. But it was since 2005, if not earlier, that tool window undocked by a double click while application is running (debugging session active) is brought to wrong position and is tending to go out of screen area being re-shown.

This is exactly the behavior in 2008 version, regardless of reworked framework and fancy icons/hints/placeholders for the window being dragged. But just now it was even better. I double clicked “Find Results 1″ window to maximize the window… and it disappeared. After looking for it through menu and other GUI elements, I finally found it at the rightmost border of the secondary monitor:

The other day I “wrote some code” to workaround an extremely stupid hardware issues. The depth of idiocy is just incredible: to release a bunch of hardware that just don’t work, release a number of firmware updates that just don’t fix the simplest thing: HTTP compatibility. If there was a single little try to see how this piece of crap comminicates with any WinHTTP based application, an error 12152 ERROR_WINHTTP_INVALID_SERVER_RESPONSE on the first second of execution would imminently come up and demonstrate that someone has to be fired without any hesitation.

Things, however, definitely went a different way with hardware still on the shelves and no firmware upgrades available on the website. Our customer got into trouble having already recommended his customer the buggy thing in amount of X and forwarded us the question of getting everything working. As we decided to make a step towards, I needed to “write some code” to settle the problem.

However, the story was about a different thing. So in order to put a comment into code that explains what kind of problem we are dealing with, I copy/pasted a fragment of HTTP request/response content from Wireshark into source file being edited within MS Visual Studio .NET IDE. Wireshark copies text with some weird line endings, I knew that. I removed extra empty lines from pasted text and actually did not expect anything to go wrong. However who appeared to be wrong was me.

Initially there seemed to be no problem, I tried to compile code, fixed some compiler errors, even started application. I was somewhat confused that the application did not hit my breakpoint while it should. Then I noticed it did not even generate code for this line. As these things do happen with development environment, I re-opened IDE, deleting .NCB for the project and Rebuilt All. The problem however did not go. After further code modification, the compiler started giving errors and shown wrong lines in build output, which did not match source code line numbers.

This started being completely stupid: I was to look at wrong identifier, search through entire source file for occurrences, see if this particular occurrence might be actually a problem for compiler and so on. I made it compiled successffully but under debugger there were still wrong line numbers which made it impossible to debug and set breakpoints.

At this point I remembered Wireshark and line endings. Just removing the comments did not worked. And since visually everything was OK with the source, there should be an easy way found to normalize text. And what I did was the following: I started new message in Mozilla Thunderbird and copy/pasted entire source file content into Thunderbird’s editor. Then copy/pasted back into Visual Studio and finally got the things fixed.

It is amazing how it might come that Microsoft Forums‘ RSS feed appears broken on Google Reader with duplicated entries for each of the items.

Regardless of whose the bug is, it seems to me that it is the Microsoft’s one, both are sophisticated applications of a large scale while RSS thing is just nothing in comparison. Was it really difficult to make the darn thing work?

It seems that everything is buggy nowadays. Some things - more buggy, some are less. I did not expect Microsoft Forums website to be so… so… of so improper quality. Especially compared to variety of popular forum software, free phpbb, google news etc.

MS WYSIWYG post editor can stand no criticism at all, it is one entire bug under FireFox browser. What I recently start getting is a view of the forum which seems to exclude weeks of recent data:

Version 2008 (9.0) of Microsoft Visual Studio .NET 2008 Service Pack 1 development environment brings new annoyances in. It seems to be still freezing and crashing on certain source code files, just as previous versions did and additionally there is a new trick in:

A termination of debugged application quite often ends up with a crash. Once crashed, IDE is still alive (thanks, really) but no debugging session will ever start again. A restart of IDE is required. Posted a crash report once again, hopefully someone is receiving them on MS side and takes to consideration.

Since ATL version 3, and actually it probably existed even since even earlier time, I have seen Microsoft Visual Studio (and its predecessor Visual C++) generating .RGS registration script with hardcoded CLSID value, ProgID values and other strings entered through IDE wizard. While it seems to work nice at the moment of generation, it still gives the problem to lose sync between identifiers if code is copied between projects or a different way. It might be an oversight originally, but was not it the obvious thing to do later to avoid possible typos of inaccurate user? Microsoft Visua Studio 2008 still generates the same code.

Previous versions of development environment offered attributed code option which in fact took care of this problem very well. Although there has been a lot of negative feedback written about C++ attributes, this was a kind of problem attributes left no chances to break: CLSID was specified once only in C++ COM class attribute and the rest of the deal was generated automatically, including IDL and registration sequence.

It is especially unclear why it was left this way because internal ATL classes already have all necessary capabilities to process wildcards. For example, default .RGS script contains an entry for COM class type library, HKCR\CLSID\{…} key, “TypeLib” value. Why would hardcode it if it can be specified through a replacement:

HKCR
{
	NoRemove CLSID
	{
		ForceRemove %CLSID% = s '...'
		{
			'TypeLib' = s '%LIBID%'

And the replacement is so easily added overriding CAtlModule’s virtual AddCommonRGSReplacements:

class CFooModule :
	public CAtlDllModuleT<CFooModule>
{
...
// CAtlModule
	HRESULT AddCommonRGSReplacements(IRegistrarBase* pRegistrar) throw()
	{
		_ATLTRY
		{
			ATLENSURE_SUCCEEDED(__super::AddCommonRGSReplacements(pRegistrar));
			ATLASSERT(!IsEqualGUID(m_libid, GUID_NULL));
			ATLENSURE_SUCCEEDED(pRegistrar->AddReplacement(L"LIBID", StringFromIdentifier(m_libid)));
		}
		_ATLCATCH(Exception)
		{
			return Exception;
		}
		return S_OK;
	}

I thought it would be implemented natively in ATL long ago, along with per-coclass replacements like %CLSID%.

This is how I like my registration script:

HKCR
{
	%PROGID% = s '%DESCRIPTION%'
	{
		CLSID = s '%CLSID%'
	}
	%VIPROGID% = s '%DESCRIPTION%'
	{
		CLSID = s '%CLSID%'
		CurVer = s '%PROGID%'
	}
	NoRemove CLSID
	{
		ForceRemove %CLSID% = s '%DESCRIPTION%'
		{
			ProgID = s '%PROGID%'
			VersionIndependentProgID = s '%VIPROGID%'
			ForceRemove 'Programmable'
			InprocServer32 = s '%MODULE%'
			{
				val ThreadingModel = s 'Both'
			}
			TypeLib = s '%LIBID%'
		}
	}
}