Fooling Around

Yesterday I received a complaint from a Most Important Customer about a weird problem on a few of their digital video recording servers. The problem was that a new version of software, client part, would out of a sudden auto-close without any notice. The problem was quite persistent and basically of not a difficult kind because such fatal problems are automatically logged with a descriptive enough log to identify the cause in an extremely effective way. However the log files appeared to be empty. Another unexpected part was that a few systems showed the issue while the other did not and software worked in a quite expected and stable way there.

After a couple of phone calls and setting up remote access using TightVNC and port forwarding I could see this myself and it was obvious that there is an exception, such as memory access vioaltion, which takes place however on a thread created outside of my code and runnning without a proper guarding section to at the very least catch the exception and log its call stack. The visual symptoms led to supposition that it is something DirectShow related and chances are high that it is either a hardware acceleration issue, or a software compatibility issue. Luckily, hardware acceleration assumption is easy to eliminate by switching off, letting it run without and seeing if there is any difference. There was not.

Then it appeared that while these system were close to clean installation state, there was another DVR software installed also, to be used for tasks not covered by our software. And my first guess was if it installed certain DriectShow filter of the crappy kind, which could interfere with our award winning software, the best of the best. A quick test of renaming “C:\Program Files\Another-DVR” into a different name making all contained binaries unopenable by path showed that nothing changed essentially. So it was GraphEdit, which was downloaded there and a list of installed filters was insepected visually.

There were a few filters that was present in the system but which were obviously installed additionally to the system, as a part of Another-DVR product or in a different way. There was a few filters by I. and then my look stumbled on something which I immediately thought of “That’s It!”. Well, taking into consideration time spent on the problem it was rather “That’s Fucking It!”.

A rename of the hosting file pvmjpg21.dll in %WINDIR%\system32 immediately fixed the problem! Oo-omm-m “PICVideo MJPEG Decompressor” by Pegasus Imaging Corporation, crap! It was one of those moments sung of by Fort Minor:

This is twenty percent crap, eighty percent lame
And a hundred percent reason to remember the name!

However, let us what was going on there. A quick look at the registration information makes it clear the video decoding bastard registered itself under MERIT_PREFERRED (0×00800000) merit and generic MEDIATYPE_Video/MEDIASUBTYPE_NULL media type. Why on earth a properly made codec would claim to be able to decode any video as prefered software? It obviously adds overhead to the entire DirectShow subworld on that system by poking its pin into anything that moves without a proper luck to actually connect.

However, it would be just an overhead, maybe even unnoticable, if the bastard was a thread safe library. But it was unfortunately not. Since my software starts showing multiple video feeds simultaneously, this piece of junk was simultaneously instantiated in multiple threads and just crashed the entire process.

P.S. Bonus: “pvmjpg21.dll crash” on Google

IDirectSoundNotify8 is an interface to get notified on playback on capture audio buffer reaching certain position in the buffer. It is a must thing when implementing ring buffers with new data continuously added to the buffer for seamless playback (continuously copied from in case of capture).

This project is a minimalistic C++ sample code to illustrate the API. To initialize the DirectSound subsystem it is required to provide a window handle, which is created using ATL’s CWindowImpl (CMessageOnlyWindowImpl).

...
ATLENSURE_SUCCEEDED(DirectSoundCreate8(NULL, &pDirectSound8, NULL));
...
ATLENSURE_SUCCEEDED(pDirectSound8->SetCooperativeLevel(Window, DSSCL_PRIORITY));
...
ATLENSURE_SUCCEEDED(pDirectSound8->CreateSoundBuffer(&BufferDescriptor, &pDirectSoundBuffer, NULL));
...
CComQIPtr<IDirectSoundNotify8, &IID_IDirectSoundNotify8> pDirectSoundNotify8 = pDirectSoundBuffer;
...
ATLENSURE_SUCCEEDED(pDirectSoundNotify8->SetNotificationPositions(g_nPositionCount, pPositionNotify));
ATLENSURE_SUCCEEDED(pDirectSoundBuffer->Play(0, 0, DSBPLAY_LOOPING));

Continue Reading »

Started as Is it possible to use local component in DLL? on microsoft.public.vc.atl newsgroup. The question is to embed a custom DirectX Media Object (DMO) into executable so that it is only available to proprietary application and not to entire system and could be reused in other applications. If in particular this DMO should be available to intelligent connect, e.g. used by embedded Windows Media Player control, the solution requires that private DMO appears fully registered as full featured regular DMO.

A while ago I proposed a solution on getting COM Class/DMO embedded into executable and temporarily exposed as availble without registry registration through CoRegisterClassObject/RegisterClassObject, however in addition to COM class registration, a DMO needs to also be specifically listed as a DMO, through DMORegister API.

In order to expose private DMO to DirectShow subsystem the following steps has to be performed:

• register COM class object as a COM class (CoRegisterClassObject) to enable its instantiation by its CLSID
• use DMORegister API to list DMO in global list (warning, seems to be requiring write permission on HKEY_CLASSES_ROOT key)
• have the filter graph of interest created
• use DMOUnregister API as early as possible to unlist private DMO from global list

ATLENSURE_SUCCEEDED(__objMap_CPrivateMediaObject.RegisterClassObject(CLSCTX_INPROC_SERVER, REGCLS_MULTI_SEPARATE));
DMO_PARTIAL_MEDIATYPE pInputMediaTypes[] = { { MEDIATYPE_Video, CPrivateMediaObject::GetXxxxSubtype() } };
ATLENSURE_SUCCEEDED(DMORegister(L"Private DMO", CLSID_PrivateMediaObject, DMOCATEGORY_VIDEO_DECODER, 0, _countof(pInputMediaTypes), pInputMediaTypes, 0, NULL));
...
ATLVERIFY(SUCCEEDED(DMOUnregister(CLSID_PrivateMediaObject, DMOCATEGORY_VIDEO_DECODER)));

The sample code/project takes an .AVI file which is a XVID video file with a FOURCC compression code changed from XVID to XXXX. The file is unplayable but there is a private DMO which accepts XXXX video and copies data as is into XVID format on its output.

A Visual C++ .NET 2008 source code is available from SVN.

I just read a post on frame pointer omission (FPO) optimization, also known as /Oy compiler flag. And a comment from Vladimir Scherbina on problems getting the feature actually work. I wondered if there has been any improvement with a Visual Studio .NET 2008 SP1. latest compiler from Microsoft and briefly - the result is yes, the feature seems to be working.

This is the reference code for the feature:

const volatile INT g_nValueA = 0;
const volatile INT g_nValueB = 0;

#pragma optimize("y", off)

__declspec(dllexport) __declspec(noinline) BOOL STDMETHODCALLTYPE A(const volatile INT& nValue = g_nValueA)
{
	return nValue != 0;
}

#pragma optimize("y", on)

__declspec(dllexport) __declspec(noinline) BOOL STDMETHODCALLTYPE B(const volatile INT& nValue = g_nValueB)
{
	return nValue != 0;
}

int _tmain(int argc, _TCHAR* argv[])
{
	A();
	B();
	return 0;
}

The code is to be compiled in Release and function A is to be not optimized while B is subject for optimization and omitting the frame pointer.

Disassembly shows for A:

Continue Reading »

Microsoft C++ compiler offers a buffer security check option since quite a long ago. It adds so called “security cookies” onto stack and checks them to be not modified by code in order to early-detect stack buffer overflow conditions. The compiler option /GS is enabled by default in Release configurations and is not applicable to debug code (which has optimizations disabled).

Here comes the trouble: release code might show unexpected message box “Buffer overrun detected!”. I have never seen such after years of development and tens of thousands of lines of code written, but luckily there are colleagues who can contribute a challenging issue to investigate.

The message is shown through C runtime’s __crtMessageBoxA function and is not only annoying and angering the customer, it also stops application operation and terminates it:

By default, an application that fails a security check displays a dialog box that states “Buffer overrun detected!” When the dialog box is closed, the application terminates.

However it is possible to override this behavior:

The CRT library offers the developer an option to use a different handler that would react to the buffer overrun in a manner more sensible to the application. The function, __set_security_error_handler, is used to install the user handler by storing the user-defined handler in the user_handler variable

In my case we would be overriding this to trace call stack at the issue point. However, let us see first what other conditions might cause a message box. CRT source has the following occurrences for__crtMessageBoxA:

• assert.c - assertion failure, which normally only applies to debug code
• crt0msg.c - _NMSG_WRITE function, typically for a fatal run time error messages
• crtlib.c - _CRTDLL_INIT fails to start on a Win32s system (VER_PLATFORM_WIN32s)
• crtmbox.c - implementation of __crtMessageBoxA (which is, among other things binds to API functions on runtime instead of static linking)
• dbgrpt.c - CrtMessageWindow function, applies to debug code only
• secfail.c - __security_error_handler function, the /GS thing

So it appears that hopefully /GS failure is the only surprising condition to display an unexpected message box.

OK, let us just see what we got finally here. Using the MSDN sample code and overriding default check failure handler:

int main()
{
	_set_security_error_handler(report_failure); // <<---------------- Override
	// declare buffer that is bigger than expected
	char large_buffer[] = "This string is longer than 10 characters!!";
	vulnerable(large_buffer);
}

Private handler is:

void __cdecl report_failure(int code, void*)
{
	ATLTRACE2(atlTraceGeneral, 0, _T("Security Error Handler: Code %d\n"), code);
	CONTEXT Context = { CONTEXT_CONTROL };
	if(GetCurrentThreadContext(&Context))
		CDebugTraceCallStack::TraceCallStack(Context);
}

and finally output:

BufferSecurityCheckSample.cpp(20): report_failure: Security Error Handler: Code 1
  BufferSecurityCheckSample!0x00402a2d report_failure (+ 125) [c:\buffersecuritychecksample\buffersecuritychecksample.cpp, 22] (+ 10) @00400000
  BufferSecurityCheckSample!0x00403d56 __security_error_handler (+ 48) [f:\vs70builds\6030\vc\crtbld\crt\src\secfail.c, 79] (+ 6) @00400000
  BufferSecurityCheckSample!0x00402e1d report_failure (+ 25) [f:\vs70builds\6030\vc\crtbld\crt\src\secchk.c, 117] (+ 9) @00400000
  BufferSecurityCheckSample!0x0040142c vulnerable (+ 44) [c:\buffersecuritychecksample\buffersecuritychecksample.cpp, 36] (+ 11) @00400000
  BufferSecurityCheckSample!0x00404019 mainCRTStartup (+ 371) [f:\vs70builds\6030\vc\crtbld\crt\src\crt0.c, 259] (+ 18) @00400000
  kernel32!0x7c817067 RegisterWaitForInputIdle (+ 73) @7c800000

The log file would immediately show the source of the run time problem.

See Also:

• /GS (Buffer Security Check)
• Compiler Security Checks In Depth

It is amazing how bugs go through versions and years. Visual Studio’s IDE dockable windows framework is cool, no doubt. But it was since 2005, if not earlier, that tool window undocked by a double click while application is running (debugging session active) is brought to wrong position and is tending to go out of screen area being re-shown.

This is exactly the behavior in 2008 version, regardless of reworked framework and fancy icons/hints/placeholders for the window being dragged. But just now it was even better. I double clicked “Find Results 1″ window to maximize the window… and it disappeared. After looking for it through menu and other GUI elements, I finally found it at the rightmost border of the secondary monitor:

The other day I “wrote some code” to workaround an extremely stupid hardware issues. The depth of idiocy is just incredible: to release a bunch of hardware that just don’t work, release a number of firmware updates that just don’t fix the simplest thing: HTTP compatibility. If there was a single little try to see how this piece of crap comminicates with any WinHTTP based application, an error 12152 ERROR_WINHTTP_INVALID_SERVER_RESPONSE on the first second of execution would imminently come up and demonstrate that someone has to be fired without any hesitation.

Things, however, definitely went a different way with hardware still on the shelves and no firmware upgrades available on the website. Our customer got into trouble having already recommended his customer the buggy thing in amount of X and forwarded us the question of getting everything working. As we decided to make a step towards, I needed to “write some code” to settle the problem.

However, the story was about a different thing. So in order to put a comment into code that explains what kind of problem we are dealing with, I copy/pasted a fragment of HTTP request/response content from Wireshark into source file being edited within MS Visual Studio .NET IDE. Wireshark copies text with some weird line endings, I knew that. I removed extra empty lines from pasted text and actually did not expect anything to go wrong. However who appeared to be wrong was me.

Initially there seemed to be no problem, I tried to compile code, fixed some compiler errors, even started application. I was somewhat confused that the application did not hit my breakpoint while it should. Then I noticed it did not even generate code for this line. As these things do happen with development environment, I re-opened IDE, deleting .NCB for the project and Rebuilt All. The problem however did not go. After further code modification, the compiler started giving errors and shown wrong lines in build output, which did not match source code line numbers.

This started being completely stupid: I was to look at wrong identifier, search through entire source file for occurrences, see if this particular occurrence might be actually a problem for compiler and so on. I made it compiled successffully but under debugger there were still wrong line numbers which made it impossible to debug and set breakpoints.

At this point I remembered Wireshark and line endings. Just removing the comments did not worked. And since visually everything was OK with the source, there should be an easy way found to normalize text. And what I did was the following: I started new message in Mozilla Thunderbird and copy/pasted entire source file content into Thunderbird’s editor. Then copy/pasted back into Visual Studio and finally got the things fixed.