Fooling Around // Software Production Line

The other day I “wrote some code” to workaround an extremely stupid hardware issues. The depth of idiocy is just incredible: to release a bunch of hardware that just don’t work, release a number of firmware updates that just don’t fix the simplest thing: HTTP compatibility. If there was a single little try to see how this piece of crap comminicates with any WinHTTP based application, an error 12152 ERROR_WINHTTP_INVALID_SERVER_RESPONSE on the first second of execution would imminently come up and demonstrate that someone has to be fired without any hesitation.

Things, however, definitely went a different way with hardware still on the shelves and no firmware upgrades available on the website. Our customer got into trouble having already recommended his customer the buggy thing in amount of X and forwarded us the question of getting everything working. As we decided to make a step towards, I needed to “write some code” to settle the problem.

However, the story was about a different thing. So in order to put a comment into code that explains what kind of problem we are dealing with, I copy/pasted a fragment of HTTP request/response content from Wireshark into source file being edited within MS Visual Studio .NET IDE. Wireshark copies text with some weird line endings, I knew that. I removed extra empty lines from pasted text and actually did not expect anything to go wrong. However who appeared to be wrong was me.

Initially there seemed to be no problem, I tried to compile code, fixed some compiler errors, even started application. I was somewhat confused that the application did not hit my breakpoint while it should. Then I noticed it did not even generate code for this line. As these things do happen with development environment, I re-opened IDE, deleting .NCB for the project and Rebuilt All. The problem however did not go. After further code modification, the compiler started giving errors and shown wrong lines in build output, which did not match source code line numbers.

This started being completely stupid: I was to look at wrong identifier, search through entire source file for occurrences, see if this particular occurrence might be actually a problem for compiler and so on. I made it compiled successffully but under debugger there were still wrong line numbers which made it impossible to debug and set breakpoints.

At this point I remembered Wireshark and line endings. Just removing the comments did not worked. And since visually everything was OK with the source, there should be an easy way found to normalize text. And what I did was the following: I started new message in Mozilla Thunderbird and copy/pasted entire source file content into Thunderbird’s editor. Then copy/pasted back into Visual Studio and finally got the things fixed.

Although I personally have no experience with tricky DirectShow filters that decide on possibility of connection not only looking at media type and other capabilities, there has been a number of cases mentioned that certain video decoders will only connect to renderers or video renderers in order to avoid interception of decoded data.

It was recently mentioned that one of such filters checked Misc Flags obtained through IAMFilterMiscFlags interface to make sure it is connected to renderer downstream. Before we proceed, let us make it clear that it is silly and can only protect from beginner, a complete newbie. At the very least it should have checked peer filter’s CLSID and compare against white list of stock video renderer: Video Renderer, Video Mixing Rendeder 7, Video Mixing Rendeder 9, Enhanced Video Renderer. A custom video renderer or even just a transformation filter with a renderer flag would be able to receive decoded data and make it available for any further processing.

What I am going to do is to take Sample Grabber Filter as a base and make a new filter from it which will only connect to renderer (it will check the flags as described above). Then another filter, which will also use Sample Grabber Filter base, will accurately fool the first one and connect to output of the first filter and will render video to complete the graph.

Read more »

A simple question being asked all over again. Given IBasicVideo, IBasicVideo2, IVMRWindowlessControl or IVMRWindowlessControl9, how to save image to file? It is easy. It is a bit easier with IBasicVideo because it is possible to query this interface directly from graph’s interface, such as  IGraphBuilder, and the call will be forwarded to video renderer. This code assumes internal bitmap format is non-paletted, which I believe is always the case.

LONG nBufferSize = 0;
ATLENSURE_SUCCEEDED(pBasicVideo->GetCurrentImage(&nBufferSize, NULL));
CHeapPtr<BITMAPINFO> pBitmapInfo;
ATLENSURE_THROW(pBitmapInfo.AllocateBytes(nBufferSize), E_OUTOFMEMORY);
ATLENSURE_SUCCEEDED(pBasicVideo->GetCurrentImage(&nBufferSize, (LONG*) (BITMAPINFO*) pBitmapInfo));
const BYTE* pnData = (const BYTE*) (&pBitmapInfo->bmiHeader + 1);
// NOTE: You might wish to handle <=8 bpp bitmaps here
ATLASSERT(pBitmapInfo->bmiHeader.biBitCount > 8);
ATLASSERT(pBitmapInfo->bmiHeader.biCompression == BI_RGB);
ATLASSERT(pBitmapInfo->bmiHeader.biSizeImage);
BITMAPFILEHEADER BitmapFileHeader;
ZeroMemory(&BitmapFileHeader, sizeof BitmapFileHeader);
BitmapFileHeader.bfType = 'MB';
BitmapFileHeader.bfSize = (DWORD) (sizeof (BITMAPFILEHEADER) + pBitmapInfo->bmiHeader.biSize + pBitmapInfo->bmiHeader.biSizeImage);
BitmapFileHeader.bfOffBits = (DWORD) (sizeof (BITMAPFILEHEADER) + pBitmapInfo->bmiHeader.biSize);
ATLENSURE_SUCCEEDED(File.Write(&BitmapFileHeader, sizeof BitmapFileHeader));
ATLENSURE_SUCCEEDED(File.Write(&pBitmapInfo->bmiHeader, pBitmapInfo->bmiHeader.biSize));
ATLENSURE_SUCCEEDED(File.Write(pnData, (DWORD) pBitmapInfo->bmiHeader.biSizeImage));

Given a DirectShow filter graph, what media samples are being sent through particular graph point? DumpMediaSamples utility gives a fast answer to this question. It prints out connection media type details (with details of VIDEOINFOHEADER, VIDEOINFOHEADER2 and WAVEFORMATEX structures corresponding to FORMAT_VideoInfo, FORMAT_VideoInfo2 and FORMAT_WaveFormatEx format types) and IMediaSample details obtained through AM_SAMPLE2_PROPERTIES structure.

First of all, it is necessary to create a graph of interest using GraphEdit utility. At the point of interest it is necessary to insert [an uninitialized] Sample Grabber Filter with the filter name “SampleGrabber” (this is the default name but if you add second filter which will be given a different name and remove first filter then, the utility would fail).

The graph may look like this:

Read more »

Mobotix cameras online, various models discovered by web spider bot, see previous list here:

Basic

• , http://85.47.147.28 - Firmware M1-V1.9.4.8 (2003-12-01)

D10D-Secure

• M10-1-38-123, http://mbsj.no-ip.org:7001 - Firmware M10-V2.1.0.9-D10 (2005-01-07)

D12D-Sec

• D12, http://cabanga.dvrdns.org:8080 - Firmware MX-V3.4.2.2 (2008-04-25)

Read more »

This continues the topic raised by previous post. As fairly noticed by The March Hare, video renderer is using hardware overlay and the benchmark is incorrect if we are to extrapolate the performance to scenario with multiple video renderers.

So, an updated test application creates 16 video renderers with 16 threads pumping two meida samples through each of the 16 filter graphs.

The screen shot shows that there is only one video overlay in use (which image was not captured and blackness is shown instead), so results may be inaccurate for one of the graph among 16. In this simple test I disregard this.

Here go the results (in all tests CPU usage is maxed out):

• YUY2 Source -> VMR: 3,480 fps
• YUY2 Source -> AVI Decompressor (converts to 24-bit RGB) -> Sample Grabber (without processing) -> Color Space Converter (converts to 32-bit RGB) -> VMR: 560 fps
• YUY2 Source -> AVI Decompressor (converts to 32-bit RGB) -> Color Space Converter -> VMR: 390 fps

Read more »

Started here: How can I overlay timestamp on the image? on microsoft.public.win32.programmer.directx.video

Let us see if RGB conversion adds any noticeable effect on streaming YUY2 video, typical output of video decompressor.

As a reference I am taking a simple YUY2 source -> Video Mixing Render Filter (VMR) graph, where source filter streams the same pre-allocated and pre-initialized data in an infinite loop:

while(WaitForSingleObject(TerminationEvent, 0) == WAIT_TIMEOUT)
{
	ATLENSURE_SUCCEEDED(m_pSourceFilter->InjectMediaSample(m_pnData, m_nDataSize));
	CRoCriticalSectionLock DataLock(m_DataCriticalSection);
	m_pnInjectedFrameCounts[0]++;
}

Video resolution is 640×480 pixels.

What is actually consuming CPU resources here is data copy into VMR’s media sample buffer and actually streaming. VMR might be blocking control waiting on rendering completion, I am leaving this for default VMR to decide (it might be hardware dependent etc).

Running at full pace, the application is rendering 510 frames per second consuming virtually no CPU. That is VMR is waiting until meida sample is rendered, this only allows streaming mentioned number of media samples per second, however rendering process does not take CPU resource, just waiting for video hardware to complete.

Read more »