Fooling Around

From a conversation:

Q: I am using Web camera with DirectShow. Camera has only capture pin, so I am looking for the best way to switch resolution on the pin while graph is running. I would like to keep preview with smaller image size and when i would like to capture an image i would like to switch to full resolution. I have to use sample grabber callback since i need access to single image to process them. Is there a way to do that without stopping and starting a graph? This works but very slow between captures.

Is it possible to just reconnect the sample grabber only with new media changes and then resume the graph? Currently i just kill the graph and start it again, but it takes few seconds to do so, so I am looking the way to reduce that time.

There is no way to switch resolution on the running graph. There is a technique to dynamically start/stop individual filters and re-negotiate resolutions (media types), but it does not work for the majority of filters. Additionally to that Sample Grabber Filter cannot change resolution too, as it passes data through.

If you only have to use switched resolution for preview, you can use Geraint’s GMFBridge Toolkit to join two graphs, and a filter that changes resolution. Combining all that you will have a running capture graph that [also] renders video to a bridge sink. In the other graph you have another bridge sink that receives video from first graph and then you resize video to the resolution of interest already in the second graph. You can stop and reconfigure only second graph to update resolution and have first graph running and capturing. This is the best you can do, or just stop your single graph and change resolution this simple way.

So a solution, which is used by many, and I can recommend it too, is to use GMFBridge bridging. Additionally, you can find questions and answers on it on MSDN DirectShow Development Forum (search for “bridge” there). With a certain effort you can duplicate this with your own code but this is more or less ready to use solution and, again, the key advantage you have that you have two graphs which you can top independently.

Would it be possible to use smart tee as a splitter on the capture pin, and then use two sample grabbers on capture and preview, one with smaller resolution and the other one with higher one?

With a Smart Tee Filter you will still have 1 graph, so no individual resolution changes without stopping the graph. Additionally to that, Smart Tee Filter will deliver same frame on its output pins, so they will have to have one resolution and no resizing takes place inside.

Also, is it possible to run 2 graphs in the same time? Same device, 2 graphs, 2 sample grabbers.

Most likely no, for only one reason: you won’t be able to have two running filters for the same device, as source filter will exclusively lock the device. So capture filter will be a single filter. You can use Infinite Tee Pin Filter to split stream between 2+ processing lines. And you can use the same bridge to pass data into another graph for further processing.

A 100-lines code snippet which illustrates how a bitmap is overlaid over displayed video with Video Mixing Renderer 9 Filter using IVMRMixerBitmap9 interface. A video clip is played (default is Windows clock.avi, but you can replace it with your longer one to see overlay is really in a loop).

http://code.assembla.com/…/VmrMixerBitmapSample01/…

VMR9AlphaBitmap AlphaBitmap;
ZeroMemory(&AlphaBitmap, sizeof AlphaBitmap);
AlphaBitmap.dwFlags = VMR9AlphaBitmap_hDC;
AlphaBitmap.hdc = Dc;
AlphaBitmap.rSrc = CRect(0, 0, 32, 32);
AlphaBitmap.rDest.left = (FLOAT) 0.75;
AlphaBitmap.rDest.top = (FLOAT) 0.75;
AlphaBitmap.rDest.right = (FLOAT) 0.95;
AlphaBitmap.rDest.bottom = (FLOAT) 0.95;
AlphaBitmap.fAlpha = 0.75;
const HRESULT nSetAlphaBitmapResult = pVmrMixerBitmap->SetAlphaBitmap(&AlphaBitmap);
ATLENSURE_SUCCEEDED(nSetAlphaBitmapResult);

With a low FPS clip like clock.avi it is clear that the overlaid image is only updated with the next “main” video frame.

Visual C++ .NET 2008 source code is available from SVN, release binary included.

This is a funny one and I wonder what is exactly causing such misbehavior. An UDP broadcasting socket is being bound to IP address 192.168.0.2:

__E(m_Socket.Create(SOCK_DGRAM, IPPROTO_UDP) != INVALID_SOCKET);
static const DWORD g_nBroadcastValue = 1;
__E(m_Socket.SetOption(SOL_SOCKET, SO_BROADCAST, &g_nBroadcastValue, sizeof g_nBroadcastValue));
ZeroMemory(&m_LocalAddress, sizeof m_LocalAddress);
m_LocalAddress.sin_family = AF_INET;
m_LocalAddress.sin_port = htons(nPort); // 7365
m_LocalAddress.sin_addr.S_un.S_addr = htonl(nLocalIpAddress); // 192.168.0.2
_Z4(atlTraceGeneral, 4, _T("m_Socket %d, m_LocalAddress.sin_addr %hs\n"), m_Socket, inet_ntoa(m_LocalAddress.sin_addr));
__E(bind(m_Socket, (SOCKADDR*) &m_LocalAddress, sizeof m_LocalAddress) != SOCKET_ERROR)

Then off this socket a broadcast message is sent using sendto:

SOCKADDR_IN RemoteAddress;
ZeroMemory(&RemoteAddress, sizeof RemoteAddress);
RemoteAddress.sin_family = AF_INET;
RemoteAddress.sin_port = htons(7364);
RemoteAddress.sin_addr.S_un.S_addr = INADDR_BROADCAST;
_Z4(atlTraceGeneral, 4, _T("BroadcastSocketData.m_Socket %d\n"), BroadcastSocketData.m_Socket);
__E(sendto(BroadcastSocketData.m_Socket, NULL, 0, 0, (const SOCKADDR*) &RemoteAddress, sizeof RemoteAddress) != SOCKET_ERROR)

The problem however is that WireShark shows the message going off the wrong network interface, with IP address 192.168.56.1, while 192.68.0.2 is an IP address on a different network adapter:

C:\>ipconfig /all

Ethernet adapter Home Area Connection:

        Connection-specific DNS Suffix  . :
        Description . . . . . . . . . . . : Intel(R) 82566DC-2 Gigabit Network Connection
        Dhcp Enabled. . . . . . . . . . . : No
        IP Address. . . . . . . . . . . . : 192.168.0.2
        Subnet Mask . . . . . . . . . . . : 255.255.0.0
        Default Gateway . . . . . . . . . : 192.168.0.1
        DNS Servers . . . . . . . . . . . : 192.168.0.1
                                            4.2.2.4

Ethernet adapter VirtualBox Host-Only Network:

        Connection-specific DNS Suffix  . :
        Description . . . . . . . . . . . : VirtualBox Host-Only Ethernet Adapter
        Dhcp Enabled. . . . . . . . . . . : No
        IP Address. . . . . . . . . . . . : 192.168.56.1
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        IP Address. . . . . . . . . . . . : fe80::a00:27ff:fe00:5ce8%7
        Default Gateway . . . . . . . . . :
        DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                            fec0:0:0:ffff::2%1
                                            fec0:0:0:ffff::3%1

The problem appears to be in collision of networks: 192.168.0.0/16 and 192.168.56.0/24. It seems that nevertheless the socket is bound to proper NIC, where the message is expected to be sent through, Windows Sockets takes a random matching  adapter for the transmission.

A correction of the IP address from 192.168.56.1 to 192.169.56.1. immediately fixes the problem. There however has been no problems with other applications, the collision did not bring any other issues.

Some more information on the topic: Windows XP Professional, 32-bit, Service Pack 3.

See also on MSDN Forums: http://social.msdn.microsoft.com/Forums/…/

Unlike video, audio is difficult to troubleshoot in a way that once an issue has been noticed it was dynamic and already in past and it may be difficult to repeat, explain, forward as a screenshot etc. So it needs a visualization in order to check certain specific of the media stream.

Here we go with a Audio Oscillogram Filter, a filter which may be inserted into audio part of the graph, it insists on PCM audio (multichannel is OK, 8 or 16 bits per sample), so decoders might be required upstream and will be automatically inserted.

Once the filter is running, it shows a tool window with a visualization of the waveform coming through. Additionally, it shows a discontinuity sample (that is, a sample with AM_SAMPLE_DATADISCONTINUITY flag)  with a thick read vertical line, and a mismatch (gap or override) in neighboring sample start and stop times with a thin red line.

Additionally to this, a window provides an option to pop up downstream audio renderer property sheet in order to check/debug rendering statistics, such as buffer fullness and break count (programmatically available via IAMAudioRendererStats interface). Unfortunately Microsoft does not provide a well done proxy/stub pair for this interface and the property page and there is no way to access this property page connecting to graph remotely. Using this filter the property sheet is shown directly from the hosting process and works correctly.

A partial Visual C++ .NET 2008 source code is available from SVN, release binary included.

File and Class Summary

Utility.dll

Utility.dll (download) hosts the following classes:

• DirectShow Filters
  • Audio Oscillogram Filter, to visualize/debug audio data

Class Overview

Audio Oscillogram Filter

The filter visualizes PCM audio data going through in a form of oscillogram.

• Filter Executable: Utility.dll
• Filter CLSID: __uuidof(PcmAudioOscillogramFilter) {1450C047-C394-469e-BBB6-A1F949B893FA}
• Filter Property Page CLSID: N/A
• Filter Merit: MERIT_DO_NOT_USE
• Filter Category: CLSID_LegacyAmFilterCategory
• Filter Interfaces: IPersistStreamInit, IPersistStream, IBaseFilter, IMediaFilter, IPersist, IAMovieSetup, IAMFilterMiscFlags, IPcmAudioOscillogramFilter, IDispatch
• Filter Pins: single input pin (Input), single ouput pin (Ouput)
• Input Pin Media Types: MEDIATYPE_Audio/MEDIASUBTYPE_PCM
• Input Pin Interfaces: IPin, IMemInputPin
• Output Pin Media Types: MEDIATYPE_Audio/MEDIASUBTYPE_PCM
• Output Pin Interfaces: IPin

Remarks

The filter will create a tool window with audio visualization when put into paused/running state. The filter graph should be controlled (IMediaControl) from a GUI aware apartment so that window messages could reach internal filter’s window.