One of the popular questions asked in relation to DirectShow filters, and also a popular task is to modify video stream while in playback. There are various reasons one might need to do it, including keying video to replace color/range, or apply text or time code on top of video, including while re-compressing footage, adjust brightness or contrast.
DirectShow BaseClasses/SDK include samples (in most cases EzRGB24 sample is the best to start for a video effect, also demonstrates use of private interface on a filter) to quick-start with the task without getting too much into detail and once this part is done, next step is to integrate filter into playback, connect it with other filters.
As a result of DircetShowflexibility, there are ways to do things not so good, while still being under impression of keeping right track.
File playback is one of the basic tasks with DirectShow. To play a file, one creates a filter graph using powerful helpers provided by Filter Graph Manager object. It might be a actually a single call IGraphBuilder::RenderFile which takes all the complexity of finding matching filters, connecting them together, dealing with splitters and renderers, video and audio. A single call resolves the problems in a convenient way – easy.
Still a simple thing of inserting your own video transformation filters breaks the simplicity. One needs to build the graph partilly, insert the effect and complete building, or build the thing and break in with a new filter. How to find insertion point? Will the other filters like intrusion? Different file types and formats.
There is an easy and elegant solution to pre-add your own effect filter into graph and start rendering a file from there. Sounds reasonable and sometimes works. The problem is however that it does not work always, and you never know when it lets you down. The graph might be build and the effect filter is never taken and is left orphaned aside of playback pipeline.
Reliable graph building assumes you are in control over building steps and allow only the level of flexibility required to connect and build parts – and this is where Intelligent Connect is still a powerful helper. With an effect, the parts are “before the effect” and “after the effect”. RenderFile is no longer an option, and one has to dive deeper into graph building API.
First of all, the building starts with the file itself: unlike RenderFile, IGraphBuilder::AddSourceFilter method adds just the first filter for a given file/format. It stops there and lets caller continue building as required. At this point, it is the right time to manually add effect filter with IFilterGraph::AddFilter (IGraphBuilder is inherited from IFilterGraph and exposes the same method).
Having both ends in the graph for the “before the effect” part, intelligent connect can be used to connect and add filters required to make the connection. For an arbitrary file format, the task may be not trivial: depending on format and installed software components, the chain may look rather different. First, some filters combine stream splitting capability with immediate access to file (or another type of resource), others rely on joint operation of Async File Source filter with a format-dependent splitter filter. Some expose elementary stream pins immediately, some provide transport stream pin.
There may be a few approaches as for addressing pin connection task (see also General Graph-Building Techniques on MSDN). Straightforwardly, one might want to call IGraphBuilder::Connect and take advantage of intelligent connect. Before this can be done, however it takes caller to select a pin of the obtained source filter to start from. There might be a few pins, including those exposing video formats, non-video formats and pre-split formats where video is behind depacketizing (demultiplexing). Considering variety of formats and support, it might make sense to make a first attempt finding a video pin (by enumerating pins and their media types, looking and AM_MEDIA_TYPE::majortype and comparing to MEDIATYPE_Video) and, if not found, taking a first output pin of any type, or going through ping trying to connect first one which succeeds in connection.
An alternate approach is to take advantage of a helper object: Capture Graph Builder. While originally it is intended to help capture graph building, it contains useful methods for general building and connecting pins. It does not own a graph itself: it is designed to operate on top of existing graph, provided by its owner. So one need to provide its existing graph and call helper methods for easy graph building. One of the methods is ICaptureGraphBuilder2::RenderStream, which connects pins of given filters. Unlike API discussed earlier, it takes filter interfaces on input and will be responsible for finding proper pins itself, which might be a good idea if you don’t want to bother yourself doing it. To specify the requested task, it takes media type argument, which in this case might be video or, if fails, any type provided that video media type will still be anyway checked on input of effect filter.
Once the part “before the effect” is done, the other part may be completed as simple as calling IGraphBuilder::Render on the output pin of the effect. This will correspond to final step of original RenderFile execution.
A tiny Visual Studio 2010 C++ project illustrates discussed techniques and it available at SVN repository: RenderStreamTest01:
- for a given media file in line 109 the project will start graph building
- a suitable replacement for a video effect filter will be a Sample Grabber filter initialized with a video type (24-bit RGB, but the line 137 can be commented out)
- switch in line 147 switches between base Connect approach and Capture Graph Builder helper
- while message box is on the screen and also showing building status, the graph can be looked at using Graph Edit or similar tool, provided that DirectShow Spy is installed; alternatively you might want to put the graph onto ROT manually
The project also illustrates a solution for recent problems referencing Sample Grabber with new SDK. Sample Grabber was obsoleted and removed from Window SDK definition file (qedit.h). In order to resolved the problem without using an older version of SDK, the definitions might be imported from type the corresponding library and (apart from used as such) copied into the source code directly, as in lines 13-60.
See also on graph building: