How to control IP Video Source JPEG video filter from C# programmatically

User’s question about Alax.Info IP Video Source DirectShow filter for JPEG/M-JPEG video:

As it stands now, in order to configure the video capture source filter, I have to instantiate it […], set the properties (camera URL, and video dimensions), then save the filter state to a data array which is then compiled into the application. That doesn’t provide a feasible system for user configuration at run time: Telling our customers we need to have the IP address(es) of their camera(s) so we can build a custom version of our software for them isn’t going to be very marketable.

[…]

I looked on the Alax.Info Web site, and didn’t see anything that looks like an API for configuration. I also didn’t see anything inside […] that looks like configuration parameters. I find it hard to believe this isn’t a common requirement, so I suspect I’m overlooking something.

Is there an “easy” way to configure the parameters of the Alax.info video capture filter at run time? Where is it documented?

Is there a structure that describes the state info that is captured [into raw configuration data array]?

Are there any other considerations I’ve overlooked?

C++ and C# sample code (Trac, SVN) appears to be not easy no discover. The code snippets include PlayJpegLocation (C++) and PlayJpegLocationSharp (C#) projects/snippets demonstrating import of IP Video Source COM interfaces via COM and use them from code.

IP Video Source comes with a COM type library which is well suited for import by development environments, specifically by Visual Studio as a reference for .NET projects. To avoid bitness issues, it is recommended to have both 32 and 64 bit versions of IP Video Source installed.

C# integration is straightforward:

First, one needs to add a reference to AlaxInfoIpVideoSource type library, the same way DirectShow.NET is added to the project. The type library is imported and is available via .NET COM Interop at this point (specifically, visible in Object Browser). In code, the definitions are in AlaxInfoIpVideoSource namespace, JpegVideoSourceFilter represents a filter class, with IJpegVideoSourceFilter interface offering filter configuration.

The filter is also compatible with generic COM persistence: it implements standard IPersistStream, IPersistStreamInit, IPersistPropertyBag and other interfaces, its state can be saved to stream or property bad and loaded back.

C++ #import and x64 builds

I already wrote earlier on 32/64-bit issues with Visual Studio. The problems are not frequent but when they happen they are pretty confusing. Here is another one today.

C++ code is simple:

    #import "libid:59941706-0000-1111-2222-7EE5C88402D2" raw_interfaces_only no_namespace

    CComPtr<IObject> pObject;
    ATLENSURE_SUCCEEDED(pObject.CoCreateInstance(__uuidof(Object)));
    BYTE* pnData;
    ATLENSURE_SUCCEEDED(pObject->Method((ULONG_PTR) (BYTE*) pnData));

A COM method returns a pointer to data – pretty straightforward, what could have gone wrong?

COM server vendor designed the library for easy .NET integration and defined the pointer argument as an integer value. They suppose the values to be used further with System.Runtime.InteropServices.Marshal class.

32-bit builds worked well and 64-bit builds experienced memory access violations. An attempt to consume the COM server from C# project showed the same problem: unexpected exception in the call.

The problem is that cross-compiler importing COM type library using LIBID takes 32-bit library even when it builds 64-bit code. This is the problem for both C++ #import "libid:..." and .NET COM reference using the identifier.

The type library imports as the following IDL in 32-bits:

[id(1)]
HRESULT Method(
                [in] unsigned long bufPtr);

It is supposed that 64-bit builds get the following import:

[id(1)]
HRESULT Method(
                [in] uint64 bufPtr);

Effectively though, 64-bit builds get the 32-bit import and the argument which is supposed to carry casted pointer value is truncated to 32-bits, ULONG type. Cast to ULONG_PTR in 64-bit C++ code is, of course, not helpful since it’s trimmed anyway further fitting the IDL argument type.

The same happens with C# build.

It was developer’s choice to publish ordinal type argument, they wanted this to be “better” and ended up in bitness mess. If the argument remained a pointer type in the IDL then even incorrect bitness would not necessarily result in value truncation.

All together it is unsafe to import [an untrusted] type library using LIBID when it comes to 64-bit builds. It’s 32-bit library to be taken and it can result in incorrect import. Instead, such build should explicitly point to 64-bit type library, for example:

#if defined(_WIN64)
    #import "Win64\ThirdParty.DLL" raw_interfaces_only no_namespace
#else
    //#import "libid:59941706-0000-1111-2222-7EE5C88402D2" raw_interfaces_only no_namespace
    #import "Win32\ThirdParty.DLL" raw_interfaces_only no_namespace
#endif

Too bad! libid looked so nice and promising.

Small correction for DirectShow BaseClasses CTransInPlaceFilter::Copy

False assertion failure in DirectShow BaseClasses transip.cpp, in CTransInPlaceFilter::Copy:

ASSERT(lDestSize >= lSourceSize && lDestSize >= lDataLength);
[...]
CopyMemory((PVOID) pDestBuffer, (PVOID) pSourceBuffer, lDataLength);

The code asserts on “lDestSize >= lSourceSize” which is true in most cases, but it does not have or need to be true. The code below copies just lDataLength bytes and assertion on buffer sizes is, generally, excessive. Valid scenarios do exist with this assertion popping up for no reason.

Build Incrementing for Visual Studio C++ Projects

Over long time I used an automatic build incrementer add-in for Visual Studio and C++ projects, which proved to be helpful. Having increments in file information, the binaries were easy to identify. It was easy to find a matching symbol information etc. Long story short, a tool like this has been a must.

The add-in has problems or downsides though. It kept patching the .RC source and touched it when no other changes existed in the build, touching source code forced rebuilds on its own and reloaded resource-related files opened in Visual Studio editors. I was annoying even though more or less acceptable.

Visual Studio 2015 Community Edition does not support add-ins because of 2015 or because it’s Community Edition. Either way it was time to update the incrementer ot make things nicer overall.

This time I preferred to change things a bit. No longer source code patching: the incrementer can be attached as a post-build event and patch VERSIONINFO resource on the built binary. This requires that current build number is kept somewhere but not in the .RC text, so I am using an additional .INI file. The good thing is that this file can still be included in version control system and the version history can be tracked relatively easily. No longer source code modification which makes code base dirty and forces another rebuild.

Command line syntax:

C:\>IncrementBuild-Win32
Syntax: IncrementBuild-Win32.exe argument [argument...]

Arguments:
  help - displays syntax
  configuration <path> - path to .INI file holding configuration information (mandatory)
  binary <path> - path to binary to be patched with file version update (mandatory)
  string <name> <value> - add, update or remove specific version information string (optional; multiple arguments possible)
  dump - print version information data block dump before and after update

Additional feature is that incrementer can attach additional version strings (see example below – it adds build configuration as a version information string).

Setting up is easy. First, the project should have a version information resource, so that the binary has data to patch in first place.

Then, there should be an .INI file which tracks version numbers. The binary will be build with .RC numbers and then incrementer will apply the least significant number from the .INI file incrementing it along the way.

[General]

[VersionInformation]
;Language=133 ;MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US)
;Version String Format=%d.%d.%d.%d
Current Build Number=4

Next thing, project post-build event needs a command for patching:

Post-Build Event in VS for C++ Project

"$(AlaxInfo_Common)\..\Utilities\IncrementBuild\_Bin\IncrementBuild-$(PlatformName).exe" configuration "$(ProjectDir)Module.ini" binary "$(TargetPath)" string "ConfigurationName" "$(ConfigurationName)" 

The command takes Module.ini from the projects directory for configuration file, patches build output and also attaches build configuration as an additional version information string.

Build output looks like this:

—— Rebuild All started: Project: EnumerateTransforms, Configuration: Release Win32 ——
stdafx.cpp
Application.cpp
Generating code
Finished generating code
EnumerateTransforms.vcxproj -> D:\Projects…_Bin\Win32\Release\EnumerateTransforms.exe
EnumerateTransforms.vcxproj -> D:\Projects…_Bin\Win32\Release\EnumerateTransforms.pdb (Full PDB)
Configuration Path: D:\Projects…\Module.ini
Binary Path: D:\Projects…_Bin\Win32\Release\EnumerateTransforms.exe
Incrementing build number, product version 1.0.0.1, file version 1.0.0.4
Applying version information string, name “ConfigurationName”, value “Release”

Presumably, it is not necessary to use same bitness tool for a binary, since version information patching API should be able to patch resources of mismatching build, but I normally use a matching tool anyway, why not?

Download links

ManifestComDependency: Adding/Removing Registration-Free COM dependencies

In one of the earlier posts I mentioned a problem with registration-free COM dependency setup up by Visual Studio as a part of build process. This post is about the tool that offers an alternate solution for the problems, and also gives more:

  • standalone, independent from build, addition/removal registration-free COM links
  • automation of registration-free COM links for native code builds

Unless one deals with ACTCTX activation contexts programmatically, taking advantage of registration-free COM is essentially a setup of respective side-by-side manifests, either standalone or embedded into executable. The manifests override standard global COM object lookup procedure and point to COM servers directly, in some way similar to the way static links to exported DLL functions work.

To enable, registration-free COM reference COM client needs to prepare respective manifest file (declare an assembly) and list its COM dependencies. The COM dependency list is not just COM server files but rather their COM classes and type libraries. That is, when it comes to automation, the tool needs to go over the COM servers and extract respective identifiers, so that it could properly shape this information and include into manifest XML.

To deal with the challenge, I am using ManifestComDependency tool (no better came to mind when it was created). The command line tool syntax is the following:

 ManifestComDependency <path> [options]
 
 Options:
  * /i[:<assembly-name>] - updates <assemblyIdentity> element replacing it with automatically generated
  * /a[:tc] <dependency-path>[|LIBID[;X.Y]] - adds file <dependency-path> dependency, optionally without type library and COM classes reference
  * /r <dependency-path> - removes dependency from file <dependency-path>
  * /e <export-path> - exports manifest to file <export-path>

The tool opens a file <path> for manifest modification. The file can be a binary file, such as .exe or .dll, in which case the tool will be working with embedded manifest resource. Or, if the path extension is .manifest or .xml, then the file is treated as standalone XML manifest file. Both options are good for manifests associated with .NET builds as well, even though this surely does not need to be managed code development – native COM consumers are okay.

/i option updates <assemblyIdentity> element in the manifest data because it is mandatory and in the same time it might so happen that it is missing or incomplete, which ends up in painful troubleshooting. The tool will name file name, architecture and version information from the binary resource information (product version; the VERSIONINFO resource has to be present).

ManifestComDependency Output

/a option adds a dependency, defined by either path or registered type library identifier the tool would use to resolve the path. t and c modifiers exclude type library and COM server information respectively. The tool created a file element for the dependency and populates it with COM servers discovered there. Type library is obtained using LoadTypeLib API and COM servers are picked up by a trick with RegOverridePredefKey and friends (e.g. see example in regsvr42 source or GraphStudioNext source – both are excellent sources of code snippets for the curious).

The task of COM server discovery assumes load of the library into process (LoadLibrary API call), which assumes bitness match between the tool and the loaded module. This is where you need both Win32 and x64 builds of the tool: if wrong bitness is detected, the tool restarts its sister build passing respective command line arguments, just like regsvr32 does with the only difference that regsvr32 uses binaries in system32 and syswow64 directories, and my tool looks for -Win32 and -x64 suffixes and the files in the same directory.

/r option removes dependencies by name. It is safe to remove certain dependency before adding it back because addition does not check if dependency already exists.

/e option writes the current version of the manifest in external file, esp. for review and troubleshooting if the actual manifest is embedded.

The order of options is important because they are executed one by one in the order of appearance. Then the result is saved/re-embedded once all tasks are done.

Command line example of re-adding registration-free COM dependencies:

ManifestComDependency-x64.exe BatchExport-x64.exe /i /r Player.dll /r mp4demux.dll /a Player.dll /a mp4demux.dll

This can be set as post-build event in the project settings, or put into batch file or run on-demand.

Download links

DirectShowSpy: Media Sample Traces

Overview

DirectShow filters pass media samples (portions of data) through graphs and details of how the streaming happens exactly is important for debugging and troubleshooting DirectShow graphs and development. A developer needs clear understanding of parts of streaming process, importance of which increases with multiple streams, threads, parallelization, cut-off times, multiple graphs working simultaneously.

Details of streaming is typically hidden from top level control over the graph where the application is controlling state of the process overall, and then filters are on their own sending data through.

DirectShowSpy provides API to filters to register media samples as well as other details of streaming process, including comments and application defined emphasis (highlighting), it stores the traces and provides UI to review and export the traces for analysis and troubleshooting.

A similar tracing is implemented by GraphStudioNext Analyzer Filter.

05

DirectShowSpy trace is different in several ways:

  1. DirectShowSpy is a drop-in module and adds troubleshooting capabilities to already built and existing application, including that it is suitable for temporary troubleshooting in production environment
    • DirectShowSpy offers tracing for filter which are private and not registered globally
    • DirectShowSpy tracing better reproduces production application environment
  2. DirectShowSpy allows supplementary application defined comments, which are registered chronologically along with media samples tracing
    • it is possible to trace not only at filter boundaries/granularity, but also internal events and steps
  3. DirectShowSpy combines tracing from multiple graphs, multiple processes and presents them in a single log

DirectShowSpy media sample trace is a sort of log capability, which is implemented with small overhead. The traces reside in RAM, backed by paging file and are automatically released with release and destruction of filter graph. The important exception is however the media sample tracing UI of DirectShowSpy. When UI is active, each (a) manual refresh of the view, and (b) each destruction of filter graph in analyzed process makes UI add a reference to trace data and data lifetime is extended up to closing of DirectShowSpy UI.

The important feature of behavior mentioned above is that media tracing data outlives, or might outlive the processes that host filter graphs. As long as UI is active, processes including terminated expose media sample trace for interactive review.

Basically the feature is to review a streaming session details obtained from the filters which registered the respective events. For example, this filter graph

01

has two filters, MPEG-4 demultiplexer and multiplexer, which register streaming events. Because the trace is chronological, it in particular allows seeings how “Stuff” filter is doing the processing: threads, timings. If “Stuff” filter registers its own events, the picture becomes more complete.

02

Using

To leverage media sample traces, a filter developer obtains ISpy interface from filter graph (which succeeds when DirectShowSpy is registered and hooks between application and DirectShow API) and creates a IMediaSampleTrace interface using ISpy::CreateMediaSampleTrace call. An example of such integration is shows in a fork of GDCL MPEG-4 filters here, in DemuxOutputPin::Active method.

It does not matter whether filter and pins are sharing IMediaSampleTrace pointers. Each CreateMediaSampleTrace creates a new trace object, which is thread safe on its own, and data is anyway combined on UI from all sources of tracing.

With no DirectShowSpy registered, QueryInterface for ISpy fails and this is the only expense/overhead of integration of media sample tracing in production code.

A developer is typically interested in registering the following events:

  • Segments starts, media sample receives and deliveries, end of stream events; DirectShowSpy introduces respective methods in IMediaSampleTrace interface: RegisterNewSegment, RegisterMediaSample, RegisterEndOfStream
  • Application defined comments, using RegisterComment method

All methods automatically track process and thread information associated with the source of the event. Other parameters include:

  • filter interface
  • abstract stream name, which is a string value and can be anything; typically it makes sense to use pin name/type or it can be pin name with appended stage of processing if developer wants to track processing steps as they happen in the filter; UI offers filtering capability for stream values, and separate column in exported text so that filter could be applied in spreadsheet software such as Excel when reviewing the log
  • user defined comment and highlighting option

RegisterMediaSample methods can be used with anything associated with a media sample, not exactly one event per processing call. The method logs media sample data (it takes AM_SAMPLE2_PROPERTIES pointer as byte array pointer) and makes it available for review with its flags and other data.

Comments can be anything and are to hold supplementary information for events happening in certain relation to streaming:

03

An application can automatically highlight the log entries to draw attention to certain events. For example, if data is streamed out of order and the filter registers the event with highlighting, the entry immediately drawing attention on UI review. Then interactive user can change the highlighting interactively as well:

04

The media trace data can be conveniently filtered right in DirectShowSpy UI, which is invoked by DoMediaSampleTracePropertySheetModal exported function, or copied to clipboard or saved as file in tab-separated values format. The file can be opened with Microsoft Excel for further review.

Limitations

  • there is a global limit on in-memory trace storage; there is no specific size in samples (it’s 8 MB for global registry of data here) and the storage is large enough to hold streaming of a movie with multiple tracks, however once in a while it is possible to hit the limit and there is no automatic recycling of data: the data is released with last copy of UI closed and graphs released in context of their respective processes
  • traces are visible from the same session only, in particular processes with elevated privileges are “visible” by similarly started DirectShowSpy UI and vice versa
  • 32-bit process traces are visible from 32-bit DirectShowSpy UI, and the same applies to 64 bits; technically it is possible to share that but it is not implemented

Download links

Additional stuff

A fork of GDCL MPEG-4 filters is uploaded to GitHub, which in particular has integration with media sample tracing and includes pre-built binaries, 32- and 64-bit versions.

Crashes in Visual C++ 2012 vs. Visual C++ 2010

Great news for those suffering from Visual Studio 2010 IDE crashes with losing recent source code changes. Visual Studio 2012 is way more stable (event with Visual Studio 2010 Platform Toolset!) and suffers from crashes without losing editor changes.

The worst thing you seem to be getting is:

Which is an access violation or stack overflow issue in a child process, so that the crash does not kill the IDE itself. The problems seem to be all around cpfe.dll, and they are so less annoying.

If you happen to experience anything of the kind, be sure to vote the bug report on MS Connect: Memory access violation and unhandled exception in cpfe.dll while editing C++ source code with Visual Studio 2012. Microsoft is still accepting feedback on this release and will hopefully resolve the problem in future VS 2012 updates.