Fooling Around

// Oprogramowanie Roman Ryltsov

A codec for The Imaging Source cameras and [discontinued] IC Capture 2.5 software

Posted on by Roman

Truly advanced industrial cameras by The Imaging Source are provided with image acquisition software, End User Software is a free IC Capture application, which ended up with a few problems to resolve.

First of all, “the older” application is a kind of discontinued. A Windows application version 2.5 was on the website as recently as a few weeks ago, but the announcement of release of IC Capture version 4 removed the older application completely.

Another The Imaging Source IC Measure application is renamed into the name of its predecessor IC Capture with a version increment… Now IC Capture 2.5 is almost gone. OK, almost – there are still some traces: https://www-st.theimagingsource.com/en-us/support/download/iccapture-2.5.1557.4007/

The question is this: the cameras are good enough to capture video with 12-bit color depth, e.g. 33U series. Is it possible to record the original video stream in certain popular video format for use in other applications?

Under “AVI Files and Codecs Support” the new application does not seem to offer AVI support anymore, and offers recording to MP4 files with H.264 or H.265 encoding. You could give it a try and record 4000×3000@20 with 12-bit wide dynamic range into one of these formats, hopefully without frame dropping.

The previous IC Capture 2.5 application in Windows had better integration options and did offer AVI recording with interoperability with third party software. It was still quite a challenge to find a suitable codec, yet it is possible now with a specialized GoPro CineForm codec designed to work with IC Capture 2.5:

  • the codec directly accepts The Imaging Source driver’s RGB64 format (original 12-bit color depth from camera without reduction to 8 bits) with WDR (wide dynamic range) option tuned on video
  • the codec retains 12-bit precision in the produced output (even though the output bitrate is pretty high!)
  • the codec does its best to not lose input frames, and uses the most of CPU with parallel processing, in order to save as much as possible from the original data acquisition; to give some estimate, an Intel i7 CPU of 12th generation or newer can handle the compression of WDR 4000×3000@20 video stream and enable its offline post-processing where all the data is securely recorded

Essentially, the codec gives fine tuning to discontinued IC Capture 2.5 extending its recording abilities so that they exceed those of current IC Capture 4 software in Windows… The produced files are nicely accepted into software like DaVinci Resolve, or good old Media Player Classic Home Cinema.

The download for the pre-release of the codec is below. Requirements:

  • Windows 10/11: minimal version is not identified, the software should be well compatible with a few last versions of Windows
  • 64-bit Windows and 64-bit of IC Capture 2.5 (x64): there is no 32-bit build as making no much sense; IC Capture 2.5 installs both Win32 and x64 variants of the application, please use x64 and there in the codec choice UI you will be able to see additional codec option
  • unregistered codec will overlay “DEMO CODEC” text, the overlay goes away with addition of valid license token for the codec (to be described later, or please get in touch for a quote)
  • the codec is designed to work only with IC Capture 2.5 x64 software
DirectShowCineFormCodec-v1.0.20250214.1Download

Demo: Webcam with YOLOv4 object detection via Microsoft Media Foundation and DirectML

Posted on by Roman

As we see announcements for more and more powerful NPUs and AI support in newer consumer hardware, here is a new demo with Artificial Intelligence attached to a good old sample camera application based on the Microsoft Media Foundation Capture Engine API.

This demo is a blend of several technologies: Media Foundation, Direct3D 11, Direct3D 12, DirectML (of Windows AI), and Direct2D, all at once!

The video data never leaves the video hardware. We have the video captured via Media Foundation, internally processed with the Windows Camera Frame Server service, and then shared through the Capture Engine API (this perhaps still does not really work as “shared access” but we are used to this). The data then is converted from Direct3D 11 to Direct3D 12 and, further, to a DirectML tensor. From there, we run the YOLOv4 model (see DirectML YOLOv4 sample) on the data while the video smoothly goes to preview presentation. As soon as the Machine Learning model produces predictions, we pass the data to a Direct2D overlay, which attaches a mask onto the video flow going to the presentation.

The compiled application’s only dependency is the yolov4.weights file (you need to download it separately and place in the same directory as the executable, there is an alternative download link); the rest is the Windows API and the application itself. As the work is handled by the GPU, you will see Task Manager showing GPU load close to 100% while CPU load is minimal.

The model is trained on 608 by 608 pixel images, and so is the model input and rescaling. That is, the resolution of the camera video does not matter much, except that the overlay mask is more accurate with higher resolution video—the overlay is burnt into the video stream itself. To show the recent progress in hardware capabilities, here are some new numbers:

  1. Intel(R) UHD Graphics 770 integrated in the Intel® Core™ i9-13900K CPU achieves around 2.5 fps for real-time video processing.
  2. AMD Radeon 780M Graphics integrated in the AMD Ryzen 9 7940HS CPU runs the processing up to four times faster, achieving around 8.0 fps.

Intel’s CPU is still the good one but its video is not so, and it’s GPU at work now. The AMD Ryzen 9 7940HS includes AMD’s dedicated AI XDNA Technology. The NPU performance is rated at up to 10 TOPS (Tera Operations Per Second), upcoming Copilot+ PCs are expected to have 40+ TOPS, so the new hardware should be a good fit for this class of applications.

Have fun!

Capture-20250112-f43fac8927726b3d389c12e7499ba3681259cc0cDownload

Minimal Requirements:

  1. Windows 11 x64 — DirectML might run on Windows 10 as well, but for the simplicity of the demo it requires Windows 11; Windows 11 ARM64 is of certain interest as well, but I don’t have hardware at hand to check)
  2. DirectX 12 compatible video adapter, which is used across APIs — for the simplicity of the demo there is no option to offload to another GPU or NPU
  3. The .weights file downloaded and placed as mentioned, besides the application in the archive above

Demo update: Web camera video with MPEG-DASH live broadcasting, now also with HLS

Posted on by Roman

Serving MPEG-DASH differs from serving HLS but as long as you have video packaged in ISO BMFF segments, adding an option to also expose content as HLS (HTTP Live Streaming, RFC 8216) is not too difficult.

Besides, being able to stream webcamera signal as MPEG-DASH using http://localhost/MediaFoundationCameraToolkit/Capture/manifest.mpd I also made it possible to use http://localhost/MediaFoundationCameraToolkit/Capture/master.m3u8 with, for example, the same Shaka Player Demo, or https://hlsjs.video-dev.org/demo/.

BTW, hls.js has even better visualization of media buffering:

Capture-20240804-9894f8a72fc402118ffd702cf76c54146a4b557aDownload

Demo update: Web camera video with MPEG-DASH live broadcasting, now with MP4 export feature

Posted on by Roman

Since the last demo appears to be quite nice, one another addition: ability to drop the internal memory video content of the application into MP4 file.

As the application works and shows video preview, it keeps H.264/AVC version of the data in memory in the form of sliding window. Now you can just hit F8 and have this video – the last two minutes, that is – written into an MP4 file.

What the application does is effectively this: it uses initialization segment from MPEG-DASH content, and concatenates all media segments that the server keeps ready. This way we have an ISO BMFF media file, of the fragmented MP4 flavor. It is playable but not nicely, so the application contintues and then right in the memory and on the fly it re-packages (there is a related bug in Windows Media Foundation which needs to be worked around, but it is what it is) this file into standard MP4 file, using here and there just Windows Media Foundation. This whole process is instant, of course.

And more to this, alternatively you can take this video snapshot even via web server! Just request http://localhost/MediaFoundationCameraToolkit/Capture/video.mp4 and you will have the application to do the same processing and preparation/export of video file, but it will be delivered to browser instead of saving to file system.

Have fun!

Capture-20240801-86cfb47182324e6006b2bae15bd532436add37e1Download