Fooling Around

// Oprogramowanie Roman Ryltsov

MediaTools to deliver video from network/IP cameras and video servers into DirectShow environment

Posted on by Roman

I decided to gather DirectShow code and filters related to processing video from network/IP cameras and video servers into a library (in fact, a few libraries) so that it could be easily used for testing, research and other purposes. As time is going to permit, documentation and sample code will be provided, further development will be carried out.

This initial post publishes two libraries (DLLs) which host DirectShow filters to receive JPEG or M-JPEG video from network using HTTP based protocol, decode video with the help of Intel IPP 6.0 library (UMC version of the JPEG implementation), as well as perform additional helpful features, including writing series of frames into .JPG files and emulating video feed from saved .JPG files, conversion filters to obtain video in 24-bit and 32-bit RGB formats, YUY2 and YV12 formats.

File and Class Summary

Acqusition.dll

Acqusition.dll (download) hosts the following classes:

  • DirectShow Filters
    • HTTP Stream Source Filter, to receive HTTP content and deliver into DirectShow environment through output pin
    • JPEG HTTP Stream Parser Filter, to parse HTTP content (typically received from HTTP Stream Source Filter) of image/jpeg and multipart/x-mixed-replace types and output JPEG frames
    • JPEG Multi File Renderer Filter, to write parsed JPEG frames (typically received from JPEG HTTP Stream Parser Filter) into files
    • JPEG Multi File Source Filter, to read JPEG files (typically written by JPEG Multi File Renderer Filter) and stream video into DirectShow environmentto emulate video feed

Acqusition.dll is dependent only on well known DLLs and does not require presence/redistribution of specific dependency files.

CodingI.dll

CodingI.dll (download) hosts the following classes:

  • DirectShow Filters
    • JPEG Frame Decoder Filter, to parse JPEG data and pass frames downstream with a corresponding mediatype provided with VIDEOINFOHEADER format and correct resolution; in case of resolution changes the filter is capable of dynamically re-agreeing media type with downstream peer
    • JPEG Decoder Filter, to decode JPEG data into 24-bit/32-bit RGB (unfortunately Intel IPP codec has limited capabilities of decoding video into YUV pixel formats)
    • YUY2 Encoder Filter, to convert RGB data into YUY2 pixel format
    • YV12 Encoder Filter, to convert RGB data into YV12 pixel format
  • Shell Extensions
    • JPEG File Resolution Shell Property Page, to provide additional information (such as sampling, color format, resolution) about .JPG and .JPEG files through a shell property page

CodingI.dll is dependent on Intel IPP 6.0 library DLLs and relies on availability of these dependencies in the system. For the CodingI.dll to be operationable, it is required to have a free, trial or registered version of Intel IPP library installed. In particular, the files that are sufficient to be present/redistributed are:

  • libguide40.dll, libiomp5md.dll
  • ippcore-6.0.dll
  • ippi-6.0.dll, ippj-6.0.dll, ipps-6.0.dll, ippcc-6.0.dll, ippvc-6.0.dll
  • ippipx-6.0.dll, ippjpx-6.0.dll, ippspx-6.0.dll, ippccpx-6.0.dll, ippvcpx-6.0.dll

Quick Usage Example

Given a M-JPEG compatible network camera on IP address 98.76.54.32, the graph constructed as shown below is capable to deliver video data and render it through standard Video Mixing Renderer Filter.

Assume the camera is StarDot NetCam SC series, with /jpeg.cgi HTTP request to query for JPEG image and /nph-mjpeg.cgi HTTP request to query for M-JPEG video feed.

To construct the graph in GraphEdit (GraphStudio), first it is required to manually add Alax.Info HTTP Stream Source Filter and provide HTTP URL for the request in the property page:

Then an Alax.Info JPEG HTTP Stream Parser Filter is added and connected downstream to the previously added source filter:

A this point it is possible to automatically render output pin and DirectShow intelligent connect will add the rest of required filters, Alax.Info JPEG Frame Decoder (I), Alax.Info JPEG Decoder (I) and Video Renderer.

However, in order for the graph to run in GraphEdit it is required to manually provide correct video resolution in the properties of JPEG Frame Decoder Filter and once applied reconnect all downstream connections up to video renderer in order to enforce new media type.

Running the graph will start streaming video.

Class Overview

HTTP Stream Source Filter

The filter sends one or series of HTTP requests to receive data from network location and stream received data into DirectShow environment.

Remarks

Filter is a live source and delivers raw data received from network HTTP request(s) in an endless manner. In case the HTTP response is of a finite Content-Length, filter will continuously repeat request. Each first media sample of a response is marked with AM_SAMPLE_DATADISCONTINUITY flag.

Filter exposes connection point and events for the top level application to be able to receive error events as well as callbacks to modify HTTP request verb and object name immediately before the request is sent to network.

Filter is using custom memory allocator through which is exposes IServiceProvider interface on the media samples through which downstream filters could gain access to IHttpStreamSourceHeaders interface and obtain HTTP response headers.

JPEG HTTP Stream Parser Filter

The filter takes data stream on the input and parses streams of image/jpeg and multipart/x-mixed-replace MIME content types into JPEG frames, so that each output media sample contains one JPEG frame.

Remarks

Filter is using custom memory allocator through which is exposes IServiceProvider interface on the media samples through which downstream filters could gain access to IHttpStreamSourceHeaders interface and obtain HTTP response headers, both original response and subheaders of multipart Content-Type parts.

JPEG MultiFile Renderer Filter

The filter received JPEG frames and writes data into separate .JPG files, one per frame, into provided directory.

Remarks

None.

JPEG MultiFile Source Filter

The filter reads contents of provided directory and stream contained .JPG files emulating a video feed.

Remarks

The filter converts file time stamps into media sample time stamps to convert file write rate into DirectShow rendering/playback rate.

JPEG Frame Decoder Filter

The filter takes raw JPEG input media samples in the input, parses JPEG headers and outputs media samples supplied with valid VIDEOINFOHEADER format with correct resolution.

Remarks

Filter is attempting to dynamically re-agree media type with downstream filter in case media type changes. Initialially in absence of data about effective video resolution, the filter uses default resolution (320×240 or overridden through private interface IJpegFrameDecoderFilter.

Filter is attempting to share memory allocators on input and output pins to be able to pass media samples transparently without copying data.

Filter exposes connection point and events for the top level application to be able to receive resolution change events.

JPEG Decoder Filter

The filter decodes input media samples of JPEG� format and outputs 24-bit/32-bit RGB media samples.

Remarks

Filter is attempting to re-agree media type on the output pin in case it receives a request to change media type on the input pin, in order to transparently support dynamic resolution change.

YUY2 Encoder Filter

The filter converts 24-bit and 32-bit RGB media samples into YUY2 pixel format.

Remarks

Filter is attempting to re-agree media type on the output pin in case it receives a request to change media type on the input pin, in order to transparently support dynamic resolution change.

YV12 Encoder Filter

The filter converts 24-bit and 32-bit RGB media samples into YV12 pixel format.

Remarks

Filter is attempting to re-agree media type on the output pin in case it receives a request to change media type on the input pin, in order to transparently support dynamic resolution change.

14 Replies to “MediaTools to deliver video from network/IP cameras and video servers into DirectShow environment”

  1. One of the visitors complained that sequence of filters could not stream M-JPEG video from Panasonic BL-C140 (also in Russian) camera (with latest firmware 3.51R00). The problem may apply to different models too. It appeared that the camera generated incorrect multipart MIME stream and provided incorrect boundary. Today’s SVN update works the problem around.

    For reference: Information request: /Get?Func=Model&Kind=1, JPEG request: /SnapshotJPEG?Resolution=320×240&Kind=1, M-JPEG request: /nphMotionJpeg?Resolution=320×240&Kind=1

  2. More on Panasonic BL-C140:

    In 320×240 it seems to be OK now, but sometimes there is a small square
    irregularly blinking in the lower right corner of the picture:

    looking on the live picture on the web page of the camera I can’t see it.

    I noticed that and frankly I am not sure what exactly is causing it. It may be an incomplete frame sent by camera (JPEG with a few trailing bytes cut out) OR incorrect Content-Length reported by camera OR incorrect JPEG decoding by Intel IPP code. Or definitely by bug possible too. I will look into that later.

    The other issue is that in 640×480 the video renderer receives no frames.
    This is what I did:
    then “render” on the output PIN
    and then “play”
    this is the result: (notice the slightly different window size)

    in VLC, the mentioned URL works without problems and in the right resolution.

    I just tried with http://camera/nphMotionJpeg?Resolution=192×144, and in this resolution, the graph won’t run, too.

    The problem with different resolutions is that filters don’t handle dynamic resolution change. They connect at 320×240 and later cannot switch to different resolution. I have dynamic resolution change attempt in my filters but Video Mixing Renderer does not seem to support that to full required extent.

    I known it would support as a part of dynamic reconnection, but I did not implement that in my filters (and it is a hosting application who rather has to do that).

    So in my sample Samples/RenderHttpMjpegVideo01 note that application connects to event of JPEG Frame Decoder using DispEventAdvise and catches resolution changes. Later is this seems to be a cause of filter graph error (_tprintf(_T(“Detected resolution change…) it recreates the entire graph with new resolution.

    In GraphEdit, you need to change URL in source filter, change default resolution as you did in frame decoder, then:

    • remove JPEG Decoder filter
    • reconnect JPEG Frame Decoder and Video Renderer, so that connect on
      640×480 resolution

    Press Play and your’re fine again!

  3. A sample application RenderHttpMjpegVideo01 that takes a HTTP URL as a command line argument and renders M-JPEG video was updated to be compiled in more or less standard environment: Visual Studio with ATL, Windows/Platform SDK, WTL.

    C:\>RenderHttpMjpegVideo01.exe http://e6-demo1.stardot.com/nph-mjpeg.cgi
    URL: http://e6-demo1.stardot.com/nph-mjpeg.cgi
    Event: nCode EC_CLOCK_CHANGED 0xD, nParameter1 0x00000000, nParameter2 0x00000000
    Event: nCode EC_PAUSED 0xE, nResult 0x00000000, nParameter2 0x00000000
    Event: nCode EC_VMR_RECONNECTION_FAILED 0x55, nResult 0x80004005, nParameter2 0x00000000
    Event: nCode EC_VMR_RENDERDEVICE_SET 0x53, RendererDeviceType 0x1, nParameter2 0x00000000
    Event: nCode EC_VIDEO_SIZE_CHANGED 0xA, Extent 384×240, nParameter2 0x00000000
    Event: nCode EC_COMPLETE 0x1, nResult 0x00000000, nParameter2 0x00000000
    Event: nCode EC_ERRORABORT 0x3, nResult 0x80004005, nParameter2 0x00000000
    Detected resolution change, new resolution is 352×240, recreating filter graph…
    Event: nCode EC_CLOCK_CHANGED 0xD, nParameter1 0x00000000, nParameter2 0x00000000
    Event: nCode EC_PAUSED 0xE, nResult 0x00000000, nParameter2 0x00000000

  4. I have written C++ apps using DirectShow several years ago. I need to write a directshow application that takes MJPEG video from an IP/Network based camera…

    So I went looking on the internet and stumbled on your website… I have downloaded the “Aquistion.dll” and the “CodingI.dll”… Do these dll somehow expose fitlers to the GraphEdit application? And then they ought to also be available to MSVC C++ apps as well.. right?

    Yes, these host filters. “regsvr32 Acquisition.dll” from command prompt and the filters are already there in available graphedit. Also check https://alax.info/blog/category/utilities/media-tools from bottom to top and this gives a brief explanation of the contained filters.

    Also RenderHttpMjpegVideo01 and RenderHttpMjpegVideo02 is a C++ code which uses these filters.

  5. I’m just browsing your website. I don’t know if you provide the MPEG4 live view of Axis demo camera?? I can live view these demo camera in MJPG, but I am looking for MPEG4 live view. Can you tell me how to get it? Thank you very much!! :)

    I have not put any MPEG-4 code available yet, and it is unlikely happen in the nearest future. However as long as Axis cameras are in question, you can use any RTSP client to access media feed from them.

  7. Is there a licensing requirement for using this software in a commercial application, not for redistribution?

    At the very moment I grant permission to use the binaries in commercial application provided that:

    • binaries are provided “as is” without any implied warranty (though normally I am responsive to reported bugs, especially accompanied with sufficient details to effectively troubleshoot them)
    • licensee agrees to use provided binaries without any modification
    • licensee agrees to settle licensing matters for Intel IPP (if used) on his own
    • licensee agrees to settle possible patent matters, such as for example for MPEG-4/H.264 video formats, on his own
    • licensee agrees to inform me on usage of the binaries in commercial application via email

    As the comment will be kept published, these conditions are subject to additional confirmation and can be changed at any time, please email me to licensing at alax info for further questions.

    Also commercial licenses for binaries packaged to be used “out of the box” with included redistributables, support, OEM/branded versions might be considered individually.

  8. I have followed all the steps to render my camera but it is not working.

    I have registered Acquisition.dll and CodingI.dll (this one after installing IPP 6.0.0.062). I have created the graph, and even changed JPEG Frame Decoder Filter resolution, but when I press Play I see nothing in the render window and the graph stops.

    I am trying to connect to an AXIS 213 PTZ camera and the connection string I am using is “http://root:@172.27.4.171:80/axis-cgi/mjpg/video.cgi?resolution=768×576” (I have tried this connection string in VLC and it works fine).

    Do you have any idea what could be the problem?

    Thanks

  9. In the connection string after the username “root” I also have the password. The post doesn’t show it beacause I wrote it between “” characters.

  11. Hi alax and thanks for answering,

    As you said I got the new DLLs (I previously unregistered the old ones). I am sorry but I am getting the “the graph could not change state” error with both, my camera and the Texel Airport Camera (I do change the JPEG Frame Decoder Filter resolution though). Is it working for you?

    I don’t know what is wrong. Any idea?

  12. I am sorry but I am getting the “the graph could not change state” error with both, my camera and the Texel Airport Camera (I do change the JPEG Frame Decoder Filter resolution though). Is it working for you?

    I don’t know what is wrong. Any idea?

    This is typically because effective resolution is different from what is set up on a decoding filter. Check screenshot 2 there. You need to set up 352×288.

    Screenshot

    What is causing this error is that filter attempt to dynamically change resolution, but Video Mixing Renderer is unable to do that, it cannot “change state” while running, so you need to provide correct resolution before running the graph.

    If you change resolution on already connected graph, delete JPEG Decoder and reconnect JPEG Frame Decoder’s output to Video Mixing Renderer again to have decoder added for changed resolution.

  13. I already did that but I still get the error. I don’t know what’s wrong but it doesn’t work for me.

    I would appreciate any other suggestion.

  14. I already did that but I still get the error. I don’t know what’s wrong but it doesn’t work for me.
    I would appreciate any other suggestion.

    OK, I tried to build the graph on a clean XP system – it worked for me. Step by step things I did:

    • Downloaded Acquisition.dll and CodingI.dll from repository
    • Copied Intel IPP redistributable files (see below).
    • Copied graphedt.exe and proppage.dll from Windows SDK
    • Registered DLLs: Acquisition.dll, CodingI.dll, proppage.dll usign regsvr32

    Build graph as described in post https://alax.info/blog/1006

    Axis camera started streaming.

    Useful attachments:

    • Directory listing of copied files: dir.txt
    • GrapgEdit saved graph to stream from the camera: untitled.grf

    What is also important is what error code is given with the message. For example, code 0x80190191 corresponds to HTTP 401 error “Unauthorized”. 0x80072efd “A connection with the server could not be established” is typically given if the camera is offline (you can enter the same URL into browser to see if the camera responds anything for given URL).

Leave a Reply