{"id":412,"date":"2008-07-12T09:55:09","date_gmt":"2008-07-12T07:55:09","guid":{"rendered":"https:\/\/alax.info\/blog\/?p=412"},"modified":"2008-07-18T18:48:07","modified_gmt":"2008-07-18T16:48:07","slug":"how-to-implement-directshow-filter-using-directx-media-object-dmo-part-2-video-processing","status":"publish","type":"post","link":"https:\/\/alax.info\/blog\/412","title":{"rendered":"How To: Implement DirectShow Filter using DirectX Media Object DMO (Part 2: Video Processing)"},"content":{"rendered":"<p>Previously on the topic:<\/p>\n<ul>\n<li><a href=\"https:\/\/alax.info\/blog\/402\">Part 1: Starting the Project<\/a><\/li>\n<\/ul>\n<p>We have the DMO filter project compilable and registered with the system and it is right time to start putting code in that allows connecting the filter to other DirectShow filters, such as video capture or video file source on the input and video renderer on the output.<\/p>\n<p>IMediaObject implementation includes the following groups of functions:<\/p>\n<ul>\n<li>object capabilities:\n<pre>STDMETHOD(GetStreamCount)(DWORD* pnInputStreamCount, DWORD* pnOutputStreamCount)\r\nSTDMETHOD(GetInputStreamInfo)(DWORD nInputStreamIndex, DWORD* pnFlags)\r\nSTDMETHOD(GetOutputStreamInfo)(DWORD nOutputStreamIndex, DWORD* pnFlags)\r\nSTDMETHOD(GetInputType)(DWORD nInputStreamIndex, DWORD nTypeIndex, DMO_MEDIA_TYPE* pMediaType)\r\nSTDMETHOD(GetOutputType)(DWORD nOutputStreamIndex, DWORD nTypeIndex, DMO_MEDIA_TYPE* pMediaType)<\/pre>\n<\/li>\n<li>current media types:\n<pre>STDMETHOD(SetInputType)(DWORD nInputStreamIndex, const DMO_MEDIA_TYPE* pMediaType, DWORD nFlags)\r\nSTDMETHOD(SetOutputType)(DWORD nOutputStreamIndex, const DMO_MEDIA_TYPE* pMediaType, DWORD nFlags)\r\nSTDMETHOD(GetInputCurrentType)(DWORD nInputStreamIndex, DMO_MEDIA_TYPE* pMediaType)\r\nSTDMETHOD(GetOutputCurrentType)(DWORD nOutputStreamIndex, DMO_MEDIA_TYPE* pMediaType)\r\nSTDMETHOD(GetInputSizeInfo)(DWORD nInputStreamIndex, DWORD* pnBufferSize, DWORD* pnMaximalLookAheadBufferSize, DWORD* pnAlignment)\r\nSTDMETHOD(GetOutputSizeInfo)(DWORD nOutputStreamIndex, DWORD* pnBufferSize, DWORD* pnAlignment)<\/pre>\n<\/li>\n<li>streaming:\n<pre>STDMETHOD(GetInputMaxLatency)(DWORD nInputStreamIndex, REFERENCE_TIME* pnMaximalLatency)\r\nSTDMETHOD(SetInputMaxLatency)(DWORD nInputStreamIndex, REFERENCE_TIME nMaximalLatency)\r\nSTDMETHOD(Flush)()\r\nSTDMETHOD(Discontinuity)(DWORD nInputStreamIndex)\r\nSTDMETHOD(AllocateStreamingResources)()<\/pre>\n<\/li>\n<li>data processing:\n<pre>STDMETHOD(GetInputStatus)(DWORD nInputStreamIndex, DWORD* pnFlags)\r\nSTDMETHOD(Lock)(LONG bLock)\r\nSTDMETHOD(ProcessInput)(DWORD nInputStreamIndex, IMediaBuffer* pMediaBuffer, DWORD nFlags, REFERENCE_TIME nTime, REFERENCE_TIME nLength)\r\nSTDMETHOD(ProcessOutput)(DWORD nFlags, DWORD nOutputBufferCount, DMO_OUTPUT_DATA_BUFFER* pOutputBuffers, DWORD* pnStatus)<\/pre>\n<\/li>\n<\/ul>\n<p><!--more--><\/p>\n<p>For a very basic filter\/DMO we will need to implement:<\/p>\n<ul>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785936(VS.85).aspx\">GetStreamCount<\/a> to indicate number of streams and number of pins on the corresponding filter<\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785930(VS.85).aspx\">GetInputStreamInfo<\/a> and <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785931(VS.85).aspx\">GetInputType<\/a> to indicate acceptable input media type<\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785934(VS.85).aspx\">GetOutputStreamInfo<\/a> and <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785935(VS.85).aspx\">GetOutputType<\/a> to indicate acceptable output media type; we will be ready to suggest output media type as soon as input media type is already agreed since media types should match<\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785952(VS.85).aspx\">SetInputType<\/a>, <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785953(VS.85).aspx\">SetOutputType<\/a>, <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785926(VS.85).aspx\">GetInputCurrentType<\/a>, <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785932(VS.85).aspx\">GetOutputCurrentType<\/a> are simple get\/put accessors with a check on setting current media type to be used<\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785928(VS.85).aspx\">GetInputSizeInfo<\/a> and <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785933(VS.85).aspx\">GetOutputSizeInfo<\/a> will indicate buffer requirements<\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785929(VS.85).aspx\">GetInputStatus<\/a> and <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785949(VS.85).aspx\">ProcessInput<\/a> will deal with accepting input data<\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785950(VS.85).aspx\">ProcessOutput<\/a> is the method to actually perform brightness and contrast conversion<\/li>\n<\/ul>\n<p>The object will have one input and one output pin with similar fixed side buffers on both ends &#8211; the simplest case possible.<\/p>\n<p>Starting with <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785931(VS.85).aspx\">GetInputType<\/a> method we are dealing with <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms783393(VS.85).aspx\">DMO_MEDIA_TYPE<\/a> structure, which describes format of the data used. It is a struct member twin of DirectShow <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms779120(VS.85).aspx\">AM_MEDIA_TYPE<\/a> with DMO API management functions <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms783377(VS.85).aspx\">Mo*MediaType<\/a>. For easy and reliable manipulation with this type we will need a ATL\/WTL-like wrapper class <em>&#8220;template &lt; BOOL t_bManaged &gt; class CDmoMediaTypeT&#8221;<\/em>, which will take care of allocating and freeing the media types.<\/p>\n<p><strong>m_pInputMediaType<\/strong> and <strong>m_pOutputMediaType<\/strong> variables will hold media types accepted and agreed for object streams (and thus, filter pins). <strong>m_DataCriticalSection<\/strong> is a critical section to ensure thread safe operation.<\/p>\n<p><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785949(VS.85).aspx\">ProcessInput<\/a> method will be receiving input buffers for further processing, however this method is not expected to write any outputs. The filter\/DMO is expected to pre-process the input and copy save pre-processing outputs in member variables or private buffers. Or instead, the filter can leave a reference to the input buffer and perform the entire processing in the following <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785950(VS.85).aspx\">ProcessOutput<\/a> call.<\/p>\n<p>Other methods <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785952(VS.85).aspx\">SetInputType<\/a>, <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785953(VS.85).aspx\">SetOutputType<\/a>, <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785924(VS.85).aspx\">Flush<\/a>, <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785923(VS.85).aspx\">Discontinuity<\/a> and <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785929(VS.85).aspx\">GetInputStatus<\/a> are also dependent on current input status, so to ease further life we are making <strong>CInput<\/strong> class to hold all input data provided with <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785949(VS.85).aspx\">ProcessInput<\/a> call until required by <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785950(VS.85).aspx\">ProcessOutput<\/a> and other implementation methods. <strong>m_Input<\/strong> member variable will hold latest input buffer.<\/p>\n<p><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785950(VS.85).aspx\">ProcessOutput<\/a> is the last method to implement and it has the real data processing. The method is called when both streams\/pins are connected, media types agreed and the graph is not stopped. At the moment of the call we should have input data already available through a reference to <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785866(VS.85).aspx\">IMediaBuffer<\/a> held by <strong>m_Input <\/strong>variable and received through <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785949(VS.85).aspx\">ProcessInput<\/a> call. <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785950(VS.85).aspx\">ProcessOutput<\/a> is to full output buffer with processed data.<\/p>\n<p>Initially we decided to use <a href=\"http:\/\/fourcc.org\/yuv.php#YUY2\">YUY2<\/a> pixel format for input and output data. The choice of YUV format is stipulated by ease of <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms798812.aspx\">processing in YUV color space<\/a>. Brightness and contrast correction affects only Y component of the pixel:<\/p>\n<blockquote><p>The following equation summarizes the steps described in the previous paragraph. C is the contrast value and B is the brightness value.<br \/>\n<strong>Y&#8217; = ((Y &#8211; 16) x C) + B + 16<\/strong><\/p><\/blockquote>\n<p>YUY2 format has pixels united into macropixel structure, one macropixel for two horizontally neighboring pixels:<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/fourcc.org\/images\/yuy2.gif\" alt=\"YUY2 Macropixel Format\" \/><\/p>\n<p>That is, input and output buffers are arrays of macropixels, where we need to update Y component using brightness and contrast correction coefficients. At the very moment we define constant member variables m_nBrightness and m_nContrast and initialize them to predefeined values of -0x0010 and 0x2000 respectively. Let brightness value range be -0x00FF (unbright) through 0x00FF (bright), with a value of 0x0000 leaving original brightness intact. The contrast value range will be 0x0000 (full fade out) through 0xFFFF, with a value of 0x4000 leaving original contrast intact. So taken constants will slightly decrease brightness and apply 50%. contrast reduction. The actual correction for one Y value is performed by function <strong>Adjust<\/strong>.<\/p>\n<p>It is important to support non-standard video strides in order to be compatible to <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms787872(VS.85).aspx\">Video Mixing Renderer Filter<\/a>. <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms787872(VS.85).aspx\">Video Mixing Renderer Filter<\/a> can request an image with extended strides by providing respective media type on the output pin of the <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms783389(VS.85).aspx\">DMO Wrapper Filter<\/a>, which is passed to the DMO output stream. <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms785950(VS.85).aspx\">ProcessOutput<\/a> should be prepared to the changed media type and extended strides by checking <strong>biWidth<\/strong> field of the <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms532290(VS.85).aspx\">BITMAPINFOHEADER<\/a> structure embedded into media type.<\/p>\n<p>Once the processing is ready, we can easily check the operation using <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms786496(VS.85).aspx\">Infinite Pin Tee Filter<\/a> duplicating video stream into two streams and rendering the first through designed filter and rendering the other (original) stream as is:<\/p>\n<p><a href=\"https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/image005.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-413\" title=\"Test graph with GraphEdit\" src=\"https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/image005-300x131.png\" alt=\"\" width=\"300\" height=\"131\" srcset=\"https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/image005-300x131.png 300w, https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/image005.png 1111w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Once the graph is started, video renderers pop up their windows:<\/p>\n<p><a href=\"https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/image006.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-414\" title=\"GraphEdit Video Renderers\" src=\"https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/image006-300x158.png\" alt=\"\" width=\"300\" height=\"158\" srcset=\"https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/image006-300x158.png 300w, https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/image006.png 658w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Source code: <a href=\"https:\/\/alax.info\/blog\/wp-content\/uploads\/2008\/07\/dmobrightnesscontrastsample02.zip\">DmoBrightnessContrastSample.02.zip<\/a> (note that Release build binary is included)<\/p>\n<p>Additional notes:<\/p>\n<ul>\n<li><a href=\"http:\/\/www.fourcc.org\/yuv.php\">YUV Formats<\/a><\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms798812.aspx\">Processing in the 8-bit YUV Color Space<\/a><\/li>\n<li><a href=\"http:\/\/en.wikipedia.org\/wiki\/Brightness\">Brightness<\/a> and <a href=\"http:\/\/en.wikipedia.org\/wiki\/Contrast_(vision)\">Contrast<\/a><\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms787872(VS.85).aspx\">Using the Video Mixing Renderer<\/a><\/li>\n<li><a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms779712(VS.85).aspx\">BITMAPINFOHEADER Structure<\/a> &#8211; see Remarks on calculating stride<\/li>\n<\/ul>\n<p>Continued by:<\/p>\n<ul>\n<li><a href=\"https:\/\/alax.info\/blog\/433\">Part 3: Persistence, Automation and Property Pages<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Previously on the topic: Part 1: Starting the Project We have the DMO filter project compilable and registered with the system and it is right time to start putting code in that allows connecting the filter to other DirectShow filters, such as video capture or video file source on the input and video renderer on&hellip; <\/p>\n<p><a class=\"moretag\" href=\"https:\/\/alax.info\/blog\/412\">Read the full article<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,10],"tags":[487,78,83,47],"class_list":["post-412","post","type-post","status-publish","format-standard","hentry","category-atl","category-video","tag-atl","tag-directshow","tag-dmo","tag-howto"],"_links":{"self":[{"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/posts\/412","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/comments?post=412"}],"version-history":[{"count":0,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/posts\/412\/revisions"}],"wp:attachment":[{"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/media?parent=412"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/categories?post=412"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/tags?post=412"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}