{"id":1857,"date":"2018-06-04T07:07:41","date_gmt":"2018-06-04T05:07:41","guid":{"rendered":"https:\/\/alax.info\/blog\/?p=1857"},"modified":"2018-06-04T10:53:10","modified_gmt":"2018-06-04T08:53:10","slug":"video-processor-mft-pixel-format-conversion-bug","status":"publish","type":"post","link":"https:\/\/alax.info\/blog\/1857","title":{"rendered":"Video Processor MFT pixel format conversion bug"},"content":{"rendered":"<p><a href=\"https:\/\/alax.info\/blog\/1821\">Not the first<\/a>, not the last. A Direct3D 11 enabled Media Foundation transfer fails to transfer sample attributes while doing the conversion.<\/p>\n<p>Why attributes are important in first place? Because we can associate data with samples\/frames and have them passed through attached to specific frame as the conversion goes and as the data transits through the pipeline.<\/p>\n<p>There is no strict rule whether a transform needs to copy attributes from input to output samples. Attributes are flexible and in this case it&#8217;s so flexible that it is not clear what the transforms actually do. Microsoft attempted to bring some order with <a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/aa366522\"><code>MFPKEY_EXATTRIBUTE_SUPPORTED<\/code><\/a> property. Let us have a look at what documentation says about the <a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/aa965264#sample_attributes\">processing model<\/a>:<\/p>\n<blockquote><p>The input samples might have attributes that must be copied to the corresponding output samples.<\/p>\n<ul>\n<li>If the MFT returns <code>VARIANT_TRUE<\/code> for the <code>MFPKEY_EXATTRIBUTE_SUPPORTED<\/code> property, the MFT must copy the attributes.<\/li>\n<li>If the <code>MFPKEY_EXATTRIBUTE_SUPPORTED<\/code> property is either <code>VARIANT_FALSE<\/code> or is not set, the client must copy the attributes.<\/li>\n<\/ul>\n<\/blockquote>\n<p>Words &#8220;client must copy the attributes&#8221; should be read as this: MFT does not give a damn about the attributes and go copy them yourself the way you like.<\/p>\n<p>Needless to say that Video Processor MFT itself has not faintest idea about this <code>MFPKEY_EXATTRIBUTE_SUPPORTED<\/code> attribute in first place, and so is the behavior it defines.<\/p>\n<p>Microsoft designed <a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/hh162913\">Video Processor MFT<\/a> as a Swiss army knife for basic conversions. The MFT has zero degrees of customization and has multiple code paths inside to perform this or that conversion.<\/p>\n<p>All together it means that small bugs inside are endless and MFT behavior is not consistent across different conversions.<\/p>\n<p>So I approached the bug itself: unlike other scenarios when the MFT does pixel format conversion it fails to copy the sample attributes. I feed a sample with attributes attached and I get output with zero attributes.<\/p>\n<p>In my case the workaround is this a wrapper MFT that intercepts <a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/ms703131\"><code>IMFTransform::ProcessInput<\/code><\/a> and <a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/ms704014\"><code>IMFTransform::ProcessOutput<\/code><\/a> calls and copies the missing attributes.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Not the first, not the last. A Direct3D 11 enabled Media Foundation transfer fails to transfer sample attributes while doing the conversion. Why attributes are important in first place? Because we can associate data with samples\/frames and have them passed through attached to specific frame as the conversion goes and as the data transits through&hellip; <\/p>\n<p><a class=\"moretag\" href=\"https:\/\/alax.info\/blog\/1857\">Read the full article<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[63,424,486],"class_list":["post-1857","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-bug","tag-media-foundation","tag-video"],"_links":{"self":[{"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/posts\/1857","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/comments?post=1857"}],"version-history":[{"count":0,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/posts\/1857\/revisions"}],"wp:attachment":[{"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/media?parent=1857"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/categories?post=1857"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/alax.info\/blog\/wp-json\/wp\/v2\/tags?post=1857"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}