Reference Signal Source: Unity 3D Integration

This is not really practical – just some fun stuff and attempt to make use of a readily available code snippet. So… reference signal in Unity 3D game engine through Media Foundation and Direct3D 11.

It is basically Low-Level Native Plugin example project (limited to Direct3D 11 and Visual Studio 2015), which works together with DirectShowReferenceSource-x64.dll doing a few things:

1. commented out standard plugin example inner loop rendering
2. plugin does COM instantiation of VideoMediaSourceTextureReader class of DirectShowReferenceSource project and passes a texture for further updates:
   • the helper class instantiates Media Foundation media source and initializes compatible resolution
   • also instantiates Source Reader to conveniently read from media source
   • starts producing video frames
3. plugin rendering loop does IVideoMediaSourceTextureReader::Update calls passing Unity time; the helper class matches generated frames against requested time and updates texture respectively

The scene thus contains a plane with custom material updated on the backend with Media Foundation produced data.

The demo is not really about performance because so many things can be done to improve it and make it well… done right. Threading, tighter Direct3D integration, use of proper Direct2D render target etc. The demo is really about integration itself. Nevertheless, 2K texture is streamed well at full rate.

Download links

• Binaries:
  • 32-bit: DirectShowReferenceSource-Win32.dll
  • 64-bit: DirectShowReferenceSource-x64.dll
  • Sample code: SVN, Trac
  • License: This software is free to use
• Unity 3D related:
  • Unity plugin code and project: SVN, Trac