// Oprogramowanie Roman Ryltsov
Back to some experiments… The current video capture API in Windows is Media Foundation Capture Engine API (AKA IMFCaptureEngine and mfcaptureengine.h). Media Foundation is layered: you can work at lower level with video capture Media Sources, but if you don’t want to go into details you have the Capture Engine. The application continues a good…
In the years 2009 to 2011, engineers from the Microsoft Media Foundation Team shared a series of blog posts containing sample code related to the hashtag#MediaFoundation API — a successor to the venerable hashtag#DirectShow.
At that time, there was a scarcity of sample source code specifically addressing this topic. Unfortunately, the passage of time and various transformations of blog sites and the Microsoft website took their toll. The original blog posts suffered, and although they were eventually recovered and reinstated as part of the team blog archive https://lnkd.in/drKBW5tW, the source code associated with those posts vanished entirely. The links now led to the dreaded HTTP 404 “Not Found” error.
However, our quest for historical preservation and the benefit of those who remain curious led us to a solution. We unearthed the missing source code and deposited it into a GitHub repository https://lnkd.in/dXRi9PZF. There, it resides — a testament to the past and a resource for those who still harbor interest in the intricacies of the Windows Media Foundation API.
Feel free to explore the repository and delve into the code. After all, sometimes even lost fragments of the digital realm can find their way back home. ????????
In the past, GeekWire featured an article https://lnkd.in/d8FMf3mH on Rainway — a prominent Seattle startup with an ambitious mission: streaming games to any device.
Our role in this endeavor was to contribute essential components to Rainway’s game streaming technology. Among these, a pivotal piece involved transforming the audiovisual content generated by standard games into a format compatible with hashtag#HTML5. Our primary objective was to extend the gaming experience to remote web browsers.
To achieve this, we repurposed our existing technology and developed a subsystem, which efficiently converted monitor video into an H.264/AVC data stream, meticulously packaged for compatibility with HTML5 Media Source Extensions (hashtag#MSE). Through hashtag#WebRTC transmission, this stream seamlessly reached remote systems and integrated into web browsers.
Throughout our journey, we engaged in thoughtful experiments. Should audio be part of a joint stream with video, or should it be delivered separately? We delved into format intricacies and explored novel ideas. Notably, while some debated the idea of video remaining entirely within the GPU realm, including video encoding, we had already implemented this with production-quality results back in 2017.
The outcome was groundbreaking software that facilitated desktop Windows gaming streaming to HTML5 browsers, mobile devices, and even hashtag#Xbox consoles. ????????
For over a decade, we have been supporting a customer’s product in the field of medical video. Our journey began with their attempts of previous years to record from webcams — an endeavor that was initially partially successful. We put our expertise to bring use of this technology to stable level, and since then we have successfully captured countless hours of footage, using both standard cameras and professional-grade hardware.
One particularly intriguing aspect of this evolution stands out.
In the early days, our recordings were typically brief, lasting only a few minutes. Moreover, the original implementation imposed a strict 10-minute limit within the application. Why? Well, at that time, there was no robust video encoder available that could handle the application’s requirements — hardware constraints, desired quality, and real-time performance. Consequently, the signal was often recorded in an uncompressed format. The time limit served a purpose: managing disk usage and preventing accidental overflows.
As the years passed, we transitioned to using proper video encoders for live content. We swiftly expanded our supported resolution to Full HD at 60 frames per second. Hardware-assisted encoding became the norm, and the duration of encoded sessions gradually increased. What started as 10-minute recordings soon extended to 15 minutes, half an hour, and eventually full procedures lasting up to 4 hours. Then came the pivotal question: “Can we record continuously for 24 hours?” The answer was a resounding yes.
Despite its long history, the ongoing support, stable performance, and improvements over time have been well worth the very moderate effort, and yes, it’s still a hashtag#DirectShow app. With a little hashtag#MediaFoundation insertion.
Simultaneously, the need for data safety during recording intensified. Fortunately, we devised a solution, which we’ll delve into further in an upcoming piece.
A customer approached us to create a virtual camera subsystem for Windows tailored to their specific use case. Their application required a video feed from either a built-in or externally connected camera. Simultaneously, they wanted to make this camera accessible to other applications.
As previously mentioned https://lnkd.in/dYte5SQ5, this longstanding challenge had been addressed through specific development, with the open-source sample known as “VCam” serving as the starting point. Remarkably, VCam predates even Stack Overflow itself. Yet recently there have still been inquiries related to VCam, as evidenced by discussions on Stack Overflow: https://lnkd.in/dyu_Ypta.
However, with the advent of Windows 11, Microsoft finally provided a solution to the virtual camera conundrum as part of their #MediaFoundation API, codenamed “MFCreateVirtualCamera”. We successfully implemented this solution for our customer. In addition to the proof of concept https://lnkd.in/djy7Uq3w that we shared in 2022, this marked our first production use of this new technology.
In essence, we re-established a #realtime video feed with nearly zero latency, effectively treating it as an independent source of video signal.
An under 100 lines of C++ code sample that plays an MP3 file with Microsoft Media Foundation. https://github.com/roman380/MediaFoundationMinimalAudioPlayback Comes from this StackOverflow question, however unlike Microsoft’s sample it does not spread simple code among multiple source code files, gives a quick way to build & run, shows modern C++ practices (including Microsoft Windows Implementation Library…
NVIDIA consumer grade hardware provides support nowadays for – mostly – three concurrent sessions. The support matrix is provided here: Video Encode and Decode GPU Support Matrix | NVIDIA Developer. This small tool starts video encoding with given signal parameters (resolution, rate) and provides an easy way to measure load of GPU video encoding engine.…