As part of its series of reveals and announcements at GDC 2022, Team Green Today revealed Nvidia is streamlining. In short, it’s a framework that Nvidia designed to make it easier to integrate super-resolution technologies into games. Fortunately, Streamline is both open-source and can support super-resolution technologies from various hardware vendors and game engines. Intel is already on board, but we have no word from AMD on its planes.
The speed at which super-resolution technologies are advancing and proliferating can be mind-boggling to users. If you feel that way, share some empathy with game developers who may have to dance around coding and recoding their games to run smoothly and be optimized for the likes of DLSS (Deep Learning Super Sampling), NIS (Nvidia Image Scaling), FSR (AMD FidelityFX Super Resolution), XeSS (Intel’s Xe Super Sampling), potential updates to these techniques, and more options besides.
It’s generally accepted that the best place to hook advanced super-resolution technology is in the graphics pipeline, before final screen rendering, so that game imagery is upscaled using the various technical but graphically light elements like the common-or-garden HUD are not, so it stays pixel-perfectly sharp. Some solutions work at the end of the pipeline, so they may work without any meaningful game developer support (or need no developer support at all), but they are often seen as suboptimal compromises.
Nvidia Streamline was designed for developers to have an easy plug and play framework between the game and the rendering API. As the creator of Streamline, Nvidia already has the DLSS and DLAA plugins ready, and it is preparing an NIS plugin, as well as a real-time denoising filter.
“Instead of manually integrating each SDK, developers simply identify what resources (motion vectors, depth, etc.) are needed for the target super-resolution plugins, then define where they want the plugins to be. ‘execute in their graphics pipeline,” Nvidia explains. “Making the integration of multiple technologies easier for developers, Streamline benefits gamers with more technologies in more games.”
Adding some weight to the value of its work, at this early stage, Nvidia already has Intel on board. In the diagram above, you can see that Intel plugins are being prepared for Streamline, which will enable Intel XeSS with just minimal effort from developers who have the framework ready, especially compared to pre-Streamline days. Finally, Nvidia left the door open for “hardware vendor #3” (aka He Who Must Not Be Named, aka AMD) to play ball and produce a plugin, to speed up the integration of their super- resolution alongside the green and blue team picks.
The open source framework above seems like an attractive solution for the game development industry. If adopted in droves, this could prove to be a significant win for Nvidia, which will almost certainly make DLSS hooked on all games that use this framework. Nvidia has a fairly established lead with the adoption of image scaling technology at present, and Streamline isn’t going to hurt it. In some ways, Streamline could ensure that DLSS is never usurped by rival technologies, especially more open technologies that may have blossomed separately later in 2022.
On the other hand, Streamline could just help everyone and ensure that users choose the litter when it comes to super resolution techniques. AMD’s FSR is already open-source, so even if AMD doesn’t directly support Streamline, someone in the OSS community could create the required plugin. Intel’s XeSS is a bit of a latecomer, but any game using Streamline for DLSS could easily add XeSS support as well. Why not, since it is simply a question of putting in a few hooks?
“Intel strongly believes in the power of open interfaces,” said Andre Bremer, vice president of AXG and director of game engineering at Intel. “We are excited to support Streamline, an open, cross-IHV framework for new graphical effects. This will simplify game developers’ integration efforts and accelerate the adoption of new technologies.”
Nvidia says Streamline is available today on GitHub supporting both DLSS and DLAA (Deep Learning Anti-Aliasing). NIS support is “coming soon”. The framework is extensible beyond super-samplers, as evidenced by Nvidia with its real-time Denoiser. Please note that Streamline supports DirectX 11 and DirectX 12 titles, but Vulkan compatibility is still in beta testing.