Nvidia DLSS 2.3: Improved FPS and rendering using AI

You can also be interested in these:

Nvidia DLSS 2.3 is one of the greatest creations of the company to improve gaming performance with minimal loss of image quality, thanks to Artificial Intelligence rendering. Behind a simple frame rate metric hides a complex neural network that learns on each iteration, repeatedly until it creates the optimal scheme to play video games in high resolution ray tracing, even with modest graphic cards.

This new Nvidia DLSS 2.3 release brings improvements to the neural network that creates the models with a new SDK that will be compatible with more than 130 games. Likewise, the Image Scaling tool is updated with a new scaling and sharpening algorithm alongside with their SDK will be released to work with any GPU and freely available for developers.

What is the new Nvidia DLSS 2.3 and how it differs from conventional spatial rescalers?

The DLSS 2.3 was created in response to AMD’s FidelityFX Super Resolution. The latter, being a variable spatial scaling tool doesn’t work the way DLSS does, as it is software-based and therefore has quite a few limitations in terms of image quality limiting its range of action on edge smoothing anti-aliasing. However, the solution from AMD is still a great novelty, since its SDK is open source and worked through any GPU as well, showing great performance at high resolutions.

DLSS or Deep Learning Super Sampling is a proprietary of Nvidia and its complexity in terms of operation necessarily requires the company’s super computers. This technology is based on Artificial Intelligence, on which a perfect frame model is created from an image rescaled to native resolution from a lower resolution to improve performance (pretty much like Intel’s Super Sampling works). The previously trained and learned patterns from the game data is processed by the neural network constituting the foundation of the process. From each game supported, the frame data and movement vectors is learnt, and ultimately, an image is generated as close as possible to the one that would be shown in the native resolution.

Nvidia DLSS is always learning — Ultra/High quality preset, 1080p DLSS quality mode, RTX 3060, Ryzen 5950X, 32GB RAM.

Typical bilinear filters, antialiasing, or the new generation FSR or Nvidia Image Scaling do not use any type of model or algorithm comparing previous frames, which results in images that are lower in detail than the original, losing texture and particles fidelity, especially in modes where performance is prioritized. This is not very noticeable on in still images, however it goes to a greater extent with moving objects, because it is based on the pure capacity of the shaders and the calculations of the current frame.

Nvidia DLSS 2.3 will be available on November 16, 2021 with Nvidia Game Ready drivers in version 496.70 or higher. We can use it directly in games that have already implemented DLSS 2.0 in the past.

What’s New in DLSS 2.3 and how it impacts the visual experience?

Being a system based on AI and learning, DLSS will have more room for improvement than spatial rescalers. In this DLSS 2.3 version specifically, the processing models are improved once again to deliver more performance as well as better image quality in the game. On this context, the “Performance”, “Balanced” and “Quality” modes implemented in version 2.0 will be maintained, and in some games like Cyberpunk there will be also an “Automatic” and “Ultra” performance modes.

DLSS will make a lot more sense the higher the resolution of the game is. For instance, to create a 4K image using the AI model, a rendering resolution of 1080p or 2K is taken. The algorithm on this new version is now capable of improving what was already good in the previous version. In fact, both in DOOM and Cyberpunk, the captures with DLSS 2.3 in Quality and Balance mode improve the sharpness of the image in native resolution, the difference is astonishing compared to DLSS 2.0.

We can see also interesting differences in performance between a game running in the Vulkan API such as DOOM Eternal and another in DirectX 12 such as Cyberpunk 2077. In the first case, we have technically inferior graphics in detail and weight. As for the DLSS, it is much more easy to match and exceed native resolution. The differences between the three modes of operation are minimal in image quality, even if we decrease the sharpness of DLSS, we still have almost the same results. Only in the effect of glare or brightness of lights we can perceive a detriment in performance with DLSS.

How does Nvidia DLSS 2.3 work? — Photo: nvidia.com

Compared to Cyberpunk 2077, turning DLSS 2.3 on in 4K, using Quality and Balance mode is negligible on both near and far objects. However, it does affect to a greater extend on the performance, showing sharp distant objects at all times with no loss of texture information.

In 2K resolution, DLSS does an outstanding job in improving the sharpness of near and far textures, albeit losing a bit of overall lightness. If we go down to 1080p, the sharpness of the lines becomes a bit exaggerated, losing a bit of resolution and generating dented or jagged borders.

We find the opposite happening in Cyberpunk, where the sharpness of textures in the native resolution is better than using DLSS. At 1080p clearly the native image is better even in Quality mode. Clearly it is not focused for Full HD unless our graphics card is not very powerful, although the results in DLSS 2.0 were quite worse to the point of losing a lot of quality, gaining little or no performance at all.

Nvidia Image Scaling with Open-Source SDK

Secondly, we must mention another key feature shipped along with the DLSS 2.3. They will include the Nvidia Image Scaling tool, known until now as Image Sharpening or Image Scaling. This is a spatial image rescaling tool more similar to what FSR does, as it uses a software-based algorithm instead of AI like DLSS.

Image Scaling went from being a simple image filter to a much more elaborate and complex function that allows improving not only image sharpness, but also performance in games. It works externally to the game, as we find it in the Nvidia control panel, in the 3D configuration section, or in a more intuitive and graphical way in Nvidia GeForce Experience.

This makes it possible to use in any game that uses an API higher than DirectX 9 on both Nvidia and AMD GPUs, as well as FSR. Likewise, Nvidia has released the SDK of its algorithm so that it is freely accessible, and developers can implement it in any 3D game or application, again, also with AMD FSR. The SDK is released this November 16, 2021. DLSS 2.3 will be also available on GitHub as a free download.

The Image Scaling update now includes an image sharpening setting that doesn’t include FSR, along with a reduction in image rendering resolution of up to 5 levels. These will adjust to the native resolution of the monitor in steps of 85%, 77%, 67%, 59% and 50%, from highest to lowest image quality. Within the game we can also modify the sharpness directly by opening Shadowplay.

Scaling %	Input Resolution for 4K Output	Input Resolution for 1440p Output	Input Resolution for 1080p Output
85%	3264 x 1836	2176 x 1224	1632 x 918
77%	2954 x 1662	1970 x 1108	1477 x 831
67%	2560 x 1440	1706 x 960	1280 x 720
59%	2259 x 1271	1506 x 847	1129 x 635
50%	1920 x 1080	1280 x 720	Not Supported by Windows

Rescaling will also be applied to the resolutions available in the control panel for those playing in windowed mode instead of full screen. An indication in the upper left corner “NIS” (Nvidia Image Scaling) will appear indicating that the function is activated. This way we can directly adjust that sharpness without leaving the game.

Nvidia ICAT tool

Last but not least, along with Nvidia DLSS 2.3 and Nvidia Image Scaling, the company will also add Nvidia ICAT. It is a tool that allows us to compare the quality between images or video in a A/B testing kind of way. It will be free and will also be available as of November 16, 2021, on the official website of the brand along with the rest of the available apps.

Why use Paint or Photoshop when mounting and comparing images if you can do it with a free tool? This is the value proposition of the Nvidia ICAT, basically an interface where we load either two images or two videos and by moving a slider, we compare the overlapping elements on a split screen. We can zoom in on them or fit them even though they are of different size and resolution, being able to do the same with the videos, also adjusting the timeline to make a comparison as accurate as possible.

A second option will be to place the images or videos side by side or one next to the other to compare them in a general way. This can come in handy especially for videos, with simultaneous playback to appreciate the differences in quality or performance between the two.

Conclusions

It had been a while since we last saw Nvidia and AMD competing with such a short range of distance, one from another. We are talking about the sense of how reactive both companies have become to respond to the countless product changes and emerging technologies impacting performance and capabilities in the industry. Nvidia DLSS 2.3 is not alone in this field since Intel is also venturing these waters. I guess competition have always ignited technology development, so it spices things up a little.

The Image Scaling update is also an option for those who use GPUs with lower performance settings, as we can compensate for the loss of definition in some 1080p games with the sharpness adjustment provided by this tool. ICAT is also appreciated to optimize our workflow when creating comparative analysis and benchmarks.