NVIDIA CEO Jensen Huang hints at AI-generated textures and objects in future games

NVIDIA looking into AI-generated textures and objects in future games

Machine learning is transforming the PC market by enabling new tools for everyday use, improving productivity, and unlocking hidden artistic talents by turning sketches into real artworks (hardly works with mine but still works). These algorithms, based on complex AI models, were trained on hundreds of graphics cards to generate a few gigabytes of data that can even fit on a local hard drive.

While it is now possible to run some text-based large language models directly on a single graphics card, and there has been significant progress in text-to-image generation, other models, such as image-to-video, can only generate short clips unless they are powered by a cluster of AI accelerators.

Recently, NVIDIA launched a technology called RTX Remix, which enables modders and gamers to remaster older titles by hooking up the render to NVIDIA tools that enhance texture resolution, object meshes, and lighting, and later add technologies like ray or path tracing into these games. This modding process is not initially done in real-time and requires users to download the RTX Remix software to do it manually and prepare special libraries. Only then it can be used as a mod.

NVIDIA CEO Jensen Huang is already looking ahead to what AI tools can bring to gamers in the future, and one such technology could be similar to what RTX Remix does but in real-time.

Question: AI has been used in games for a while now, I’m thinking DLSS and now ACE. Do you think it’s possible to apply multimodality AIs to generate frames? Answer (Jensen Huang): AI for gaming – we already use it for neural graphics, and we can generate pixels based off of few input pixels. We also generate frames between frames – not interpolation, but generation. In the future we’ll even generate textures and objects, and the objects can be of lower quality and we can make them look better. We’ll also generate characters in the games – think of a group of six people, two may be real, and the others may be long-term use AIs. The games will be made with AI, they’ll have AI inside, and you’ll even have the PC become AI using G-Assist. You can use the PC as an AI assistant to help you game. GeForce is the biggest gaming brand in the world, we only see it growing, and a lot of them have AI in some capacity. We can’t wait to let more people have it.

According to Jensen, in the future, it could be possible to generate textures and render objects at higher quality to make them look better. While we already have this capability through RTX Remix, this new technology would most probably work in real-time, similar to how DLSS works, maybe like RTX Remix but without any involvement from gamers themselves.

But perhaps the most interesting AI-powered technology that NVIDIA came up with recently is ACE. It creates digital humans as NPCs in games which can use generative AI to talk to gamers. While ACE looks very promising, no games have adopted this technology outside of some demos.

What NVIDIA just needs to remember is that AI-generated content is not a substitute for human creativity, but it may assist developers in creating better, more engaging games.

Source: More than Moore via Reddit