NVIDIA shows off how gamers will converse with any NPC thanks to AI

During its generative AI keynote at Computex 2023, NVIDIA introduced Avatar Cloud Engine (ACE), a groundbreaking technology that enables gamers to have natural conversations with non-playable characters (NPCs) and receive appropriate responses. The company showcased a demo called Kairos, set in a dystopian Ramen shop, where a playable character interacted with an NPC named Jin.

Read: This is what the ROG Ally costs in South Africa

In the demo, the player engaged in a conversation with Jin, asking, “Hey Jin, how are you?” to which Jin replied, “Unfortunately, not so good.” Jin further expressed concerns about the rising crime in the area, explaining that the ramen shop had been caught in the crossfire.

Although the dialogue in the demo may appear a bit stiff, NVIDIA’s intention was to demonstrate the potential of seamlessly speaking into a headset and receiving contextually appropriate responses from NPCs, creating a more immersive and natural interaction compared to conventional gameplay scenarios. ChatGPT, an AI language model, might have produced more fluid dialogue, but the objective was to highlight the concept.

NVIDIA collaborated with Convai to develop the demo and promote ACE, which can be deployed both in the cloud and locally on NVIDIA hardware. ACE utilizes NVIDIA NeMo for constructing, customizing, and deploying large language models that can be tailored with lore and character backstories. It incorporates guardrails to ensure appropriate conversations and employs Riva, a speech recognition and speech-to-text tool. Additionally, NVIDIA’s Omniverse Audio2Face enables the instant creation of expressive facial animations for game characters to synchronize with speech tracks.

The demo was built using Unreal Engine 5 to showcase NVIDIA’s ray-tracing capabilities and other GPU features. While the visual aspects of the demo are impressive, the AI dialogue has room for improvement, a factor NVIDIA acknowledges. The company did not announce specific games that will utilize the ACE technology, but it mentioned that Stalker 2: Heart of Chernobyl and Fort Solis will leverage Omniverse Audio2Face.

NVIDIA’s Avatar Cloud Engine represents a significant advancement in gaming technology, promising more immersive and interactive gameplay experiences by enabling natural conversations with NPCs. The potential for improved AI dialogue holds promise for future game development, enhancing the overall gaming experience for players.