Skyrim Mod Uses ChatGPT and Other AI Tools to Give NPCs a Memory and Endless Things to Say

A Skyrim modder named Art From The Machine has shown off a mod they’re working on that uses ChatGPT and other AI tools to give NPCs a memory of your adventure, endless things for them to say, and a way to ask them questions with your voice.

As reported by PC Gamer, this mod uses ChatGPT in concert with xVASynth for text-to-speech that lets NPCs speak these new answers with an AI version of their voice and Whisper for speech-to-text so players can use their own voice via a mic to ask questions to them.

As you can see in the video here, the new responses from the NPCs aren’t quite natural as of yet and can be a bit slow, but it shows promise of a world where NPCs can comment on nearly everything you do and even answer questions directly from the player that aren’t just canned responses from a list of dialogue options.

For example, the video released by Art From The Machine shows a conversation with Ulfberth War-Bear from Warmaiden’s in Whiterun, and the player can ask him what the store’s hours of operation are and even how much time remains until the store closed based on the in-game clock.

He is also seen describing a sword the player picks up, showing that any NPC – with the proper knowledge, of course – could dynamically explain various items in the world to you as part of a conversation instead of having that info solely in text form in some menu.

“It seems to be a well-crafted iron sword with a soul gem embedded in the hilt. The enchantment on it allows the wielder to capture the souls of their enemies,” Ulfberth War-Bear says.

As for NPCs remembering your conversations, Art From The Machine is working on using a basic memory system in tandem with ChatGPT to help make this a possibility.

“I have a basic memory system set up where I ask ChatGPT to summarize the conversation on exit to help condense it down for future prompts,” Art From The Machine said on Reddit. “There are much more sophisticated tools out there to handle memory though such as Langchain which I am hoping to implement in the future.”

While the tech is obviously impressive, one of the biggest ongoing questions will be if AI-driven text can ever match that of a human. Sure, it’s cool to have endless content in a game, but does it matter all that much if it doesn’t feel real or layered or hand-crafted with the context of our own past experiences and identity? Only time will tell.

For more, check out all of our content from IGN’s AI Week, including if AI will spell doom for animation, how AI could change video games forever, how gamers are craeting brand-new RPGs using ChatGPT, and why ChatGPT seems to be everywhere.

Have a tip for us? Want to discuss a possible story? Please send an email to [email protected].

Adam Bankhurst is a news writer for IGN. You can follow him on Twitter @AdamBankhurst and on Twitch.