Search

More Filters

Show More
Add Listing
Go Back

Interactive LLM Powered NPCs

0
New
Design
+1
Media & Avatars
Report

Report Abuse

Basic Information

Interactive LLM Powered NPCs is an open source project that enables natural, voice-driven conversations with non-player characters in existing games without modifying game source code. It captures microphone input, transcribes speech, identifies the target NPC using facial recognition, and generates context-aware responses via a large language model that draws on public world data and character-specific vector databases. The system produces speech and synchronized facial animation using SadTalker and text to speech, then replaces or overlays the game's face pixels so characters appear to speak. The project includes templates, Jupyter notebooks, and a folder structure for adding new games and characters, and lists prerequisites and installation steps such as Python 3.10.6, FFmpeg, and Cohere API integration. It targets open world titles like Cyberpunk 2077, Assassin's Creed, and GTA 5 to add immersive dialogue capabilities.

Links

Stars
657
App URL
https://github.com/AkshitIreddy/Interactive-LLM-Powered-NPCs
Github Repository
https://github.com/AkshitIreddy/Interactive-LLM-Powered-NPCs/blob/main/README.md

Categorization

App Details

Features
The repository integrates multiple components to create immersive NPC interactions. It captures audio and converts it to text, uses DeepFace for facial recognition and emotion detection, and generates character-aware responses with Cohere language models and LangChain. Character and world knowledge are stored in local Chromadb vector stores to provide memory and context. Pre-conversation JSON templates guide voice and style. SadTalker generates facial animation synchronized with generated speech, and Edge-TTS or custom voice modules produce audio. The project includes utilities and Jupyter notebooks to create public and character vector databases, build face recognition representations, and configure voice creation via a voice.py function signature. It provides installation instructions, model download scripts, and optional GPT-4 support notes. The design supports non-visual interactions, multi-monitor setups, and GPU acceleration for animation.
Use Cases
This project enhances player immersion by retrofitting existing games with conversational, context-aware NPCs that remember prior exchanges and reflect character personalities. Players can speak to background or named side characters using a microphone and webcam and receive synchronized facial animation and speech without waiting for official game updates. Modders and creators can extend compatibility by adding game folders, preparing world and character knowledge files, generating vector databases via provided notebooks, and customizing voices and pre-conversation lines. The system supports emotion-aware responses by reading player expressions, works when characters are off-screen, and reduces modding complexity by avoiding game code changes. Documentation and examples guide setup, and the repository encourages contributions of new games and improvements.

Categories

Content Creation (5042)
Monetization & Business (2246)
Development Tools (1407)
Marketing & Social Media (1403)
Education & Career (1380)
Multi-Agent Platforms & Frameworks (1009)
Health & Life (957)
AI Resources (920)
Agents & Automation (778)
Data & Analytics (764)
Productivity Tools (679)
Writing & Editing (666)

Similar Listings

listing image

sd webui agent scheduler

0 (0 review)
listing image

WavCraft

0 (0 review)
listing image

wcgw

0 (0 review)
listing image

Memory Cache

0 (0 review)
listing image

Sage

0 (0 review)

Featured Listings

Tags

Please fill the required fields*