Multi Agent Medical Assistant

Report Abuse

Basic Information

This repository provides an AI-powered multi-agent chatbot and demonstration system designed to assist with medical diagnosis, research, and patient interactions. It combines multiple specialized agents orchestrated by LangGraph to coordinate language models, computer vision models for medical imaging, retrieval-augmented generation (RAG) with a Qdrant vector store, and a real-time web search agent. The project is packaged as a FastAPI backend with optional Docker deployment and includes data ingestion scripts to populate the vector database. It also integrates Docling for document parsing, Hugging Face reranking, confidence-based agent handoff, Eleven Labs for voice, and human-in-the-loop verification. The codebase is organized to teach multi-agent orchestration, advanced RAG techniques, and production-ready AI patterns for healthcare workflows.

Links

App Details

Features
The project implements a multi-agent architecture where specialized agents handle diagnosis, retrieval, reasoning, and imaging analysis. Advanced RAG features include Docling-based parsing of PDFs into text, tables and images, LLM-driven semantic chunking and query expansion, hybrid Qdrant search combining BM25 sparse and dense vector retrieval, and Hugging Face cross-encoder reranking of retrieved chunks. Medical imaging agents cover brain tumor detection, chest X-ray classification, and skin lesion segmentation using computer vision models. Additional features include confidence-based agent-to-agent handoff to reduce hallucinations, input-output guardrails via LangChain, source links for referenced documents and images, voice interaction using Eleven Labs, and an intuitive frontend for healthcare users.
Use Cases
This repository is helpful for healthcare practitioners and developers who want an integrated assistant that combines imaging analysis, literature retrieval, and conversational support. It speeds clinical research and preliminary diagnostic workflows by surfacing relevant papers and document excerpts via RAG and by providing model-based image assessments. Safety measures such as reranking, confidence scoring, guardrails, and human-in-the-loop validation reduce the risk of incorrect recommendations. The system is deployable locally or in Docker, supports ingestion of domain documents into a vector database, and demonstrates practical patterns for building scalable, modular multi-agent medical applications. It is also useful as a learning resource on agent orchestration and RAG design.

Please fill the required fields*