damn-vulnerable-llm-agent

Report Abuse

Basic Information

Damn Vulnerable LLM Agent is an educational, intentionally insecure chatbot implementation built as a ReAct-style agent using Langchain. It is designed for security researchers, developers, and enthusiasts to explore and reproduce prompt injection attacks against LLM agents, with a specific focus on Thought/Action/Observation injection techniques described in related security research. The repository is an adaptation of a Capture The Flag challenge and includes a runnable demo via Streamlit, configuration for different LLM backends, environment templates, and example payloads and flags to guide experimentation. Its primary purpose is to provide a controlled, reproducible environment to understand how prompt injection can manipulate the agent loop and how certain tool interactions can be exploited or misused.

Links

App Details

Features
The project simulates a vulnerable chatbot environment with a ReAct agent implemented via Langchain. It includes examples and explicit payloads demonstrating Thought/Action/Observation injection and UNION-style SQL injection vectors used to extract hidden flags. The repo provides multiple runtime options and templates, including OpenAI, HuggingFace, and Ollama configurations managed through llm-config.yaml and .env templates. It ships a Streamlit-based UI for interactive testing, Dockerfile and build/run instructions for containerized execution, and illustrative challenge scenarios derived from a CTF. The README documents installation steps, running instructions, and example exploit payloads to reproduce vulnerabilities and learn attack mechanics.
Use Cases
This repository helps researchers and practitioners learn hands-on about prompt injection threats against agentic LLM systems by providing a reproducible, instrumented playground. Users can run the agent locally or in a container, swap LLM backends, and test how malicious inputs can override system instructions, manipulate tool calls, and exfiltrate data. The included challenge flags and payload examples make it easier to validate attack techniques and experiment with mitigations. It also serves as a teaching aid for threat modeling, red-team exercises, and developing safer agent patterns by exposing common pitfalls in tool integration and prompt management. Contributions are welcomed to expand model support and defensive strategies.

Please fill the required fields*