ai testing agent

Report Abuse

Basic Information

This repository provides an AI Testing Agent that uses a Large Language Model via OpenRouter to automate API testing tasks. The agent is designed to generate a human-readable test plan for an API, produce pytest-compatible Python test code based on that plan, run the generated tests, and accept natural-language feedback to refine or extend the test suite. It assumes a REST-style API by default but allows customization through prompts so users can describe their actual endpoints, parameters, and expected responses. The toolchain includes a simple example FastAPI app for local testing and a LangChain-based chatbot interface for interactive commands. Environment variables control the LLM API key and an optional target API base URL, and the project is released under an open-source license.

Links

Categorization

App Details

Features
Provides four primary actions: Plan to produce a textual test plan, Generate to create a generated_tests.py pytest file, Run to execute the tests with pytest, and Feedback to iteratively refine tests. Implements a LangChain agent (agent.py) that exposes these tools and agent_tools.py that invokes api_tester.py in subprocesses. api_tester.py constructs prompts and calls the LLM through OpenRouter to produce plans and code, and the generated_tests.py file is overwritten each time generation runs. Includes a sample main.py FastAPI application, recommended dependencies and installation steps, environment variable support for OPENROUTER_API_KEY and optional TEST_API_URL, and guidance for deterministic output via a temperature=0.0 setting.
Use Cases
Automates repetitive and error-prone parts of API test creation by turning concise prompts into structured test plans and executable pytest code, saving developer time when onboarding or validating endpoints. The interactive LangChain agent lets users iterate by giving feedback in plain language to update expectations or add new cases, which helps align generated tests with real authorization and response behaviors. Default configuration targets localhost but can point to a real API via TEST_API_URL. The tool integrates with common Python tooling and provides troubleshooting tips to resolve authorization, route mismatches, and failing tests, making it practical for teams that want rapid, LLM-assisted test generation and continuous refinement.

Please fill the required fields*