paramount
Basic Information
Paramount is a developer-focused tool for recording and evaluating AI chat interactions so expert reviewers can perform quality assurance, capture ground truth, and run automated regression tests on LLM-driven systems. The package instruments AI functions via a provided decorator to capture inputs and outputs, stores recordings to a configurable backend, and exposes a local UI and API for human subject matter experts to inspect and score chat transcripts. It is intended to run entirely offline in a private environment. Configuration is done through a paramount.toml file which is autogenerated on first run and maps which function parameters correspond to chat lists, roles, and content. The repo includes an example script, CLI entry to launch the UI, and a Dockerfile for containerized deployments.
Links
App Details
-
$$$$ -
$$$$