openai testing agent demo

Report Abuse

Basic Information

This monorepo demonstrates how to use OpenAI's CUA model and the computer use tool together with Playwright to automate frontend testing. It contains three cooperating applications: a Next.js frontend UI to configure and watch tests, a Node-based cua-server that communicates with the CUA model and drives Playwright to interact with a browser, and a sample e-commerce app used as the target under test. The README includes setup and run instructions, environment variable guidance, example demo login credentials, and local endpoints for the frontend, sample app, and CUA server. The project is intended as a developer-facing demo and example implementation to explore automating UI interactions with a model-driven agent. The repository cautions that the computer use feature is in preview and should not be trusted in authenticated or high-stakes production environments.

Links

App Details

Features
Model-driven UI testing using OpenAI CUA and the computer use tool. Playwright integration to spin up a browser instance and execute UI actions. A Next.js frontend that provides a configuration interface and live view of test execution. A Node cua-server that orchestrates model communication and drives browser interactions, exposing a websocket endpoint for local runs. A sample-test-app that serves as an example target with demo credentials and an .env example. Project scripts and instructions for installing dependencies, running npx playwright install, and starting all apps with npm run dev. Configurable test cases and target URL via the UI or frontend/lib/constants.ts. Core testing logic centralized in the cua-server for reuse in other applications.
Use Cases
This demo helps developers explore automating frontend test scenarios by showing how a model can follow test case instructions and perform UI actions until completion. It reduces manual interaction by letting the CUA-driven agent click, fill forms, and navigate within a Playwright-controlled browser while a frontend UI shows progress. The included sample app and demo credentials make it easy to try the workflow locally. The separation of frontend, server, and sample app illustrates how to integrate the model-driven testing core into existing projects. Setup and run steps enable quick experimentation, and the project highlights limitations by warning that computer use is in preview and not safe for sensitive or production data.

Please fill the required fields*