MCP-First Testing Execution Infrastructure
OpenTester is a testing execution engine designed for AI coding tools (Claude Code, Cursor, OpenCode, etc.). It provides a unified DSL format and MCP interface, enabling Agents to generate, execute, and manage test cases, achieving an automated "code-test-fix" workflow.
┌─────────────────────────────────────────┐
│ AI Agent (Claude Code / Cursor / ...) │
│ ├─ Generate DSL test cases │
│ ├─ Decide testing strategies │
│ └─ Analyze failure reasons │
├─────────────────────────────────────────┤
│ OpenTester (MCP Server) │
│ ├─ Validate DSL syntax │
│ ├─ Execute tests (CLI/GUI/TUI) │
│ ├─ Store cases/projects │
│ └─ Return structured results │
├─────────────────────────────────────────┤
│ Web UI (Auxiliary Observation Panel) │
│ ├─ View execution progress │
│ ├─ Debug cases (create/edit) │
│ └─ View history reports │
└─────────────────────────────────────────┘
Design Principles:
- Agent Intelligence: Test generation and failure analysis are handled by the Agent
- OpenTester Execution: Focuses on DSL validation and test execution
- MCP-First: All core features exposed through MCP
- Web UI Auxiliary: Visual monitoring and debugging, not required
pip install opentester
# Or with uv
uv pip install opentester# Clone repository
git clone https://github.com/kznr02/OpenTester.git
cd OpenTester
# Install backend
uv pip install ./backend
# Install frontend dependencies (optional)
cd frontend
npm installcd backend
uv run opentester start# Start both FastAPI + MCP (foreground mode with prefixed logs)
opentester start
# Run in background
opentester start --daemon
# Start only API
opentester start --api
# Start only MCP
opentester start --mcp
# Custom ports
opentester start --api-port 8080 --mcp-port 8081
# Check status
opentester status
# Stop services
opentester stop
# Environment check
opentester doctorForeground Mode Log Output:
[API] INFO: Started server process [12345]
[API] INFO: Waiting for application startup.
[MCP] INFO: Started MCP server on port 8001
[API] INFO: Application startup complete.
Add MCP configuration in .claude/settings.json:
{
"mcpServers": {
"opentester": {
"url": "http://localhost:8001/mcp"
}
}
}Note: Claude Code's HTTP MCP support may be incomplete. We recommend using STDIO mode (see below) or Claude Desktop.
Add to Claude Desktop configuration file:
Windows: %APPDATA%\Claude\settings.json
{
"mcpServers": {
"opentester": {
"command": "opentester",
"args": ["mcp"]
}
}
}macOS: ~/Library/Application Support/Claude/settings.json
{
"mcpServers": {
"opentester": {
"command": "opentester",
"args": ["mcp"]
}
}
}Linux: ~/.config/Claude/settings.json
{
"mcpServers": {
"opentester": {
"command": "opentester",
"args": ["mcp"]
}
}
}Conversation with Claude Code:
You: Help me test the login functionality
Claude: I'll create tests for the login feature...
[Generate DSL test case]
[Call MCP: validateDSL] ✓ Validation passed
[Call MCP: createProject] ✓ Project created
[Call MCP: saveCase] ✓ Case saved
[Call MCP: runCase]
Executing...
✓ All passed
OpenTester/
├── backend/ # FastAPI + Python
│ ├── opentester/
│ │ ├── api/ # REST API (for Web UI)
│ │ ├── core/ # Execution engine, storage
│ │ ├── models/ # Pydantic models (DSL, Project)
│ │ └── mcp/ # MCP Server (core)
│ └── pyproject.toml
├── frontend/ # React + TypeScript + Vite
│ └── src/ # Web UI (observation panel)
├── docs/
│ ├── SKILL_PROMPT.md # Agent Skill Prompt template
│ ├── DSL_SPEC.md # DSL syntax specification
│ └── MCP.md # MCP interface documentation
├── README.md # This document
└── LICENSE # MIT License
After Agent generates DSL, OpenTester validates syntax correctness:
version: "1.0"
meta:
name: "Login Test"
steps:
- action: exec
command: "curl http://localhost:3000/login"
- action: assert
assertion:
type: stdout_contains
expected: "token"Supports multiple execution targets:
- CLI: subprocess command execution
- GUI: PyAutoGUI + AI vision (Phase 2)
- TUI: pexpect terminal interaction (Phase 3)
- Projects stored in
~/.opentester/projects/ - PRD content persisted with projects (provided by Agent)
- Test case version management
- Template library for reusable DSL patterns
- WebSocket real-time execution progress push
- Web UI visual display
- Execution history traceability
| Tool | Description |
|---|---|
listProjects |
List all test projects |
getProject |
Get project details (including case DSL) |
createProject |
Create project (Agent provides PRD/context) |
deleteProject |
Delete project |
validateDSL |
Validate DSL syntax |
saveCase |
Save test case (DSL generated by Agent) |
deleteCase |
Delete test case |
runCase |
Execute single case |
runProject |
Execute entire project |
stopExecution |
Stop execution |
getExecutionStatus |
Get execution status |
getExecutionLog |
Get detailed logs |
listTemplates |
List saved templates |
createTemplate |
Create DSL template |
instantiateTemplate |
Create case from template |
See MCP Interface Documentation for details.
Agents using OpenTester need to include DSL generation specifications. Refer to SKILL_PROMPT.md
YAML-based test definition language. See DSL_SPEC.md for details.
Developers refer to DEVELOPMENT.md
Auxiliary features:
- View project list and details
- Edit DSL cases (Monaco editor)
- Monitor execution progress
- View history reports
Note: Web UI is not the main entry point. All core features are provided through MCP.
- FastAPI (Web UI / REST API): http://localhost:8000
- MCP Server: http://localhost:8001/mcp
- API Docs: http://localhost:8000/docs
- Web UI Dev Server: http://localhost:5173
- Project data:
~/.opentester/projects/{project_id}.json - Execution logs:
~/.opentester/executions/ - PID files:
~/.opentester/pids/ - Templates:
~/.opentester/templates/
Refer to DISTRIBUTION.md for:
- PyPI publishing
- PyInstaller packaging
- Docker images
- System package managers
Quick build executable:
cd backend
pip install pyinstaller
pyinstaller opentester.spec
# Output: dist/opentester.exe (Windows) or dist/opentester (Linux/Mac)We welcome contributions! Please see CONTRIBUTING.md for guidelines.
OpenTester - MCP-First Testing Execution Platform