Skip to content

Add criterion benchmarks for MCP tool latency #432

@louis030195

Description

@louis030195

Context

Need to measure and track latency for MCP tools and SDK functions.

Current state

  • windows_benchmarks.rs: Window tree building, UI automation latency
  • unified_latency_benchmark.rs: Input event capture latency

Missing

MCP tool-level benchmarks

Proposed

Add crates/terminator-mcp-agent/benches/:

  • tool_latency.rs: MCP tool benchmarks (criterion)
  • transpiler_bench.rs: TS transpilation benchmarks

Metrics to measure

Tool Metrics
execute_browser_script Transpile time (TS vs JS), extension roundtrip
run_command Engine startup (bun vs node vs python)
get_window_tree Tree size vs latency
click/type_text Locator resolution + action

Why Rust/criterion

  • Statistical rigor (percentiles, regression detection)
  • HTML reports
  • CI integration
  • Single source of truth for all SDKs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions