llm-speedtest-mcp is an MCP (Model Context Protocol) server that benchmarks AI model inference speed. This document provides a complete security analysis of the tool.

1. User launches AI tool (Claude Desktop, Cursor, Hermes, etc.)
2. AI tool starts llm-speedtest-mcp via MCP protocol (stdio transport)
3. User invokes a benchmark tool via the AI tool
4. llm-speedtest-mcp reads API keys from environment variables
5. llm-speedtest-mcp makes direct HTTPS streaming requests to provider APIs
6. llm-speedtest-mcp measures timing (TTFT, TPS, latency)
7. llm-speedtest-mcp returns formatted results to the AI tool via MCP
8. AI tool displays results to the user

That's the entire data flow. No steps are skipped or omitted.

• Keys are read via process.env[ENV_VAR_NAME] only
• Supported env vars: OPENAI_API_KEY, ANTHROPIC_API_KEY, GROQ_API_KEY, OPENROUTER_API_KEY, DEEPSEEK_API_KEY, MINIMAX_API_KEY, ZAI_API_KEY/ZHIPU_API_KEY, KIMI_API_KEY/MOONSHOT_API_KEY
• No alternative key input mechanism exists
• No UI, no prompts, no config files for keys

• Keys are passed as HTTP headers in fetch() calls to the respective provider API
• OpenAI-compatible: Authorization: Bearer ${key}
• Anthropic: x-api-key: ${key}
• Keys are held in memory for the duration of the HTTP request only
• Keys are never written to disk, logs, console, or any output

• ❌ Never logged (not even partial/masked)
• ❌ Never stored to filesystem
• ❌ Never sent to any server other than the intended provider API
• ❌ Never included in MCP tool responses
• ❌ Never included in error messages
• ❌ Never passed to third-party services

The tool makes HTTPS connections only to the following endpoints:

No other outbound connections are made. No telemetry, no analytics, no crash reporting, no update checks.

• ❌ No file reads (except the Node.js runtime loading the script itself)
• ❌ No file writes (no database, no logs, no temp files)
• ❌ No config file parsing
• The only I/O is stdio (MCP transport) and network (API calls)

Total runtime dependencies: 1 (the MCP SDK)

No analytics, no tracking, no heavy frameworks. The entire runtime dependency tree is minimal.

• Compromised provider APIs — We cannot control what OpenAI/Anthropic/etc. do with keys sent to their APIs. This is inherent to using any AI tool.
• Compromised local environment — If the user's machine is compromised, env vars are already accessible to malware. This tool adds no additional attack surface.
• MCP transport security — The MCP stdio transport runs locally. If the local machine is compromised, the transport is already insecure regardless of this tool.

The entire source is under 500 lines in a single file (src/index.ts). To audit:

1. Check imports — Only MCP SDK and Node.js built-ins. No suspicious imports.
2. Check network calls — All fetch() calls use hardcoded URLs from the PROVIDERS array.
3. Check key usage — Search for getKey() and apiKey — only used in HTTP headers.
4. Check file I/O — Search for fs, readFile, writeFile — none exist.
5. Check logging — Search for console.log, console.error — only startup/error, never with keys.

This tool is designed with a zero-trust-except-the-user model:

• Trusts the user's environment variables (they're already on the machine)
• Trusts the provider APIs (the user chose to use them)
• Trusts nothing and no one else
• No data leaves the machine except API calls the user explicitly initiated
• The codebase is small enough to verify all of these claims in 5 minutes