-
Notifications
You must be signed in to change notification settings - Fork 91
Description
Summary
Add an “Env & Model Health” dashboard that can check and display the health of the currently selected env and its models (completion, chat, tools, embeddings). The goal is to give users a clear, centralized view of whether their configuration is actually usable, instead of discovering problems only when completions/agent calls fail.
Problem
Today, users have to infer whether their env/models are correctly configured and reachable:
Local llama-server processes may not be running or may be on the wrong port.
Remote endpoints may be misconfigured (wrong URL, missing /v1, bad API key, etc).
Tools/embeddings models might not be selected or have no endpoint set at all.
When something is wrong, the extension typically fails only at the time of use (completion, chat, agent tools), and error messages can be fragmented. There is no single place to see “is my current env healthy?” across all model types.
Proposed Solution
Introduce an env and model health-check system plus a sidebar dashboard:
Health-check logic
For each model type (completion, chat, tools, embeddings), resolve the effective endpoint based on:
The selected model’s configuration, or
The corresponding endpoint_* settings.
Perform lightweight HTTP checks (e.g., HEAD/GET with a timeout) using the existing Axios request configuration and API keys from Configuration.
Classify results into statuses such as:
Healthy (reachable, expected response)
Unreachable (connection error / timeout)
Authentication error (4xx)
Misconfigured (no endpoint / obvious config issue)
Env & Model Health Dashboard
New view in the existing Llama sidebar webview that shows:
Currently selected env name and description.
Status for each model type (completion/chat/tools/embeddings) with short messages.
If relevant, a short hint on how to fix common problems (e.g., “No tools model selected”, “Endpoint empty, select an env with tools model or set endpoint_tools”).
A command like llama-vscode: Show Env & Model Health that opens/focuses this dashboard.
Integration with existing flows
Optional: add an action in the llama-vscode menu (or env/model quick picks) to “Run health check” for the current env.
Health checks should run asynchronously and avoid blocking the UI.*_
Implementation Notes (suggested)
Health logic:
Extend ModelService with a generic checkModelHealth(model, type) helper.
Extend EnvService with a checkEnvHealth(env) that aggregates per-model checks.
UI:
Add new state to llama-webview-provider to expose health results.
Implement a React/TS component under
ui/
to render the dashboard (statuses, messages, actions).
These are suggestions; maintainers may choose a different implementation that achieves the same behavior.
Benefits
Users can quickly see whether their env is actually usable before invoking completions or the agent.
Makes debugging configuration issues (ports, endpoints, API keys) significantly easier.
Reduces support questions/issues related to misconfiguration.
Acceptance Criteria
A command (e.g. llama-vscode: Show Env & Model Health) is available in the Command Palette and opens a health view in the Llama sidebar.
For a selected env with:
A reachable completion endpoint → the dashboard shows it as healthy.
An unreachable tools endpoint (e.g. wrong port) → the dashboard shows it as unhealthy with a short explanation.
No tools model / endpoint configured → the dashboard explicitly reports that tools are not configured.
Health checks run asynchronously and do not freeze the editor while executing.