feat(llm): add generic multi-provider LLM support

[danieljustus]

Replace hardcoded AnthropicClient dependency with a provider-agnostic
LLMClient protocol and factory pattern.

New providers supported:

• Anthropic (existing, refactored to implement protocol)
• OpenAI (new adapter with retry logic and JSON mode)
• Ollama (new stdlib-only adapter for local models)

Key changes:

• LLMClient Protocol with structural subtyping
• create_llm_client() factory with env-var config
• Generic exceptions (LLMClientError, LLMProviderError)
• ReplyClassifier and generate_rebuttal accept optional client param
• CLI commands gain --provider and --model options
• New OPENERASEME_LLM_PROVIDER and OPENERASEME_LLM_MODEL env vars
• Backward compatible: ANTHROPIC_API_KEY still works

New test coverage: 146 tests across 8 new test files.

[Copilot]

Pull request overview

This PR refactors OpenEraseMe’s LLM integration from a hardcoded Anthropic dependency into a provider-agnostic client protocol + factory, adding first-class adapters for OpenAI and Ollama and wiring provider/model selection through the CLI and services.

Changes:

• Introduces LLMClient protocol + create_llm_client() factory (env/CLI-configured) and unifies error/usage record shapes.
• Adds new provider adapters: OpenAIClient (retry + JSON mode) and OllamaClient (stdlib-only HTTP).
• Updates reply classification and rebuttal generation flows to accept an injected client and adds unit tests for the new providers/factory/CLI args.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
uv.lock	Adds openai (and transitive deps) and bumps project version/extras metadata.
pyproject.toml	Introduces openai and ollama extras.
.env.example	Documents new OPENERASEME_LLM_PROVIDER / OPENERASEME_LLM_MODEL and provider keys.
src/openeraseme/llm/protocol.py	Adds provider-agnostic UsageRecord + LLMClient protocol + generic exceptions.
src/openeraseme/llm/factory.py	Adds provider registry + env-driven factory to construct provider clients lazily.
src/openeraseme/llm/openai_client.py	New OpenAI adapter with retries, cost tracking, and JSON mode support.
src/openeraseme/llm/ollama_client.py	New Ollama adapter using urllib with basic availability checks and usage capture.
src/openeraseme/llm/anthropic_client.py	Refactors Anthropic client to use shared protocol types and generic exception base classes.
src/openeraseme/llm/__init__.py	Re-exports factory/protocol symbols as the openeraseme.llm public surface.
src/openeraseme/adapters/triage/classifier.py	ReplyClassifier now uses injected/factory-created generic LLM client.
src/openeraseme/adapters/triage/responder.py	generate_rebuttal now uses injected/factory-created generic LLM client + fallback path.
src/openeraseme/services/reply.py	Services now create the LLM client via factory using provider/model inputs.
src/openeraseme/services/doctor.py	Adds “LLM config” section to doctor output showing provider/key/model status.
src/openeraseme/cli/app.py	CLI switches from --api-key to --provider/--model for classify/rebuttal commands.
skills/triage-broker-replies.md	Updates triage workflow docs to use provider/model flags and new env vars.
skills/triage-agent-setup.md	Adds agent self-setup doc (install/config/cron examples) for multi-provider triage.
tests/unit/test_classifier.py	Updates classifier tests for factory/client injection behavior.
tests/unit/test_responder.py	Updates rebuttal tests for factory/client injection behavior.
tests/unit/test_reply_service.py	New tests ensuring services call create_llm_client(provider, model).
tests/unit/test_llm_protocol.py	New tests for protocol UsageRecord and generic exception hierarchy.
tests/unit/test_llm_factory.py	New tests for provider listing, env fallbacks, lazy import failures, and client creation.
tests/unit/test_openai_client.py	New OpenAI adapter tests (fake SDK, retry, JSON mode, cost computation).
tests/unit/test_ollama_client.py	New Ollama adapter tests (availability, payload, error handling, usage).
tests/unit/test_anthropic_adapter.py	New tests for Anthropic structural subtyping + exception/usage compatibility.
tests/unit/test_cli_args.py	New tests asserting CLI accepts --provider/--model and rejects --api-key.

Comments suppressed due to low confidence (2)

src/openeraseme/cli/app.py:513

• The CLI removed the --api-key option for classify-reply. There are still repo docs/examples that invoke openeraseme classify-reply ... --api-key ..., so this is a user-facing breaking change. Consider keeping --api-key as a deprecated alias (forwarded to create_llm_client(api_key=...)) or updating/removing all references across the repo in the same PR to avoid broken guidance/scripts.

@app.command(name="classify-reply")
def classify_reply(
    ctx: typer.Context,
    request_id: int = typer.Argument(
        ...,
        help="Request ID to classify the reply for",
    ),
    provider: str = typer.Option(
        None,
        "--provider",
        envvar="OPENERASEME_LLM_PROVIDER",
        help="LLM provider: anthropic, openai, ollama",
    ),
    model: str = typer.Option(
        None,
        "--model",
        envvar="OPENERASEME_LLM_MODEL",
        help="Model name (provider-specific)",
    ),
    save: bool = typer.Option(
        True,
        "--save/--no-save",
        help="Save classification result to DB",
    ),
) -> None:
    result = handle_classify_reply(
        request_id,
        provider,
        model,
        save,
        ctx.obj["output"],
    )
    _render(ctx.obj["output"], result)

src/openeraseme/cli/app.py:615

• The CLI removed the --api-key option for generate-rebuttal. There are still repo docs/examples that invoke openeraseme generate-rebuttal ... --api-key ..., so this is a user-facing breaking change. Consider keeping --api-key as a deprecated alias (forwarded to create_llm_client(api_key=...)) or updating/removing all references across the repo in the same PR to avoid broken guidance/scripts.

@app.command(name="generate-rebuttal")
def generate_rebuttal_cmd(
    ctx: typer.Context,
    request_id: int = typer.Argument(
        ...,
        help="Request ID to generate rebuttal for",
    ),
    provider: str = typer.Option(
        None,
        "--provider",
        envvar="OPENERASEME_LLM_PROVIDER",
        help="LLM provider: anthropic, openai, ollama",
    ),
    model: str = typer.Option(
        None,
        "--model",
        envvar="OPENERASEME_LLM_MODEL",
        help="Model name (provider-specific)",
    ),
    save: bool = typer.Option(
        True,
        "--save/--no-save",
        help="Save rebuttal to DB",
    ),
) -> None:
    result = handle_generate_rebuttal(
        request_id,
        provider,
        model,
        save,
        ctx.obj["output"],
    )
    _render(ctx.obj["output"], result)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.