Make it easier to setup other models in the CLI

[kannib]

Maybe it's just me but I've found it unnecessary difficult to get it working with Gemini in the CLI. Not sure where is pulling the settings from (the settings folder on my directory is empty) but it refuses to use Gemini and says I need an OpenAI key.
I tried to find samples and further instructions on how to properly use the CLI but most of the examples are for python scripts.
Why isn't there a default .JSON I can just edit?

[dosubot]

Hi @kannib! I'm Dosu and I’m helping the paper-qa team.

The CLI defaults to OpenAI's GPT-4O and looks for config files in ~/.config/paperqa/settings/. If your settings folder is empty, it falls back to the package's built-in defaults, which require an OpenAI key—hence the error you see when trying to use Gemini without explicit config. There isn't a default .json file for Gemini; you need to create or copy one manually if you want to override the defaults. The expected format is shown in files like paperqa/configs/debug.json and in tutorials.

To use Gemini, you must set all relevant fields: llm, llm_config, summary_llm, summary_llm_config, and agent.agent_llm/agent.agent_llm_config. If any are missing, the CLI may still try to use OpenAI as a fallback, triggering the key error. Note: Gemini currently can't be used as agent_llm due to a limitation with tool calls—use another provider (like Claude or DeepSeek) for agent_llm and Gemini for the others source.

Here's a sample config you can adapt and save as ~/.config/paperqa/settings/gemini.json:

{
  "llm": "gemini/gemini-2.0-flash-exp",
  "llm_config": {
    "model_list": [
      {
        "model_name": "gemini/gemini-2.0-flash-exp",
        "litellm_params": {
          "model": "gemini/gemini-2.0-flash-exp",
          "api_key": "<YOUR_GEMINI_API_KEY>"
        }
      }
    ],
    "rate_limit": {
      "gemini/gemini-2.0-flash-exp": "4000000 per 1 minute"
    }
  },
  "summary_llm": "gemini/gemini-2.0-flash-exp",
  "summary_llm_config": {
    "model_list": [
      {
        "model_name": "gemini/gemini-2.0-flash-exp",
        "litellm_params": {
          "model": "gemini/gemini-2.0-flash-exp",
          "api_key": "<YOUR_GEMINI_API_KEY>"
        }
      }
    ],
    "rate_limit": {
      "gemini/gemini-2.0-flash-exp": "4000000 per 1 minute"
    }
  },
  "agent": {
    "agent_llm": "claude-3-5-sonnet-20240620",
    "agent_llm_config": {
      "model_list": [
        {
          "model_name": "claude-3-5-sonnet-20240620",
          "litellm_params": {
            "model": "anthropic/claude-3-5-sonnet-20240620",
            "api_key": "<YOUR_CLAUDE_API_KEY>"
          }
        }
      ],
      "rate_limit": {
        "claude-3-5-sonnet-20240620": "40000 per 1 minute"
      }
    }
  }
}

After saving, run the CLI with --settings gemini to use this config. Make sure to set your API keys in the config or as environment variables. More details and examples are in these issues and the settings tutorial.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}