Skip to content

Modify for ollama use instead of just OpenAI #30

@mpadge

Description

@mpadge

Great piece of software @d42me! It'd be pretty easy to code the two main LLM entry points to allow a range of interfaces instead of just OpenAI. In particular, using ollama would open the whole system up to effectively unrestricted model choice. The completions of ollama are perfectly compatible with openai, and from what I can see of your models.py, the code should need only minimal modification. The embeddings in your vectordb.py would also need minimal modification to use ollama embeddings.

The biggest advantage would be the ability to use unlimited numbers of simulations, as everything would be local and free of additional API costs. Thoughts?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions