Add Sqlite with sqlite-vec as an alternative for Codebase Indexing #7280
adaptive-shield-matrix
started this conversation in
Feature Requests
Replies: 1 comment
-
What you need is already there, but you have to wait |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Problem
The official Wiki only describes/recommends qdrant (cloud + self hosted) + gemini for vector databases.
https://docs.roocode.com/features/codebase-indexing
I think SQLite should be an option as well, because it makes Roo code less reliant on external services and has much lower memory consumption than running the services locally in a docker container.
Performance comparsion
Here is a video comparing vector databases
https://www.youtube.com/watch?v=X0PwwfcGSHU
Benchmark code
https://github.com/andrisgauracs/Vector-DB-Benchmark-for-Music-Semantic-Search
I added SQLite (with some PRAGMA performance options) and
I'm getting even better throughput than all the dedicated vector databases.
andrisgauracs/Vector-DB-Benchmark-for-Music-Semantic-Search#1
My bechmark code with sqlite included
https://github.com/adaptive-shield-matrix/Vector-DB-Benchmark-for-Music-Semantic-Search
Embedding model
The all-MiniLM-L6-v2 seems like a good choice for code as well,
since code is (nearly) always exclusively in english and requires about 400 mb of memory to run.
https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
Beta Was this translation helpful? Give feedback.
All reactions