feat: local vector store

[NaccOll]

Related GitHub Issue

Closes: #5682

Description

Add Sqlite as a local vector storage tool, refer to vscode-copilot-chat

Qdrant is still used by default, but users can switch according to their own preferences

Need to upgrade the Node version

Test Procedure

Unit Test:

src\services\code-index\vector-store_tests_\local-vector-store.spec.ts

Integration test

1. After loading the RooCode extension
2. Click the index button, switch the Vector storage to local, and click Save
3. Click Start, wait for the index to complete, and .roo/vector/vector_store.db will be generated
4. Use the codebase_search tool

Pre-Submission Checklist

• Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
• Scope: My changes are focused on the linked issue (one major feature/fix per PR).
• Self-Review: I have performed a thorough self-review of my code.
• Testing: New and/or updated tests have been added to cover my changes (if applicable).
• Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
• Contribution Guidelines: I have read and agree to the Contributor Guidelines.

---

Important

Adds SQLite as a local vector storage option, requiring a Node version upgrade and updating configurations and UI to support the new feature.

• Behavior:
  • Adds SQLite as a local vector storage option in local-vector-store.ts, allowing users to switch from the default Qdrant.
  • Requires Node version upgrade to >=22.14.0 in .nvmrc and package.json.
• Configuration:
  • Updates codebase-index.ts and ClineProvider.ts to include codebaseIndexVectorStoreProvider and codebaseIndexLocalVectorStoreDirectory options.
  • Modifies config-manager.ts to handle new vector store configurations.
• UI:
  • Updates CodeIndexPopover.tsx to allow selection between Qdrant and local vector store.
  • Adds localization strings for new settings in multiple language files.
• Testing:
  • Adds unit tests in local-vector-store.spec.ts to validate local vector store functionality.
  • Integration tests ensure correct behavior when switching vector storage options.

^{This description was created by for 3f937dc6ccf924c66e23ae0f50e0cbc4c4f3b036. You can customize this summary. It will automatically update as commits are pushed.}

[NaccOll]

@daniel-lxs Is this solution feasible?

I have always been troubled by the memory consumption problem of qdrant, especially when Docker is just started, it will load all the codebase into memory. And because of some features of WIndows11, the committed memory will occupy space and may not be released

By referring to copilot, I use sqlite to complete the task. Of course, upgrading NodeJs is not all good, especially since node:sqlite is still an experimental feature. But this solution can really reduce my memory pressure. When not setting the storage path, the codebase index of each project is also managed together with the project

[daniel-lxs]

@NaccOll Hey, it should be possible, but it will require a lot of testing.

I do see there is an unit test that is failing can you take a look?

[NaccOll]

@daniel-lxs

I have fixed the unit test issue at the code level.

The current error is because I am using NodeJs22, while the project is configured with 20. I did this because NodeJs22 has SQLite built in and does not require additional dependencies.

If the project does not upgrade NodeJS, then I can achieve the goal by replacing the third-party SQLite dependency.

[NaccOll]

rebased on v3.23.15 and improve performance in larger codebase

[NaccOll]

Redesign the DB to improve performance and save space. A new PR will be opened later.