fix: resolve vector dimension mismatch error when switching embedding models (#5616)

[roomote]

Description

Fixes #5616

This PR resolves the vector dimension error that occurs when switching between embedding models with different vector dimensions (e.g., from 2048-dimension models to Google Gemini's 768-dimension model).

Problem

When users switched from a high-dimensional embedding model (2048 dimensions) to a lower-dimensional model like Google Gemini (768 dimensions), the Qdrant vector store would fail to recreate the collection, resulting in:

• "Vector dimension error: expected dim: 2048, got 768"
• Red error status in the indexing interface
• Inability to complete re-indexing

Solution

Enhanced the QdrantVectorStore with robust dimension mismatch handling:

Key Changes Made

1. Enhanced Collection Recreation Logic

   • Added atomic collection deletion with verification
   • Implemented step-by-step error handling for better debugging
   • Added proper type checking for vector configuration extraction

2. Improved Error Reporting

   • Detailed error context based on failure stage (deletion, verification, or creation)
   • User-friendly error messages with actionable information
   • Preserved original error context using the cause property

3. Robust Verification Process

   • Verification that collection is actually deleted before attempting recreation
   • Brief delay to ensure deletion is processed by Qdrant
   • Comprehensive logging for better debugging

Files Changed

• src/services/code-index/vector-store/qdrant-client.ts - Enhanced collection recreation logic
• src/services/code-index/vector-store/__tests__/qdrant-client.spec.ts - Added comprehensive test coverage

Testing

• ✅ All existing tests pass (59/59 QdrantVectorStore tests)
• ✅ All code-index tests pass (329/329 total tests)
• ✅ Added 5 new comprehensive test cases for dimension mismatch scenarios:
  • Successful collection recreation with new dimensions
  • Deletion failure handling
  • Creation failure handling
  • Verification failure handling
  • Specific 2048→768 dimension scenario from the issue
• ✅ No linting errors
• ✅ TypeScript compilation successful

Verification of Acceptance Criteria

• ✅ Issue reproduction: The exact scenario from issue Indexing fails after switching vector model dimensions due to dimension mismatch error #5616 (2048→768 dimensions) is now handled gracefully
• ✅ Error handling: Users receive clear, actionable error messages instead of cryptic database errors
• ✅ System stability: Atomic operations prevent the system from getting stuck in inconsistent states
• ✅ Backward compatibility: All existing functionality continues to work as expected

Impact

• Better user experience: Clear error messages instead of cryptic database errors
• Improved reliability: Atomic operations prevent inconsistent states
• Enhanced maintainability: Well-tested code with comprehensive error scenarios
• Robust error handling: Graceful handling of edge cases like partial deletion failures

Checklist

• Code follows project style guidelines
• Self-review completed
• Comments added for complex logic
• No breaking changes
• All tests passing
• TypeScript compilation successful
• Comprehensive test coverage added

---

Important

Fixes vector dimension mismatch in QdrantVectorStore by enhancing collection management and error handling.

• Behavior:
  • Fixes vector dimension mismatch error in QdrantVectorStore when switching embedding models.
  • Implements atomic collection deletion and recreation with verification in qdrant-client.ts.
  • Adds detailed error reporting and user-friendly messages for dimension mismatches.
• Testing:
  • Adds 5 new test cases in qdrant-client.spec.ts for dimension mismatch scenarios, including successful recreation and error handling.
  • Ensures all existing tests pass, maintaining backward compatibility.
• Misc:
  • Updates ClineProvider.ts to include new configuration options for embedding models.

^{This description was created by for e29f633. You can customize this summary. It will automatically update as commits are pushed.}

[delve-auditor]

✅ No security or compliance issues detected. Reviewed everything up to 15ec56a.

Security Overview

• 🔎 Scanned files: 3 changed file(s)

Detected Code Changes

Change Type	Relevant files
Enhancement	► ClineProvider.ts     Add new codebase index configuration options ► qdrant-client.ts     Improve vector store collection handling     Add enhanced error handling and validation ► qdrant-client.spec.ts     Add tests for vector store dimension handling

Reply to this PR with @delve-auditor followed by a description of what change you want and we'll auto-submit a change to this PR to implement it.