feat: implement Qdrant memory optimization with constants in @roo-code/types

[roomote]

This PR implements Qdrant memory optimization to address issue #6262 by reducing RAM consumption when loading project indexes.

Changes Made

🏗️ Architecture

• Created comprehensive Qdrant configuration constants in @roo-code/types package
• Added memory optimization config fields to codebase-index schema
• Updated all affected services to use centralized constants

🧠 Memory Optimization Features

• On-disk storage: Configured Qdrant to store vectors and HNSW indexes on disk by default
• Memory-mapped files: Added support for segments larger than 50k vectors
• Configurable thresholds: Made memory optimization settings configurable via schema
• HNSW search parameter: Kept ef parameter at 128 (not reduced to 64) for testing purposes

📁 Files Modified

• packages/types/src/qdrant.ts - New constants file with memory optimization defaults
• packages/types/src/codebase-index.ts - Added memory optimization schema fields
• src/services/code-index/config-manager.ts - Updated to use constants from types package
• src/services/code-index/vector-store/qdrant-client.ts - Implemented memory optimization config
• src/services/code-index/service-factory.ts - Updated to pass memory optimization config
• All related test files updated to handle new configuration parameters

🧪 Testing

• All existing tests pass (168 total tests across modified files)
• Added comprehensive test coverage for new configuration parameters
• Maintained backward compatibility

Impact

This implementation should significantly reduce Qdrant memory consumption from ~1.5GB to much lower levels by storing vectors and indexes on disk instead of in RAM, while maintaining search performance through memory-mapped files and optimized HNSW configuration.

Fixes #6262

---

Important

Implements Qdrant memory optimization by storing vectors and indexes on disk, using memory-mapped files, and updating configuration and tests.

• Behavior:
  • Implements Qdrant memory optimization by storing vectors and HNSW indexes on disk in qdrant-client.ts.
  • Supports segments larger than 50k vectors with memory-mapped files.
  • Keeps ef parameter at 128 for testing.
• Configuration:
  • Adds Qdrant configuration constants in qdrant.ts.
  • Updates codebase-index.ts to include memory optimization schema fields.
  • Centralizes configuration in config-manager.ts and service-factory.ts.
• Testing:
  • Updates tests in qdrant-client.spec.ts to cover new configuration parameters.
  • Ensures all existing tests pass and maintains backward compatibility.

^{This description was created by for 33c22f7. You can customize this summary. It will automatically update as commits are pushed.}

[roomote]

I have reviewed this PR implementing Qdrant memory optimization. Overall, this is an excellent implementation that addresses issue #6262 effectively. The code quality is high, with good separation of concerns and comprehensive test coverage.

Review Summary

Critical Issues (Must Fix):

1. Missing memory optimization fields in config snapshot - In src/services/code-index/config-manager.ts line 183, the PreviousConfigSnapshot creation is missing the memory optimization fields (useOnDiskStorage, memoryMapThreshold, hnswEfSearch) that are defined in the interface. This could cause restart detection issues when memory optimization settings change.

2. Unused import in config interface - In src/services/code-index/interfaces/config.ts line 3, QdrantMemoryOptimizationConfig is imported but not used in the interface definitions, suggesting potential inconsistency in type usage.

Important Suggestions (Should Consider):

3. Test coverage for memory optimization behavior - The tests in src/services/code-index/vector-store/__tests__/qdrant-client.spec.ts verify that memory optimization config is passed to collection creation but do not test the actual memory optimization behavior or edge cases (e.g., what happens when useOnDiskStorage is false, or when memoryMapThreshold is exceeded).

4. Constants organization and documentation - The QDRANT_QUANTIZATION_CONFIG_DEFAULTS in packages/types/src/qdrant.ts are only used in search operations. Could these be better organized or documented for their specific use case to improve maintainability?

Minor Improvements (Nice to Have):

5. Enhanced documentation with examples - The new constants are well-documented, but could benefit from examples showing the memory impact of different settings to help developers understand the trade-offs.

6. Type safety improvement - Consider making the QdrantMemoryOptimizationConfig interface more strict by requiring at least one property to be defined, preventing empty configuration objects.

Positive Aspects

• ✅ Excellent implementation of requirements from issue #6262
• ✅ Comprehensive test coverage updates across all affected files
• ✅ Good separation of concerns with constants properly placed in types package
• ✅ Proper backward compatibility maintained
• ✅ HNSW ef parameter correctly kept at 128 as requested for testing
• ✅ Clean integration with existing configuration management
• ✅ Well-structured constants with clear documentation

The implementation should significantly reduce Qdrant memory consumption as intended. The architecture is sound and the code follows established patterns in the codebase.

[daniel-lxs]

@roomote-agent remove the new settings that you added, these memory optimizations will always be on

[roomote]

Acknowledged! I'll remove the new settings that were added and make the memory optimizations always on. Working on this now.

[roomote]

✅ Done! I've removed the configurable memory optimization settings as requested. The memory optimizations are now always enabled.

Changes Made:

• 🗑️ Removed memory optimization settings from codebase-index.ts schema (codebaseIndexUseOnDiskStorage, codebaseIndexMemoryMapThreshold, codebaseIndexHnswEfSearch)
• 🗑️ Removed memory optimization properties and logic from config-manager.ts
• 🔧 Updated qdrant-client.ts to always use memory optimization defaults from constants
• 🔧 Updated service-factory.ts to remove memory optimization config passing
• 🧪 Updated all related test files to remove memory optimization configuration expectations

Result:

Memory optimizations (on-disk storage, memory-mapped files, optimized HNSW settings) are now always enabled using the constants from @roo-code/types. Users can no longer configure these settings - they're baked in for optimal performance.

✅ All tests pass (374/374 in code-index service)
✅ All linting and type checking pass
✅ Changes pushed and CI is running

The Qdrant memory optimizations will now always be active, reducing RAM consumption from ~1.5GB to much lower levels while maintaining search performance.