Draft
Conversation
EmanueleDeRossi1
force-pushed
the
feature/embedding-service
branch
from
ea91432 to
f3254fc
Compare
EmanueleDeRossi1
had a problem deploying
to
test
GitHub Actions
Failure
EmanueleDeRossi1
force-pushed
the
feature/embedding-service
branch
from
f3254fc to
79338f8
Compare
EmanueleDeRossi1
had a problem deploying
to
test
GitHub Actions
Failure
EmanueleDeRossi1
had a problem deploying
to
test
GitHub Actions
Failure
EmanueleDeRossi1
had a problem deploying
to
test
GitHub Actions
Failure
- Rename get_model() → get_language_model() and get_embedder() → get_embedding_model() - Rename ModelConfig → LanguageModelConfig and EmbedderConfig → EmbeddingModelConfig - Keep deprecated aliases for backward compatibility
- Renamed get_model() to get_language_model() across SDK - Renamed DEFAULT_MODEL_NAME to DEFAULT_LANGUAGE_MODEL_NAME in all providers - Renamed PROVIDER_REGISTRY to LANGUAGE_MODEL_PROVIDER_REGISTRY
Rename DEFAULT_GENERATION_MODEL → DEFAULT_LANGUAGE_MODEL_PROVIDER and DEFAULT_MODEL_NAME → DEFAULT_LANGUAGE_MODEL_NAME across all services
- rename model_type to purpose in _get_user_model and related functions to avoid confusion between model_type terminology (which refers to whether model is either language/embedding model)
Add Rhesis as the default embedding model provider, following the same pattern as the language model: Backend changes: - Update constants to use consistent naming (DEFAULT_EMBEDDING_MODEL_PROVIDER) - Create default Rhesis embedding model during organization initialization - Store both language_model_id and embedding_model_id in user settings - Update generate/embedding endpoint to use new constants SDK changes: - Implement complete RhesisEmbedder class with generate() and generate_batch() - Add factory function for Rhesis embedding model - Register "rhesis" provider in EMBEDDING_MODEL_REGISTRY - Update DEFAULT_EMBEDDING_MODEL_PROVIDER from "openai" to "rhesis" This enables users to use Rhesis-hosted embeddings by default while still allowing custom embedding model configuration.
- use correct import (DEFAULT_LANGUAGE_MODEL_PROVIDER) in tests - remove unused aliases (DEFAULT_MODELS, DEFAULT_PROVIDER)
Implement dedicated Embedding table with support for multiple vector dimensions (768, 1536, 3072) using separate columns for better performance and type safety. Embeddings have a polymorphic relationship pattern, now supporting Test and Source entities. Key changes: - Add Embedding model with vector_768, vector_1536, vector_3072 columns - Add pgvector dependency for PostgreSQL vector operations - Create Alembic migration with HNSW indexes - Add polymorphic embeddings relationship to Test and Source models - Add constraint ensuring exactly one vector column is populated per record
- Update PostgreSQL images from postgres:16-alpine to pgvector/pgvector:pg16 across all environments (local, k8s, CI/CD)
…ext search Add support for 384, 768, 1024, and 1536-dimensional embeddings with pgvector HNSW indexes. Introduce embedding configuration tracking with config_hash, text_hash, status and weight fields Add EmbeddingConfig utility class and improved property accessors for dynamic column selection. Update initial data
Add abstract mixin requiring entities to implement to_searchable_text() for embeddings and full-text search. Implement for Source and Test models.
This fixes the metaclass conflict where Test was inherit both from Base (which uses sqlachlemy's DeclaritiveMeta metaclass) and ABC (which uses ABCMeta metaclass)
…stration Add reusable base class that provides: - Celery worker availability checking with timeout - Automatic fallback from async to sync execution - Batch processing with single worker check optimization - Generic typing support for different return types This pattern enables services to seamlessly work in both production (with Celery workers) and development (without workers) environments.
- Extend AsyncService for standardized async/sync orchestration - Replace custom worker checking with inherited implementation - Simplify enqueue_enrichment using execute_with_fallback - Use batch_execute for efficient multi-trace processing
- Add dimension column to Model table - Auto-detect and store embedding dimensions during connection test
…ltiple identical active embeddings
…int param The endpoint parameter was never used for embedding models, only for LLMs. Remove it from _test_embedding_connection
Implement EmbeddingGenerator class to generate and manage embeddings for any embeddable entity in the system. Key features: - Generate embeddings using configurable embedding models - Compute content and configuration hashes for deduplication - Automatic detection and marking of stale embeddings - Support for multiple entity types with dynamic model lookup - Proper error handling for invalid entity types and missing entities
EmanueleDeRossi1
force-pushed
the
feature/embedding-service
branch
from
734c69a to
24fe6d0
Compare
EmanueleDeRossi1
had a problem deploying
to
test
GitHub Actions
Failure
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces changes from the
feature/embedding-servicebranch.📝 Summary
📁 Files Changed ( 17 files)
📋 Commit Details
✅ Checklist
🧪 Testing
📸 Screenshots (if applicable)
🔗 Related Issues