Skip to content

Commit d8d0c59

Browse files
mpartipilog-despot
mpartipilo
and
g-despot
authored
Refactor vector search API with flexible syntax and enhanced type safety (#262)
Co-authored-by: Ivan Despot <66276597+g-despot@users.noreply.github.com>
1 parent 24f1cf7 commit d8d0c59

File tree

107 files changed

+15417
-4977
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

107 files changed

+15417
-4977
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -484,3 +484,5 @@ $RECYCLE.BIN/
484484

485485
# Vim temporary swap files
486486
*.swp
487+
488+
.serena

‎docs/VECTOR_API_CHANGELOG.md‎

Lines changed: 380 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,380 @@
1+
# Vector Search API Changelog
2+
3+
## Version TBD - API Consolidation
4+
5+
### Summary
6+
7+
Consolidated vector search API from 35+ overloads to ~20 core overloads with extensive implicit conversions.
8+
9+
### Changes
10+
11+
#### New Types
12+
13+
- `VectorSearchInput` - Central type for all vector search inputs with extensive implicit conversions
14+
- `VectorSearchInput.Builder` - Lambda builder for complex multi-target scenarios
15+
- `VectorSearchInput.FactoryFn` - Delegate for creating VectorSearchInput via lambda expressions with implicit conversion
16+
- `HybridVectorInput` - Discriminated union for hybrid search vector inputs (VectorSearchInput, NearTextInput, or NearVectorInput)
17+
- `HybridVectorInput.FactoryFn` - Delegate for creating HybridVectorInput via lambda builder with `.NearVector()` or `.NearText()` methods
18+
- `NearVectorInput` - Wrapper for vector input with optional thresholds (replaces `HybridNearVector`)
19+
- `NearVectorInput.FactoryFn` - Delegate for creating NearVectorInput via lambda builder with target vector configuration
20+
- `NearTextInput` - Server-side vectorization with target vectors (replaces `HybridNearText`)
21+
- `NearTextInput.FactoryFn` - Delegate for creating NearTextInput via lambda builder with target vector configuration
22+
- `TargetVectors` - Static factory methods for target vector configuration
23+
- `TargetVectorsBuilder` - Lambda builder for target vectors
24+
25+
#### Removed Types
26+
27+
- `IHybridVectorInput` - Marker interface removed in favor of `HybridVectorInput`
28+
- `INearVectorInput` - Marker interface removed
29+
- `HybridNearVector` - Renamed to `NearVectorInput`
30+
- `HybridNearText` - Renamed to `NearTextInput`
31+
32+
#### Overload Reduction
33+
34+
| Client | Method | Before | After |
35+
|--------|--------|--------|-------|
36+
| QueryClient | NearVector | 10+ | 4 |
37+
| QueryClient | NearText | 2 | 4 |
38+
| QueryClient | NearMedia | 2 | 2 |
39+
| QueryClient | Hybrid | 4+ | 9 |
40+
| GenerateClient | NearVector | 6+ | 4 |
41+
| GenerateClient | NearText | 2 | 4 |
42+
| GenerateClient | NearMedia | 2 | 2 |
43+
| GenerateClient | Hybrid | 4+ | 8 |
44+
| AggregateClient | NearVector | 6+ | 4 |
45+
| AggregateClient | NearText | 2 | 4 |
46+
| AggregateClient | NearMedia | 2 | 2 |
47+
| AggregateClient | Hybrid | 3+ | 4 |
48+
| TypedQueryClient | NearText | 2 | 4 |
49+
| TypedQueryClient | NearMedia | 2 | 2 |
50+
| TypedGenerateClient | NearText | 2 | 4 |
51+
| TypedGenerateClient | NearMedia | 2 | 2 |
52+
53+
#### API Changes
54+
55+
1. **HybridVectorInput discriminated union for vectors parameter**
56+
- Before: `Hybrid(string? query, IHybridVectorInput? vectors, ...)`
57+
- After: `Hybrid(string? query, HybridVectorInput? vectors, ...)` - accepts VectorSearchInput, NearTextInput, or NearVectorInput via implicit conversions
58+
59+
2. **VectorSearchInput replaces multiple input types**
60+
- Before: Separate overloads for `float[]`, `Vector`, `Vectors`, tuple enumerables
61+
- After: Single `VectorSearchInput` with implicit conversions from all types
62+
63+
3. **Builder replaces separate overloads**
64+
- Before: `VectorSearchInputBuilder` as standalone class
65+
- After: `VectorSearchInput.Builder` as nested class
66+
67+
4. **FactoryFn delegate enables lambda syntax**
68+
- New: `VectorSearchInput.FactoryFn` delegate with implicit conversion to `VectorSearchInput`
69+
- Allows: `vectors: b => b.TargetVectorsSum(("title", vec1), ("desc", vec2))` syntax
70+
71+
5. **Targets embedded in VectorSearchInput**
72+
- Before: Separate `targets` parameter on some methods
73+
- After: Targets configured via `VectorSearchInput.Builder` methods (Sum, ManualWeights, etc.)
74+
75+
6. **TargetVectors uses static factory methods**
76+
- Before: `new SimpleTargetVectors(["title", "description"])`
77+
- After: `TargetVectors.Sum("title", "description")` or `new[] { "title", "description" }` (implicit)
78+
79+
7. **NearTextInput includes TargetVectors**
80+
- Before: Separate `targetVector` parameter on Hybrid methods
81+
- After: `NearTextInput.TargetVectors` property is single source of truth
82+
83+
8. **Convenience overloads for text-only Hybrid search**
84+
- New: Overloads without `vectors` parameter for all Hybrid methods
85+
- These delegate to main Hybrid method with `vectors: null`
86+
- Simplifies pure text search: `Hybrid("query")` instead of `Hybrid("query", vectors: null)`
87+
88+
9. **Convenience overloads for NearText target vectors**
89+
- New: Overloads accepting `TargetVectors?` for all NearText methods (QueryClient, GenerateClient, AggregateClient, TypedQueryClient, TypedGenerateClient)
90+
- Allows passing string arrays directly: `targets: new[] { "vec1", "vec2" }`
91+
- Allows static factory methods: `targetVectors: TargetVectors.Sum("vec1", "vec2")`
92+
- Lambda builder syntax still available: `targetVectors: tv => tv.TargetVectorsSum("vec1", "vec2")`
93+
- Matches pattern already established by NearVector methods with VectorSearchInput
94+
95+
10. **NearVectorInput FactoryFn constructor for consistency**
96+
- New: Constructor accepting `VectorSearchInput.FactoryFn` for lambda builder syntax
97+
- Enables: `new NearVectorInput(v => v.TargetVectorsSum(("title", vec1), ("desc", vec2)))`
98+
- Matches pattern established by NearTextInput
99+
100+
11. **HybridVectorInput lambda builder for unified target vector syntax**
101+
- New: `HybridVectorInput.FactoryFn` delegate enables lambda builder pattern for Hybrid search
102+
- Syntax: `v => v.NearVector(certainty: 0.8).TargetVectorsManualWeights(("title", 1.2, vec1), ("desc", 0.8, vec2))`
103+
- Syntax: `v => v.NearText(["query"]).TargetVectorsManualWeights(("title", 1.2), ("desc", 0.8))`
104+
- Eliminates need to construct `NearVectorInput` or `NearTextInput` explicitly
105+
- Available across all clients: QueryClient, GenerateClient, AggregateClient, TypedQueryClient
106+
- Unifies target vector configuration directly within the Hybrid method call
107+
108+
12. **NearMedia lambda builder pattern for unified media search**
109+
- **BREAKING CHANGE**: Old NearImage and NearMedia methods removed entirely
110+
- New: Single `NearMedia` method using lambda builder pattern with `NearMediaInput.FactoryFn`
111+
- Syntax: `m => m.Image(imageBytes).TargetVectorsSum("title", "description")`
112+
- Syntax: `m => m.Video(videoBytes, certainty: 0.8f).TargetVectorsManualWeights(("visual", 1.2), ("audio", 0.8))`
113+
- Supports all media types: Image, Video, Audio, Thermal, Depth, IMU
114+
- Optional target vectors via implicit conversion: `m => m.Image(imageBytes)` works without `.Build()`
115+
- Available across all clients: QueryClient, GenerateClient, AggregateClient, TypedQueryClient, TypedGenerateClient
116+
- Certainty and distance configured in builder (not method parameters) for consistency with NearText/NearVector patterns
117+
118+
#### Migration from Old NearMedia API
119+
120+
**Simple image search:**
121+
122+
```csharp
123+
// Before (removed)
124+
await collection.Query.NearImage(imageBytes);
125+
126+
// After (required)
127+
await collection.Query.NearMedia(m => m.Image(imageBytes));
128+
```
129+
130+
**With certainty and target vectors:**
131+
132+
```csharp
133+
// Before (removed)
134+
await collection.Query.NearImage(imageBytes, certainty: 0.8, targetVectors: t => t.TargetVectorsSum("v1", "v2"));
135+
136+
// After (required)
137+
await collection.Query.NearMedia(m => m.Image(imageBytes, certainty: 0.8f).TargetVectorsSum("v1", "v2"));
138+
```
139+
140+
**Media type specification:**
141+
142+
```csharp
143+
// Before (removed)
144+
await collection.Query.NearMedia(videoBytes, NearMediaType.Video, distance: 0.3);
145+
146+
// After (required)
147+
await collection.Query.NearMedia(m => m.Video(videoBytes, distance: 0.3f));
148+
```
149+
150+
**All media types:**
151+
152+
```csharp
153+
// All supported via unified lambda builder pattern
154+
await collection.Query.NearMedia(m => m.Image(imageBytes));
155+
await collection.Query.NearMedia(m => m.Video(videoBytes));
156+
await collection.Query.NearMedia(m => m.Audio(audioBytes));
157+
await collection.Query.NearMedia(m => m.Thermal(thermalBytes));
158+
await collection.Query.NearMedia(m => m.Depth(depthBytes));
159+
await collection.Query.NearMedia(m => m.IMU(imuBytes));
160+
```
161+
162+
### Migration Examples
163+
164+
165+
**Simple hybrid search:**
166+
167+
```csharp
168+
// Before
169+
await collection.Query.Hybrid("search query");
170+
171+
// After (implicit conversion from string)
172+
await collection.Query.Hybrid("search query");
173+
```
174+
175+
**Hybrid with vectors:**
176+
177+
```csharp
178+
// Before
179+
await collection.Query.Hybrid("search query", vectors: new[] { 1f, 2f, 3f });
180+
181+
// After (tuple implicit conversion)
182+
await collection.Query.Hybrid(("search query", new[] { 1f, 2f, 3f }));
183+
```
184+
185+
**Hybrid with target vectors:**
186+
187+
```csharp
188+
// Before
189+
await collection.Query.Hybrid(
190+
"search query",
191+
vectors: new HybridNearText("banana"),
192+
targetVector: ["title", "description"] // separate parameter
193+
);
194+
195+
// After (targets inside NearTextInput)
196+
await collection.Query.Hybrid(
197+
new NearTextInput(
198+
"banana",
199+
TargetVectors: TargetVectors.Sum("title", "description")
200+
)
201+
);
202+
203+
// Or with implicit string array conversion
204+
await collection.Query.Hybrid(
205+
new NearTextInput(
206+
"banana",
207+
TargetVectors: new[] { "title", "description" }
208+
)
209+
);
210+
```
211+
212+
**Lambda builder for vectors:**
213+
214+
```csharp
215+
await collection.Query.Hybrid(
216+
"search query",
217+
v => v.TargetVectorsSum(
218+
("title", new[] { 1f, 2f }),
219+
("description", new[] { 3f, 4f })
220+
)
221+
);
222+
```
223+
224+
**Target vectors with weights:**
225+
226+
```csharp
227+
// Use static factory methods
228+
var targetVectors = TargetVectors.ManualWeights(("title", 1.2), ("desc", 0.8));
229+
await collection.Query.Hybrid(
230+
new NearTextInput("banana", TargetVectors: targets)
231+
);
232+
233+
// Or with RelativeScore
234+
var targetVectors = TargetVectors.RelativeScore(("title", 0.7), ("desc", 0.3));
235+
```
236+
237+
**NearText with target vectors:**
238+
239+
```csharp
240+
// Before: lambda builder required
241+
await collection.Query.NearText(
242+
"search query",
243+
targetVectors: tv => tv.TargetVectorsSum("title", "description")
244+
);
245+
246+
// After: multiple options available
247+
// Option 1: String array (simplest)
248+
await collection.Query.NearText(
249+
"search query",
250+
targets: new[] { "title", "description" }
251+
);
252+
253+
// Option 2: Static factory method
254+
await collection.Query.NearText(
255+
"search query",
256+
targetVectors: TargetVectors.Sum("title", "description")
257+
);
258+
259+
// Option 3: Lambda builder (still works)
260+
await collection.Query.NearText(
261+
"search query",
262+
targetVectors: tv => tv.TargetVectorsSum("title", "description")
263+
);
264+
```
265+
266+
**Simple vector search (no changes needed):**
267+
268+
```csharp
269+
// Works unchanged via implicit conversion
270+
await collection.Query.NearVector(new[] { 1f, 2f, 3f });
271+
```
272+
273+
**NearVectorInput with lambda builder:**
274+
275+
```csharp
276+
// Before: only accepts VectorSearchInput directly
277+
new NearVectorInput(new[] { 1f, 2f, 3f });
278+
279+
// After: also accepts lambda builder
280+
new NearVectorInput(
281+
v => v.TargetVectorsSum(("title", vec1), ("desc", vec2)),
282+
Certainty: 0.8f
283+
);
284+
```
285+
286+
**Named vector with target:**
287+
288+
```csharp
289+
// Before
290+
await collection.Query.NearVector(new[] { 1f, 2f, 3f }, targetVector: "title");
291+
292+
// After
293+
await collection.Query.NearVector(v => v.Single("title", new[] { 1f, 2f, 3f }));
294+
```
295+
296+
**Multi-target with weights:**
297+
298+
```csharp
299+
await collection.Query.NearVector(
300+
v => v.TargetVectorsManualWeights(
301+
("title", 1.2, new[] { 1f, 2f }),
302+
("desc", 0.8, new[] { 3f, 4f })
303+
)
304+
);
305+
```
306+
307+
**Hybrid with NearVector and target vectors (NEW unified syntax):**
308+
309+
```csharp
310+
// Before: construct NearVectorInput explicitly
311+
await collection.Query.Hybrid(
312+
"test",
313+
new NearVectorInput(
314+
VectorSearchInput.Combine(
315+
TargetVectors.ManualWeights(("title", 1.2), ("desc", 0.8)),
316+
("title", new[] { 1f, 2f }),
317+
("desc", new[] { 3f, 4f })
318+
),
319+
Certainty: 0.8f
320+
)
321+
);
322+
323+
// After: lambda builder unifies configuration
324+
await collection.Query.Hybrid(
325+
"test",
326+
v => v.NearVector(certainty: 0.8f)
327+
.ManualWeights(
328+
("title", 1.2, new[] { 1f, 2f }),
329+
("desc", 0.8, new[] { 3f, 4f })
330+
)
331+
);
332+
```
333+
334+
**Hybrid with NearText and target vectors (NEW unified syntax):**
335+
336+
```csharp
337+
// Before: construct NearTextInput explicitly
338+
await collection.Query.Hybrid(
339+
"test",
340+
new NearTextInput(
341+
["concept1", "concept2"],
342+
TargetVectors: TargetVectors.ManualWeights(("title", 1.2), ("desc", 0.8))
343+
)
344+
);
345+
346+
// After: lambda builder unifies configuration
347+
await collection.Query.Hybrid(
348+
"test",
349+
v => v.NearText(["concept1", "concept2"])
350+
.ManualWeights(("title", 1.2), ("desc", 0.8))
351+
);
352+
```
353+
354+
**Hybrid with NearText including Move parameters:**
355+
356+
```csharp
357+
await collection.Query.Hybrid(
358+
"test",
359+
v => v.NearText(
360+
["concept"],
361+
certainty: 0.7f,
362+
moveTo: new Move("positive", 0.5f),
363+
moveAway: new Move("negative", 0.3f)
364+
)
365+
.Sum("title", "description")
366+
);
367+
```
368+
369+
### Breaking Changes
370+
371+
- `IHybridVectorInput` and `INearVectorInput` interfaces removed
372+
- `HybridNearVector` renamed to `NearVectorInput`
373+
- `HybridNearText` renamed to `NearTextInput`
374+
- `Hybrid` method `vectors` parameter now uses `HybridVectorInput?` discriminated union type
375+
- `SimpleTargetVectors` and `WeightedTargetVectors` constructors are now internal
376+
- `VectorSearchInput.Builder` constructor is now internal (use `FactoryFn` lambda syntax instead)
377+
- `targetVector` parameter removed from Hybrid methods - use `NearTextInput.TargetVectors` instead
378+
- Removed tuple enumerable overloads (`IEnumerable<(string, Vector)>`)
379+
- `targetVector` string parameter removed from NearVector methods (use builder instead)
380+
- Overload resolution may change in edge cases with implicit conversions

0 commit comments

Comments
 (0)