You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: Build Vector Search with Couchbase .NET Semantic Kernel Connector and OpenAI
6
6
short_title: Vector Search with Semantic Kernel
7
7
description:
8
-
- Build a semantic search application using Couchbase BHIVE vector index with Semantic Kernel.
8
+
- Build a semantic search application using Couchbase Hyperscale vector index with Semantic Kernel.
9
9
- Learn to use the Couchbase .NET Vector Store Connector for Microsoft Semantic Kernel.
10
10
- Discover how to generate embeddings with OpenAI and store them in Couchbase.
11
11
- Perform vector similarity searches with filtering using SQL++ and ANN_DISTANCE.
@@ -26,14 +26,14 @@ length: 30 Mins
26
26
## Repository Links
27
27
28
28
-**Connector Repository**: [couchbase-semantic-kernel](https://github.com/Couchbase-Ecosystem/couchbase-semantic-kernel) - The official Couchbase .NET Vector Store Connector for Microsoft Semantic Kernel
29
-
-**This Example**: [CouchbaseVectorSearchDemo](https://github.com/Couchbase-Ecosystem/couchbase-semantic-kernel/tree/Support-Bhive-and-Composite-Index/CouchbaseVectorSearchDemo) - Complete working example demonstrating vector search with Couchbase
29
+
-**This Example**: [CouchbaseVectorSearchDemo](https://github.com/couchbase-examples/couchbase-semantic-kernel-quickstart/tree/main/CouchbaseVectorSearchDemo) - Complete working example demonstrating vector search with Couchbase
30
30
31
31
## Introduction
32
32
33
33
This demo showcases the **Semantic Kernel Couchbase connector** - a .NET library that bridges Microsoft's Semantic Kernel framework with Couchbase's vector search capabilities. The connector provides a seamless integration that allows developers to build AI-powered applications using familiar Semantic Kernel abstractions while leveraging Couchbase's vector indexing for high-performance semantic search.
34
34
35
35
The connector supports three index types:
36
-
-**BHIVE** (Hyperscale Vector Index) - for pure vector search at scale ← *Used in this demo*
36
+
-**Hyperscale Vector Index** - for pure vector search at scale ← *Used in this demo*
37
37
-**Composite Vector Index** - for vector search with heavy scalar filtering
38
38
-**FTS** (Full-Text Search) - for hybrid text + semantic search
39
39
@@ -49,7 +49,7 @@ This makes the connector ideal for RAG (Retrieval-Augmented Generation) applicat
49
49
50
50
### 2. OpenAI API Access
51
51
-**OpenAI API Key** - Get one from: https://platform.openai.com/api-keys
52
-
- Used for generating text embeddings with `text-embedding-ada-002` model
52
+
- Used for generating text embeddings with `text-embedding-3-small` model
53
53
- Ensure you have sufficient API quota for embedding generation
54
54
55
55
### 3. Development Environment
@@ -79,7 +79,7 @@ Update `appsettings.Development.json` with your credentials:
79
79
{
80
80
"OpenAI": {
81
81
"ApiKey": "your-openai-api-key-here",
82
-
"EmbeddingModel": "text-embedding-ada-002"
82
+
"EmbeddingModel": "text-embedding-3-small"
83
83
},
84
84
"Couchbase": {
85
85
"ConnectionString": "couchbase://localhost",
@@ -133,22 +133,22 @@ Ensure you have the bucket, scope, and collection ready in Couchbase:
133
133
134
134
This step demonstrates how the connector works with Semantic Kernel's vector store abstractions:
135
135
136
-
**Getting the Collection** - The demo uses `CouchbaseVectorStore.GetCollection<TKey, TRecord>()` to obtain a collection reference configured for BHIVE index:
136
+
**Getting the Collection** - The demo uses `CouchbaseVectorStore.GetCollection<TKey, TRecord>()` to obtain a collection reference configured for Hyperscale index:
IndexName="bhive_glossary_index", //BHIVE index name
143
+
IndexName="hyperscale_glossary_index", //Hyperscale index name
144
144
SimilarityMetric="cosine"
145
145
}
146
146
);
147
147
```
148
148
149
-
The `CouchbaseQueryCollectionOptions` works with both BHIVE and composite indexes - simply specify the appropriate index name. For FTS indexes, use `CouchbaseSearchCollection` with `CouchbaseSearchCollectionOptions` instead.
149
+
The `CouchbaseQueryCollectionOptions` works with both Hyperscale and Composite indexes - simply specify the appropriate index name. For FTS indexes, use `CouchbaseSearchCollection` with `CouchbaseSearchCollectionOptions` instead.
150
150
151
-
**Automatic Embedding Generation** - The connector integrates with Semantic Kernel's `IEmbeddingGenerator` interface to automatically generate embeddings from text. When you provide an embedding generator (in this case, OpenAI's `text-embedding-ada-002`), the text is automatically converted to vectors:
151
+
**Automatic Embedding Generation** - The connector integrates with Semantic Kernel's `IEmbeddingGenerator` interface to automatically generate embeddings from text. When you provide an embedding generator (in this case, OpenAI's `text-embedding-3-small`), the text is automatically converted to vectors:
152
152
153
153
```csharp
154
154
// Generate embedding from text
@@ -176,12 +176,12 @@ This creates 6 sample glossary entries with technical terms, generates embedding
176
176
}
177
177
```
178
178
179
-
### Step 3: BHIVE Index Creation
179
+
### Step 3: Hyperscale Index Creation
180
180
181
-
This demo uses a **BHIVE (Hyperscale Vector Index)** - optimized for pure vector searches without heavy scalar filtering. After documents are inserted, the demo creates the BHIVE index:
181
+
This demo uses a **Hyperscale Vector Index** - optimized for pure vector searches without heavy scalar filtering. After documents are inserted, the demo creates the Hyperscale index:
-**Similarity**: `cosine` (optimal for OpenAI embeddings)
198
198
-**Include Fields**: Non-vector fields for faster retrieval
199
199
-**Quantization**: `IVF,SQ8` (Inverted File with 8-bit scalar quantization)
200
200
201
-
> **Note**: Composite vector indexes can be created similarly by adding scalar fields to the index definition. Use composite indexes when your queries frequently filter on scalar values before vector comparison. For this demo, we use BHIVE since we're demonstrating pure semantic search capabilities.
201
+
> **Note**: Composite vector indexes can be created similarly by adding scalar fields to the index definition. Use Composite indexes when your queries frequently filter on scalar values before vector comparison. For this demo, we use Hyperscale since we're demonstrating pure semantic search capabilities.
202
202
203
203
### Step 4: Vector Search Operations
204
204
205
-
The demo performs two types of searches using the connector's `SearchAsync()` method with the BHIVE index:
205
+
The demo performs two types of searches using the connector's `SearchAsync()` method with the Hyperscale index:
206
206
207
207
#### Pure Vector Search
208
208
@@ -230,7 +230,7 @@ LIMIT 1
230
230
231
231
#### Filtered Vector Search
232
232
233
-
Even with a BHIVE index (designed for pure vector search), the connector supports filtering using LINQ expressions with `VectorSearchOptions`:
233
+
Even with a Hyperscale index (designed for pure vector search), the connector supports filtering using LINQ expressions with `VectorSearchOptions`:
234
234
```csharp
235
235
// Search with scalar filter
236
236
varresults=awaitcollection.SearchAsync(
@@ -255,15 +255,15 @@ LIMIT 1
255
255
**Query**: *"How do I provide additional context to an LLM?"*
256
256
**Expected Result**: Finds "RAG" entry within AI category
257
257
258
-
> **Note**: While BHIVE indexes support filtering as shown above, for scenarios where you frequently filter on scalar values with highly selective filters, consider using a **composite vector index** instead. The index creation syntax is similar - just add the scalar fields to the index definition. The connector's `SearchAsync()` method works identically with both index types.
258
+
> **Note**: While Hyperscale indexes support filtering as shown above, for scenarios where you frequently filter on scalar values with highly selective filters, consider using a **Composite vector index** instead. The index creation syntax is similar - just add the scalar fields to the index definition. The connector's `SearchAsync()` method works identically with both index types.
259
259
260
260
## Understanding Vector Index Configuration
261
261
262
262
Couchbase offers three types of vector indexes optimized for different use cases:
-**Creation**: Similar to BHIVE but includes scalar fields in the index definition
279
+
-**Creation**: Similar to Hyperscale but includes scalar fields in the index definition
280
280
281
281
**3. FTS (Full-Text Search) Indexes**
282
282
- Uses Couchbase Search API via `CouchbaseSearchCollection`
@@ -289,7 +289,7 @@ Couchbase offers three types of vector indexes optimized for different use cases
289
289
All three index types work with the same Semantic Kernel abstractions (`SearchAsync()`, `UpsertAsync()`, etc.). The main difference is which collection class you instantiate and the underlying query engine.
290
290
291
291
**Choosing the Right Type**:
292
-
- Start with **BHIVE** for pure vector searches and large datasets
292
+
- Start with **Hyperscale** for pure vector searches and large datasets
293
293
- Use **Composite** when scalar filters eliminate large portions of data before vector comparison
294
294
- Use **FTS** when you need hybrid search combining full-text and semantic search
295
295
@@ -330,15 +330,15 @@ dotnet run
330
330
331
331
### Expected Output
332
332
```
333
-
Couchbase BHIVE Vector Search Demo
333
+
Couchbase Hyperscale Vector Search Demo
334
334
====================================
335
-
Using OpenAI model: text-embedding-ada-002
335
+
Using OpenAI model: text-embedding-3-small
336
336
Step 1: Ingesting data into Couchbase vector store...
337
337
Data ingestion completed
338
338
339
-
Step 2: Creating BHIVE vector index manually...
340
-
Executing BHIVE index creation query...
341
-
BHIVE vector index 'bhive_glossary_index' already exists.
339
+
Step 2: Creating Hyperscale vector index manually...
340
+
Executing Hyperscale index creation query...
341
+
Hyperscale vector index 'hyperscale_glossary_index' already exists.
342
342
343
343
Step 3: Performing vector search...
344
344
Found: API
@@ -370,7 +370,7 @@ The Couchbase Semantic Kernel connector provides a seamless integration between
370
370
371
371
**Vector Store Classes:**
372
372
-**`CouchbaseVectorStore`** - Main entry point for vector store operations
373
-
-**`CouchbaseQueryCollection`** - Collection class for BHIVE and Composite indexes (SQL++)
373
+
-**`CouchbaseQueryCollection`** - Collection class for Hyperscale and Composite indexes (SQL++)
374
374
-**`CouchbaseSearchCollection`** - Collection class for FTS indexes (Search API)
375
375
376
376
**Common Methods (all index types):**
@@ -380,7 +380,7 @@ The Couchbase Semantic Kernel connector provides a seamless integration between
380
380
-**`VectorSearchOptions`** - Configures search behavior including filters and result count
381
381
382
382
**Configuration Options:**
383
-
-**`CouchbaseQueryCollectionOptions`** - For BHIVE and Composite indexes
383
+
-**`CouchbaseQueryCollectionOptions`** - For Hyperscale and Composite indexes
384
384
-**`CouchbaseSearchCollectionOptions`** - For FTS indexes
385
385
386
386
For more documentation, visit the [connector repository](https://github.com/Couchbase-Ecosystem/couchbase-semantic-kernel).
0 commit comments