Skip to content

Commit fb32a56

Browse files
author
Nitin Kanukolanu
committed
Update FLAT migration notebook to use all-mpnet-base-v2 with 768 dimensions
- Change from all-MiniLM-L6-v2 (384 dims) to all-mpnet-base-v2 (768 dims) - Update embedding model references throughout the notebook - Update compression configuration description for 768 dimensions - Maintain compatibility with existing schema structure
1 parent bdf2756 commit fb32a56

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

python-recipes/vector-search/07_flat_to_svs_vamana_migration.ipynb

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
"\n",
2626
"- Redis Stack 8.2.0+ with RediSearch 2.8.10+\n",
2727
"- Existing vector index with substantial data (1000+ documents recommended)\n",
28-
"- Vector embeddings (384 dimensions using sentence-transformers/all-MiniLM-L6-v2)"
28+
"- Vector embeddings (768 dimensions using sentence-transformers/all-mpnet-base-v2)"
2929
]
3030
},
3131
{
@@ -193,13 +193,13 @@
193193
],
194194
"source": [
195195
"# Configuration for demonstration \n",
196-
"dims = 384 # sentence-transformers/all-MiniLM-L6-v2 - 384 dims\n",
196+
"dims = 768 # sentence-transformers/all-mpnet-base-v2 - 768 dims\n",
197197
"\n",
198198
"num_docs = len(movies_data) # Use actual dataset size\n",
199199
"\n",
200200
"print(\n",
201201
" \"📊 Migration Assessment\",\n",
202-
" f\"Vector dimensions: {dims} (sentence-transformers/all-MiniLM-L6-v2)\",\n",
202+
" f\"Vector dimensions: {dims} (sentence-transformers/all-mpnet-base-v2)\",\n",
203203
" f\"Dataset size: {num_docs} movie documents\",\n",
204204
" \"Data includes: title, genre, rating, description\",\n",
205205
" sep=\"\\n\"\n",
@@ -311,7 +311,7 @@
311311
"from sentence_transformers import SentenceTransformer\n",
312312
"\n",
313313
"print(\"🔄 Generating embeddings for movie descriptions...\")\n",
314-
"embedding_model=\"sentence-transformers/all-MiniLM-L6-v2\"\n",
314+
"embedding_model=\"sentence-transformers/all-mpnet-base-v2\"\n",
315315
"\n",
316316
"try:\n",
317317
" # Try to use sentence-transformers for real embeddings\n",
@@ -413,7 +413,7 @@
413413
"**Lower-Dimensional Vectors (<1024 dims)**: Uses **LVQ compression** without dimensionality reduction. Memory priority uses LVQ4 (4 bits), speed uses LVQ4x8 (12 bits),\n",
414414
"balanced uses LVQ4x4 (8 bits). Achieves 60-87% memory savings.\n",
415415
"\n",
416-
"**Our Configuration (384 dims)**: Will use **LVQ compression** as we're below the 1024 dimension threshold. This provides excellent compression without dimensionality reduction.\n",
416+
"**Our Configuration (768 dims)**: Will use **LVQ compression** as we're below the 1024 dimension threshold. This provides excellent compression without dimensionality reduction.\n",
417417
"\n",
418418
"## Available Compression Types\n",
419419
"- **LVQ4/LVQ4x4/LVQ4x8**: 4/8/12 bits per dimension\n",

0 commit comments

Comments
 (0)