Skip to content

Commit e75f1b8

Browse files
rbs333rbs333
committed
show multi-level
1 parent 592b79f commit e75f1b8

File tree

1 file changed

+216
-32
lines changed

1 file changed

+216
-32
lines changed

‎docs/user_guide/05_hash_vs_json.ipynb‎

Lines changed: 216 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
},
2828
{
2929
"cell_type": "code",
30-
"execution_count": 1,
30+
"execution_count": 5,
3131
"metadata": {},
3232
"outputs": [],
3333
"source": [
@@ -311,36 +311,13 @@
311311
"metadata": {},
312312
"source": [
313313
"### Working with JSON\n",
314-
"Redis also supports native **JSON** objects. These can be multi-level (nested) objects, with full JSONPath support for updating/retrieving sub elements:\n",
315-
"\n",
316-
"```python\n",
317-
"{\n",
318-
" \"name\": \"bike\",\n",
319-
" \"metadata\": {\n",
320-
" \"model\": \"Deimos\",\n",
321-
" \"brand\": \"Ergonom\",\n",
322-
" \"type\": \"Enduro bikes\",\n",
323-
" \"price\": 4972,\n",
324-
" }\n",
325-
"}\n",
326-
"```\n",
327314
"\n",
328315
"JSON is best suited for use cases with the following characteristics:\n",
329316
"- Ease of use and data model flexibility are top concerns\n",
330317
"- Application data is already native JSON\n",
331318
"- Replacing another document storage/db solution"
332319
]
333320
},
334-
{
335-
"cell_type": "markdown",
336-
"metadata": {},
337-
"source": [
338-
"#### Full JSON Path support\n",
339-
"Because Redis enables full JSON path support, when creating an index schema, elements need to be indexed and selected by their path with the desired `name` AND `path` that points to where the data is located within the objects.\n",
340-
"\n",
341-
"> By default, RedisVL will assume the path as `$.{name}` if not provided in JSON fields schema."
342-
]
343-
},
344321
{
345322
"cell_type": "code",
346323
"execution_count": 11,
@@ -505,11 +482,223 @@
505482
"source": [
506483
"jindex.delete()"
507484
]
485+
},
486+
{
487+
"cell_type": "markdown",
488+
"metadata": {},
489+
"source": [
490+
"# Working with nested data in JSON\n",
491+
"\n",
492+
"Redis also supports native **JSON** objects. These can be multi-level (nested) objects, with full JSONPath support for updating/retrieving sub elements:\n",
493+
"\n",
494+
"```json\n",
495+
"{\n",
496+
" \"name\": \"Specialized Stump jumper\",\n",
497+
" \"metadata\": {\n",
498+
" \"model\": \"Stumpjumper\",\n",
499+
" \"brand\": \"Specialized\",\n",
500+
" \"type\": \"Enduro bikes\",\n",
501+
" \"price\": 3000\n",
502+
" },\n",
503+
"}\n",
504+
"```\n",
505+
"\n",
506+
"#### Full JSON Path support\n",
507+
"Because Redis enables full JSON path support, when creating an index schema, elements need to be indexed and selected by their path with the desired `name` AND `path` that points to where the data is located within the objects.\n",
508+
"\n",
509+
"> By default, RedisVL will assume the path as `$.{name}` if not provided in JSON fields schema. If nested provide path as `$.object.attribute`\n",
510+
"\n",
511+
"### As an example:"
512+
]
513+
},
514+
{
515+
"cell_type": "code",
516+
"execution_count": 32,
517+
"metadata": {},
518+
"outputs": [
519+
{
520+
"name": "stderr",
521+
"output_type": "stream",
522+
"text": [
523+
"/Users/robert.shelton/.pyenv/versions/3.11.9/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
524+
" warnings.warn(\n"
525+
]
526+
}
527+
],
528+
"source": [
529+
"import warnings\n",
530+
"from redisvl.utils.vectorize import HFTextVectorizer\n",
531+
"\n",
532+
"warnings.filterwarnings(\"ignore\", category=FutureWarning, message=r\".*clean_up_tokenization_spaces.*\")\n",
533+
"\n",
534+
"emb_model = HFTextVectorizer()\n",
535+
"\n",
536+
"bike_data = [\n",
537+
" {\n",
538+
" \"name\": \"Specialized Stump jumper\",\n",
539+
" \"metadata\": {\n",
540+
" \"model\": \"Stumpjumper\",\n",
541+
" \"brand\": \"Specialized\",\n",
542+
" \"type\": \"Enduro bikes\",\n",
543+
" \"price\": 3000\n",
544+
" },\n",
545+
" \"description\": \"The Specialized Stumpjumper is a versatile enduro bike that dominates both climbs and descents. Features a FACT 11m carbon fiber frame, FOX FLOAT suspension with 160mm travel, and SRAM X01 Eagle drivetrain. The asymmetric frame design and internal storage compartment make it a practical choice for all-day adventures.\"\n",
546+
" },\n",
547+
" {\n",
548+
" \"name\": \"bike_2\",\n",
549+
" \"metadata\": {\n",
550+
" \"model\": \"Slash\",\n",
551+
" \"brand\": \"Trek\",\n",
552+
" \"type\": \"Enduro bikes\",\n",
553+
" \"price\": 5000\n",
554+
" },\n",
555+
" \"description\": \"Trek's Slash is built for aggressive enduro riding and racing. Featuring Trek's Alpha Aluminum frame with RE:aktiv suspension technology, 160mm travel, and Knock Block frame protection. Equipped with Bontrager components and a Shimano XT drivetrain, this bike excels on technical trails and enduro race courses.\"\n",
556+
" }\n",
557+
"]\n",
558+
"\n",
559+
"bike_data = [{**d, \"bike_embedding\": emb_model.embed(d[\"description\"])} for d in bike_data]\n",
560+
"\n",
561+
"bike_schema = {\n",
562+
" \"index\": {\n",
563+
" \"name\": \"bike-json\",\n",
564+
" \"prefix\": \"bike-json\",\n",
565+
" \"storage_type\": \"json\", # JSON storage type\n",
566+
" },\n",
567+
" \"fields\": [\n",
568+
" {\n",
569+
" \"name\": \"model\",\n",
570+
" \"type\": \"tag\",\n",
571+
" \"path\": \"$.metadata.model\" # note the '$'\n",
572+
" },\n",
573+
" {\n",
574+
" \"name\": \"brand\",\n",
575+
" \"type\": \"tag\",\n",
576+
" \"path\": \"$.metadata.brand\" # note the '$'\n",
577+
" },\n",
578+
" {\n",
579+
" \"name\": \"price\",\n",
580+
" \"type\": \"numeric\",\n",
581+
" \"path\": \"$.metadata.price\" # note the '$'\n",
582+
" },\n",
583+
" {\n",
584+
" \"name\": \"bike_embedding\",\n",
585+
" \"type\": \"vector\",\n",
586+
" \"attrs\": {\n",
587+
" \"dims\": len(bike_data[0][\"bike_embedding\"]),\n",
588+
" \"distance_metric\": \"cosine\",\n",
589+
" \"algorithm\": \"flat\",\n",
590+
" \"datatype\": \"float32\"\n",
591+
" }\n",
592+
"\n",
593+
" }\n",
594+
" ],\n",
595+
"}"
596+
]
597+
},
598+
{
599+
"cell_type": "code",
600+
"execution_count": 39,
601+
"metadata": {},
602+
"outputs": [],
603+
"source": [
604+
"# construct a search index from the json schema\n",
605+
"bike_index = SearchIndex.from_dict(bike_schema)\n",
606+
"\n",
607+
"# connect to local redis instance\n",
608+
"bike_index.connect(\"redis://localhost:6379\")\n",
609+
"\n",
610+
"# create the index (no data yet)\n",
611+
"bike_index.create(overwrite=True)"
612+
]
613+
},
614+
{
615+
"cell_type": "code",
616+
"execution_count": 40,
617+
"metadata": {},
618+
"outputs": [
619+
{
620+
"data": {
621+
"text/plain": [
622+
"['bike-json:6c32bfb9d5114e23866d564af694a15b',\n",
623+
" 'bike-json:a22a2757f0f043f4a0abdd4ad5322989']"
624+
]
625+
},
626+
"execution_count": 40,
627+
"metadata": {},
628+
"output_type": "execute_result"
629+
}
630+
],
631+
"source": [
632+
"bike_index.load(bike_data)"
633+
]
634+
},
635+
{
636+
"cell_type": "code",
637+
"execution_count": 43,
638+
"metadata": {},
639+
"outputs": [],
640+
"source": [
641+
"from redisvl.query import VectorQuery\n",
642+
"\n",
643+
"vec = emb_model.embed(\"I'd like a bike for aggressive riding\")\n",
644+
"\n",
645+
"v = VectorQuery(vector=vec,\n",
646+
" vector_field_name=\"bike_embedding\",\n",
647+
" return_fields=[\n",
648+
" \"brand\",\n",
649+
" \"name\",\n",
650+
" ]\n",
651+
" )\n",
652+
"\n",
653+
"\n",
654+
"results = bike_index.query(v)"
655+
]
656+
},
657+
{
658+
"cell_type": "code",
659+
"execution_count": 42,
660+
"metadata": {},
661+
"outputs": [
662+
{
663+
"data": {
664+
"text/plain": [
665+
"[{'id': 'bike-json:a22a2757f0f043f4a0abdd4ad5322989',\n",
666+
" 'vector_distance': '0.519989073277',\n",
667+
" 'brand': 'Trek'},\n",
668+
" {'id': 'bike-json:6c32bfb9d5114e23866d564af694a15b',\n",
669+
" 'vector_distance': '0.657624483109',\n",
670+
" 'brand': 'Specialized'}]"
671+
]
672+
},
673+
"execution_count": 42,
674+
"metadata": {},
675+
"output_type": "execute_result"
676+
}
677+
],
678+
"source": [
679+
"results"
680+
]
681+
},
682+
{
683+
"cell_type": "markdown",
684+
"metadata": {},
685+
"source": [
686+
"# Cleanup"
687+
]
688+
},
689+
{
690+
"cell_type": "code",
691+
"execution_count": 44,
692+
"metadata": {},
693+
"outputs": [],
694+
"source": [
695+
"bike_index.delete()"
696+
]
508697
}
509698
],
510699
"metadata": {
511700
"kernelspec": {
512-
"display_name": "Python 3.8.13 ('redisvl2')",
701+
"display_name": "Python 3",
513702
"language": "python",
514703
"name": "python3"
515704
},
@@ -523,14 +712,9 @@
523712
"name": "python",
524713
"nbconvert_exporter": "python",
525714
"pygments_lexer": "ipython3",
526-
"version": "3.9.12"
715+
"version": "3.11.9"
527716
},
528-
"orig_nbformat": 4,
529-
"vscode": {
530-
"interpreter": {
531-
"hash": "9b1e6e9c2967143209c2f955cb869d1d3234f92dc4787f49f155f3abbdfb1316"
532-
}
533-
}
717+
"orig_nbformat": 4
534718
},
535719
"nbformat": 4,
536720
"nbformat_minor": 2

0 commit comments

Comments
 (0)