Skip to content

Commit 7c22034

Browse files
tylerhutchersontylerhutcherson
committed
Merge branch 'main' into fix/RAAE-595-async-client-tweaks
2 parents 3abd8ee + 1172c49 commit 7c22034

File tree

1 file changed

+223
-32
lines changed

1 file changed

+223
-32
lines changed

‎docs/user_guide/05_hash_vs_json.ipynb‎

Lines changed: 223 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
},
2828
{
2929
"cell_type": "code",
30-
"execution_count": 1,
30+
"execution_count": 5,
3131
"metadata": {},
3232
"outputs": [],
3333
"source": [
@@ -308,36 +308,13 @@
308308
"metadata": {},
309309
"source": [
310310
"### Working with JSON\n",
311-
"Redis also supports native **JSON** objects. These can be multi-level (nested) objects, with full JSONPath support for updating/retrieving sub elements:\n",
312-
"\n",
313-
"```python\n",
314-
"{\n",
315-
" \"name\": \"bike\",\n",
316-
" \"metadata\": {\n",
317-
" \"model\": \"Deimos\",\n",
318-
" \"brand\": \"Ergonom\",\n",
319-
" \"type\": \"Enduro bikes\",\n",
320-
" \"price\": 4972,\n",
321-
" }\n",
322-
"}\n",
323-
"```\n",
324311
"\n",
325312
"JSON is best suited for use cases with the following characteristics:\n",
326313
"- Ease of use and data model flexibility are top concerns\n",
327314
"- Application data is already native JSON\n",
328315
"- Replacing another document storage/db solution"
329316
]
330317
},
331-
{
332-
"cell_type": "markdown",
333-
"metadata": {},
334-
"source": [
335-
"#### Full JSON Path support\n",
336-
"Because Redis enables full JSON path support, when creating an index schema, elements need to be indexed and selected by their path with the desired `name` AND `path` that points to where the data is located within the objects.\n",
337-
"\n",
338-
"> By default, RedisVL will assume the path as `$.{name}` if not provided in JSON fields schema."
339-
]
340-
},
341318
{
342319
"cell_type": "code",
343320
"execution_count": 11,
@@ -499,11 +476,230 @@
499476
"source": [
500477
"jindex.delete()"
501478
]
479+
},
480+
{
481+
"cell_type": "markdown",
482+
"metadata": {},
483+
"source": [
484+
"# Working with nested data in JSON\n",
485+
"\n",
486+
"Redis also supports native **JSON** objects. These can be multi-level (nested) objects, with full JSONPath support for updating/retrieving sub elements:\n",
487+
"\n",
488+
"```json\n",
489+
"{\n",
490+
" \"name\": \"Specialized Stump jumper\",\n",
491+
" \"metadata\": {\n",
492+
" \"model\": \"Stumpjumper\",\n",
493+
" \"brand\": \"Specialized\",\n",
494+
" \"type\": \"Enduro bikes\",\n",
495+
" \"price\": 3000\n",
496+
" },\n",
497+
"}\n",
498+
"```\n",
499+
"\n",
500+
"#### Full JSON Path support\n",
501+
"Because Redis enables full JSON path support, when creating an index schema, elements need to be indexed and selected by their path with the desired `name` AND `path` that points to where the data is located within the objects.\n",
502+
"\n",
503+
"> By default, RedisVL will assume the path as `$.{name}` if not provided in JSON fields schema. If nested provide path as `$.object.attribute`\n",
504+
"\n",
505+
"### As an example:"
506+
]
507+
},
508+
{
509+
"cell_type": "code",
510+
"execution_count": 45,
511+
"metadata": {},
512+
"outputs": [
513+
{
514+
"name": "stderr",
515+
"output_type": "stream",
516+
"text": [
517+
"/Users/robert.shelton/.pyenv/versions/3.11.9/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
518+
" warnings.warn(\n"
519+
]
520+
}
521+
],
522+
"source": [
523+
"from redisvl.utils.vectorize import HFTextVectorizer\n",
524+
"\n",
525+
"emb_model = HFTextVectorizer()\n",
526+
"\n",
527+
"bike_data = [\n",
528+
" {\n",
529+
" \"name\": \"Specialized Stump jumper\",\n",
530+
" \"metadata\": {\n",
531+
" \"model\": \"Stumpjumper\",\n",
532+
" \"brand\": \"Specialized\",\n",
533+
" \"type\": \"Enduro bikes\",\n",
534+
" \"price\": 3000\n",
535+
" },\n",
536+
" \"description\": \"The Specialized Stumpjumper is a versatile enduro bike that dominates both climbs and descents. Features a FACT 11m carbon fiber frame, FOX FLOAT suspension with 160mm travel, and SRAM X01 Eagle drivetrain. The asymmetric frame design and internal storage compartment make it a practical choice for all-day adventures.\"\n",
537+
" },\n",
538+
" {\n",
539+
" \"name\": \"bike_2\",\n",
540+
" \"metadata\": {\n",
541+
" \"model\": \"Slash\",\n",
542+
" \"brand\": \"Trek\",\n",
543+
" \"type\": \"Enduro bikes\",\n",
544+
" \"price\": 5000\n",
545+
" },\n",
546+
" \"description\": \"Trek's Slash is built for aggressive enduro riding and racing. Featuring Trek's Alpha Aluminum frame with RE:aktiv suspension technology, 160mm travel, and Knock Block frame protection. Equipped with Bontrager components and a Shimano XT drivetrain, this bike excels on technical trails and enduro race courses.\"\n",
547+
" }\n",
548+
"]\n",
549+
"\n",
550+
"bike_data = [{**d, \"bike_embedding\": emb_model.embed(d[\"description\"])} for d in bike_data]\n",
551+
"\n",
552+
"bike_schema = {\n",
553+
" \"index\": {\n",
554+
" \"name\": \"bike-json\",\n",
555+
" \"prefix\": \"bike-json\",\n",
556+
" \"storage_type\": \"json\", # JSON storage type\n",
557+
" },\n",
558+
" \"fields\": [\n",
559+
" {\n",
560+
" \"name\": \"model\",\n",
561+
" \"type\": \"tag\",\n",
562+
" \"path\": \"$.metadata.model\" # note the '$'\n",
563+
" },\n",
564+
" {\n",
565+
" \"name\": \"brand\",\n",
566+
" \"type\": \"tag\",\n",
567+
" \"path\": \"$.metadata.brand\"\n",
568+
" },\n",
569+
" {\n",
570+
" \"name\": \"price\",\n",
571+
" \"type\": \"numeric\",\n",
572+
" \"path\": \"$.metadata.price\"\n",
573+
" },\n",
574+
" {\n",
575+
" \"name\": \"bike_embedding\",\n",
576+
" \"type\": \"vector\",\n",
577+
" \"attrs\": {\n",
578+
" \"dims\": len(bike_data[0][\"bike_embedding\"]),\n",
579+
" \"distance_metric\": \"cosine\",\n",
580+
" \"algorithm\": \"flat\",\n",
581+
" \"datatype\": \"float32\"\n",
582+
" }\n",
583+
"\n",
584+
" }\n",
585+
" ],\n",
586+
"}"
587+
]
588+
},
589+
{
590+
"cell_type": "code",
591+
"execution_count": 46,
592+
"metadata": {},
593+
"outputs": [],
594+
"source": [
595+
"# construct a search index from the json schema\n",
596+
"bike_index = SearchIndex.from_dict(bike_schema)\n",
597+
"\n",
598+
"# connect to local redis instance\n",
599+
"bike_index.connect(\"redis://localhost:6379\")\n",
600+
"\n",
601+
"# create the index (no data yet)\n",
602+
"bike_index.create(overwrite=True)"
603+
]
604+
},
605+
{
606+
"cell_type": "code",
607+
"execution_count": 47,
608+
"metadata": {},
609+
"outputs": [
610+
{
611+
"data": {
612+
"text/plain": [
613+
"['bike-json:de92cb9955434575b20f4e87a30b03d5',\n",
614+
" 'bike-json:054ab3718b984532b924946fa5ce00c6']"
615+
]
616+
},
617+
"execution_count": 47,
618+
"metadata": {},
619+
"output_type": "execute_result"
620+
}
621+
],
622+
"source": [
623+
"bike_index.load(bike_data)"
624+
]
625+
},
626+
{
627+
"cell_type": "code",
628+
"execution_count": 48,
629+
"metadata": {},
630+
"outputs": [],
631+
"source": [
632+
"from redisvl.query import VectorQuery\n",
633+
"\n",
634+
"vec = emb_model.embed(\"I'd like a bike for aggressive riding\")\n",
635+
"\n",
636+
"v = VectorQuery(vector=vec,\n",
637+
" vector_field_name=\"bike_embedding\",\n",
638+
" return_fields=[\n",
639+
" \"brand\",\n",
640+
" \"name\",\n",
641+
" \"$.metadata.type\"\n",
642+
" ]\n",
643+
" )\n",
644+
"\n",
645+
"\n",
646+
"results = bike_index.query(v)"
647+
]
648+
},
649+
{
650+
"cell_type": "markdown",
651+
"metadata": {},
652+
"source": [
653+
"**Note:** As shown in the example if you want to retrieve a field from json object that was not indexed you will also need to supply the full path as with `$.metadata.type`."
654+
]
655+
},
656+
{
657+
"cell_type": "code",
658+
"execution_count": 49,
659+
"metadata": {},
660+
"outputs": [
661+
{
662+
"data": {
663+
"text/plain": [
664+
"[{'id': 'bike-json:054ab3718b984532b924946fa5ce00c6',\n",
665+
" 'vector_distance': '0.519989073277',\n",
666+
" 'brand': 'Trek',\n",
667+
" '$.metadata.type': 'Enduro bikes'},\n",
668+
" {'id': 'bike-json:de92cb9955434575b20f4e87a30b03d5',\n",
669+
" 'vector_distance': '0.657624483109',\n",
670+
" 'brand': 'Specialized',\n",
671+
" '$.metadata.type': 'Enduro bikes'}]"
672+
]
673+
},
674+
"execution_count": 49,
675+
"metadata": {},
676+
"output_type": "execute_result"
677+
}
678+
],
679+
"source": [
680+
"results"
681+
]
682+
},
683+
{
684+
"cell_type": "markdown",
685+
"metadata": {},
686+
"source": [
687+
"# Cleanup"
688+
]
689+
},
690+
{
691+
"cell_type": "code",
692+
"execution_count": 44,
693+
"metadata": {},
694+
"outputs": [],
695+
"source": [
696+
"bike_index.delete()"
697+
]
502698
}
503699
],
504700
"metadata": {
505701
"kernelspec": {
506-
"display_name": "Python 3.8.13 ('redisvl2')",
702+
"display_name": "Python 3",
507703
"language": "python",
508704
"name": "python3"
509705
},
@@ -517,14 +713,9 @@
517713
"name": "python",
518714
"nbconvert_exporter": "python",
519715
"pygments_lexer": "ipython3",
520-
"version": "3.9.12"
716+
"version": "3.11.9"
521717
},
522-
"orig_nbformat": 4,
523-
"vscode": {
524-
"interpreter": {
525-
"hash": "9b1e6e9c2967143209c2f955cb869d1d3234f92dc4787f49f155f3abbdfb1316"
526-
}
527-
}
718+
"orig_nbformat": 4
528719
},
529720
"nbformat": 4,
530721
"nbformat_minor": 2

0 commit comments

Comments
 (0)