DOC-4835 updated text and examples for redis-py

Author: andy-stark-redis

content/develop/clients/redis-py/queryjson.md

@@ -12,7 +12,7 @@ categories:
 description: Learn how to use the Redis query engine with JSON
 linkTitle: Index and query JSON
 title: Example - Index and query JSON documents
-weight: 2
+weight: 3
 ---
 
 This example shows how to create a

content/develop/clients/redis-py/transpipe.md

@@ -12,7 +12,7 @@ categories:
 description: Learn how to use Redis pipelines and transactions
 linkTitle: Pipelines/transactions
 title: Pipelines and transactions
-weight: 2
+weight: 5
 ---
 
 Redis lets you send a sequence of commands to the server together in a batch.

content/develop/clients/redis-py/vecsearch.md

@@ -9,10 +9,10 @@ categories:
 - oss
 - kubernetes
 - clients
-description: Store and query vector embeddings with Redis
-linkTitle: Vector embeddings
-title: Vector embeddings
-weight: 5
+description: Learn how to index and query vector embeddings with Redis
+linkTitle: Index and query vectors
+title: Index and query vectors
+weight: 4
 ---
 
 [Redis Query Engine]({{< relref "/develop/interact/search-and-query" >}})
@@ -28,196 +28,207 @@ similarity of an embedding generated from some query text with embeddings stored
 or JSON fields, Redis can retrieve documents that closely match the query in terms
 of their meaning.
 
-The [RedisVL]({{< relref "/develop/clients/redis-vl" >}}) library provides a
-high-level Python API to help you use vector embeddings with Redis Query
-Engine. The example below shows how RedisVL can retrieve data
-that has a similar meaning to the query text you supply.
+In the example below, we use the
+[`sentence-transformers`](https://pypi.org/project/sentence-transformers/)
+library to generate vector embeddings to store and index with
+Redis Query Engine.
 
 ## Initialize
 
-Start by importing the required classes:
+Install [`redis-py`]({{< relref "/develop/clients/redis-py" >}}) if you
+have not already done so. Also, install `sentence-transformers` with the
+following command:
+
+```bash
+pip install sentence-transformers
+```
+
+In a new Python source file, start by importing the required classes:
 
 ```python
-from redisvl.utils.vectorize.text.huggingface import (
-    HFTextVectorizer
-)
-from redisvl.index import SearchIndex
-from redisvl.query import VectorQuery
+from sentence_transformers import SentenceTransformer
+from redis.commands.search.query import Query
+from redis.commands.search.field import TextField, TagField, VectorField
+from redis.commands.search.indexDefinition import IndexDefinition, IndexType
+
+import numpy as np
+import redis
 ```
 
 The first of these imports is the
-[`HFTextVectorizer`](https://docs.redisvl.com/en/stable/api/vectorizer.html#hftextvectorizer)
-class, which generates
-an embedding from a section sample of text. Here, we create an instance of `HFTextVectorizer` 
-that uses the
+`SentenceTransformer` class, which generates an embedding from a section of text.
+Here, we create an instance of `SentenceTransformer` that uses the
 [`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
 model for the embeddings. This model generates vectors with 384 dimensions, regardless
 of the length of the input text, but note that the input is truncated to 256
 tokens (see
 [Word piece tokenization](https://huggingface.co/learn/nlp-course/en/chapter6/6)
-at the [Hugging Face](https://huggingface.co/) docs to learn more about how tokens
+at the [Hugging Face](https://huggingface.co/) docs to learn more about the way tokens
 are related to the original text).
 
 ```python
-hf = HFTextVectorizer(model="sentence-transformers/all-MiniLM-L6-v2")
+model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
 ```
 
 ## Create the index
 
-RedisVL's
-[`SearchIndex`](https://docs.redisvl.com/en/stable/api/searchindex.html#searchindex-api)
-class provides the
-[`from_dict()`](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.SearchIndex.from_dict)
-method, which lets you specify your index schema with a Python dictionary, as shown
-below. Another option is
-[`from_yaml()`](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.SearchIndex.from_yaml),
-which loads the index schema from a [YAML](https://en.wikipedia.org/wiki/YAML) file.
+Connect to Redis and delete any index previously created with the
+name `vector_idx`. (The `dropindex()` call throws an exception if
+the index doesn't already exist, which is why you need the
+`try: except:` block.)
+
+```python
+r = redis.Redis(decode_responses=True)
 
+try:
+    r.ft("vector_idx").dropindex(True)
+except redis.exceptions.ResponseError:
+    pass
+```
+
+Next, we create the index.
 The schema in the example below specifies hash objects for storage and includes
 three fields: the text content to index, a
 [tag]({{< relref "/develop/interact/search-and-query/advanced-concepts/tags" >}})
 field to represent the "genre" of the text, and the embedding vector generated from
-the original text content. The attributes of the vector field (specified in the
-`attrs` object) specify [HNSW]({{< relref "/develop/interact/search-and-query/advanced-concepts/vectors#hnsw-index" >}}) indexing, the
+the original text content. The `embedding` field specifies
+[HNSW]({{< relref "/develop/interact/search-and-query/advanced-concepts/vectors#hnsw-index" >}}) 
+indexing, the
 [L2]({{< relref "/develop/interact/search-and-query/advanced-concepts/vectors#distance-metrics" >}})
 vector distance metric, `Float32` values to represent the vector's components,
 and 384 dimensions, as required by the `all-MiniLM-L6-v2` embedding model.
 
 ```python
-index = SearchIndex.from_dict({
-    "index": {
-        "name": "vector_idx",
-        "prefix": "doc",
-        "storage_type": "hash",
-    },
-    "fields": [
-        {"name": "content", "type": "text"},
-        {"name": "genre", "type": "tag"},
-        {
-            "name": "embedding",
-            "type": "vector",
-            "attrs": {
-                "algorithm": "HNSW",
-                "dims": 384,
-                "distance_metric": "L2",
-                "datatype": "FLOAT32",
-            },
-        },
-    ],
-})
-```
-
-When you have created the `SearchIndex` object, you must connect to the Redis
-server using the
-[`connect()`](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.SearchIndex.connect)
-method and then use the
-[`create()`](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.SearchIndex.create)
-method to create the index in the database. The `overwrite` parameter for `create()`
-ensures that the index you create replaces any existing index with
-the same name. The `drop` parameter deletes any objects that were
-indexed by the index you are replacing.
+schema = (
+    TextField("content"),
+    TagField("genre"),
+    VectorField("embedding", "HNSW", {
+        "TYPE": "FLOAT32",
+        "DIM": 384,
+        "DISTANCE_METRIC":"L2"
+    })
+)
 
-```python
-index.connect("redis://localhost:6379")
-index.create(overwrite=True, drop=True)
+r.ft("vector_idx").create_index(
+    schema,
+    definition=IndexDefinition(
+        prefix=["doc:"], index_type=IndexType.HASH
+    )
+)
 ```
 
 ## Add data
 
 You can now supply the data objects, which will be indexed automatically
-when you add them using the
-[`load()`](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.SearchIndex.load)
-method. Use the
-[`embed()`](https://docs.redisvl.com/en/stable/api/vectorizer.html#redisvl.utils.vectorize.text.huggingface.HFTextVectorizer.embed)
-method of the `HFTextVectorizer` instance to create the embedding that represents the `content` field. By default, `embed()` returns a list of `float` values to represent the embedding
-vector, but you can also supply the `as_buffer` parameter to encode this list as a
-binary string. Use the string representation when you are indexing hash objects
+when you add them with [`hset()`]({{< relref "/commands/hset" >}}), as long as
+you use the `doc:` prefix specified in the index definition.
+
+Use the `model.encode()` method of `SentenceTransformer`
+as shown below to create the embedding that represents the `content` field.
+The `astype()` option that follows the `model.encode()` call specifies that
+we want a vector of `float32` values. The `tobytes()` option encodes the
+vector components together as a single binary string rather than the
+default Python list of `float` values.
+Use the binary string representation when you are indexing hash objects
 (as we are here), but use the default list of `float` for JSON objects.
 
 ```python
-hf = HFTextVectorizer(model="sentence-transformers/all-MiniLM-L6-v2")
-
-data = [
-    {
-        "content": "That is a very happy person",
-        "genre": "persons",
-        "embedding": hf.embed("That is a very happy person", as_buffer=True)
-    },
-    {
-        "content": "That is a happy dog",
-        "genre": "pets",
-        "embedding": hf.embed("That is a happy dog", as_buffer=True)
-    },
-    {
-        "content": "Today is a sunny day",
-        "genre": "weather",
-        "embedding": hf.embed("Today is a sunny day", as_buffer=True)
-    }
-]
+content = "That is a very happy person"
+
+r.hset("doc:0", mapping={
+    "content": content,
+    "genre": "persons",
+    "embedding": model.encode(content).astype(np.float32).tobytes(),
+})
+
+content = "That is a happy dog"
+
+r.hset("doc:1", mapping={
+    "content": content,
+    "genre": "pets",
+    "embedding": model.encode(content).astype(np.float32).tobytes(),
+})
+
+content = "Today is a sunny day"
 
-index.load(data)
+r.hset("doc:2", mapping={
+    "content": content,
+    "genre": "weather",
+    "embedding": model.encode(content).astype(np.float32).tobytes(),
+})
 ```
 
 ## Run a query
 
-After you have created the index, you are ready to run a query against it.
+After you have created the index and added the data, you are ready to run a query.
 To do this, you must create another embedding vector from your chosen query
 text. Redis calculates the similarity between the query vector and each
 embedding vector in the index as it runs the query. It then ranks the
 results in order of this numeric similarity value.
 
-Create a
-[`VectorQuery`](https://docs.redisvl.com/en/stable/api/query.html#vectorquery)
-instance by supplying the embedding vector for your
-query text, along with the hash or JSON field to match against and the number
-of results you want, as shown in the example below. Then, call the 
-[`query()`](https://docs.redisvl.com/en/stable/api/searchindex.html#redisvl.index.SearchIndex.query)
-method of `SearchIndex` with your `VectorQuery` instance to run the query.
+The code below creates the query embedding using `model.encode()`, as with
+the indexing, and passes it as a parameter when the query executes
+(see
+[Vector search]({{< relref "/develop/interact/search-and-query/query/vector-search" >}})
+for more information about using query parameters with embeddings).
 
 ```python
-query = VectorQuery(
-    vector=hf.embed("That is a happy person"),
-    vector_field_name="embedding",
-    return_fields=["content"],
-    num_results=3,
+q = Query(
+    "*=>[KNN 3 @embedding $vec AS vector_distance]"
+).return_field("score").dialect(2)
+
+query_text = "That is a happy person"
+
+res = r.ft("vector_idx").search(
+    q, query_params={
+        "vec": model.encode(query_text).astype(np.float32).tobytes()
+    }
 )
 
-results = index.query(query)
-print(results)
+print(res)
 ```
 
 The code is now ready to run, but note that it may take a while to complete when
 you run it for the first time (which happens because RedisVL must download the
 `all-MiniLM-L6-v2` model data before it can
-generate the embeddings). When you run the code, it outputs the following results
-as a list of dict objects:
+generate the embeddings). When you run the code, it outputs the following result
+object (slightly formatted here for clarity):
 
 ```Python
-[
-    {
-        'id': 'doc:cf3c28e7fdd44753af3080a9a7f8d8e9',
-        'vector_distance': '0.114169985056',
-        'content': 'That is a very happy person'
-    },
-    {
-        'id': 'doc:614508f297b644d0be47dde5373bb059',
-        'vector_distance': '0.610845386982',
-        'content': 'That is a happy dog'
-    },
-    {
-        'id': 'doc:930a7dfca0d74808baee490d747c9534',
-        'vector_distance': '1.48624813557',
-        'content': 'Today is a sunny day'
-    }
-]
+Result{
+    3 total,
+    docs: [
+        Document {
+            'id': 'doc:0',
+            'payload': None,
+            'vector_distance': '0.114169985056',
+            'content': 'That is a very happy person'
+        },
+        Document {
+            'id': 'doc:1',
+            'payload': None,
+            'vector_distance': '0.610845386982',
+            'content': 'That is a happy dog'
+        },
+        Document {
+            'id': 'doc:2',
+            'payload': None,
+            'vector_distance': '1.48624813557',
+            'content': 'Today is a sunny day'
+        }
+    ]
+}
 ```
 
 Note that the results are ordered according to the value of the `vector_distance`
 field, with the lowest distance indicating the greatest similarity to the query.
-As you would expect, the text *"That is a very happy person"* is the result that is
-most similar in meaning to the query text *"That is a happy person"*.
+As you would expect, the result for `doc:0` with the content text *"That is a very happy person"*
+is the result that is most similar in meaning to the query text
+*"That is a happy person"*.
 
 ## Learn more
 
-See the [RedisVL documentation](https://docs.redisvl.com/en/stable/index.html)
-for more details about its features and examples of how to use them.
+See
+[Vector search]({{< relref "/develop/interact/search-and-query/query/vector-search" >}})
+for more information about the indexing options, distance metrics, and query format
+for vectors.