DOC-5533 updated vector search example to use same library as vector sets example

Author: andy-stark-redis

content/develop/clients/go/vecsearch.md

@@ -29,9 +29,8 @@ or JSON fields, Redis can retrieve documents that closely match the query in ter
 of their meaning.
 
 In the example below, we use the
-[`huggingfaceembedder`](https://pkg.go.dev/github.com/henomis/[email protected]/embedder/huggingface)
-package from the [`LinGoose`](https://pkg.go.dev/github.com/henomis/[email protected])
-framework to generate vector embeddings to store and index with
+[`Hugot`](https://pkg.go.dev/github.com/knights-analytics/hugot)
+library to generate vector embeddings to store and index with
 Redis Query Engine.  The code is first demonstrated for hash documents with a
 separate section to explain the
 [differences with JSON documents](#differences-with-json-documents).
@@ -47,38 +46,23 @@ for more information.
 
 ## Initialize
 
-Start a new Go module with the following command:
-
-```bash 
-go mod init vecexample
-```
-
-Then, in your module folder, install
-[`go-redis`]({{< relref "/develop/clients/go" >}})
-and the 
-[`huggingfaceembedder`](https://pkg.go.dev/github.com/henomis/[email protected]/embedder/huggingface)
-package:
+First, install [`go-redis`]({{< relref "/develop/clients/go" >}})
+if you haven't already done so. Then, install
+[`Hugot`](https://pkg.go.dev/github.com/knights-analytics/hugot)
+using the following command:
 
 ```bash
-go get github.com/redis/go-redis/v9
-go get github.com/henomis/lingoose/embedder/huggingface
+go get github.com/knights-analytics/hugot
 ```
 
 Add the following imports to your module's main program file:
 
 {{< clients-example set="home_query_vec" step="import" lang_filter="Go" >}}
 {{< /clients-example >}}
 
-You must also create a [HuggingFace account](https://huggingface.co/join)
-and add a new access token to use the embedding model. See the
-[HuggingFace](https://huggingface.co/docs/hub/en/security-tokens)
-docs to learn how to create and manage access tokens. Note that the
-account and the `all-MiniLM-L6-v2` model that we will use to produce
-the embeddings for this example are both available for free.
-
 ## Add a helper function
 
-The `huggingfaceembedder` model outputs the embeddings as a
+The `Hugot` model outputs the embeddings as a
 `[]float32` array. If you are storing your documents as
 [hash]({{< relref "/develop/data-types/hashes" >}}) objects, then you
 must convert this array to a `byte` string before adding it as a hash field.
@@ -119,11 +103,10 @@ and 384 dimensions, as required by the `all-MiniLM-L6-v2` embedding model.
 
 ## Create an embedder instance
 
-You need an instance of the `huggingfaceembedder` class to
+You need an instance of the `FeatureExtractionPipeline` class to
 generate the embeddings. Use the code below to create an
 instance that uses the `sentence-transformers/all-MiniLM-L6-v2`
-model, passing your HuggingFace access token to the `WithToken()`
-method.
+model:
 
 {{< clients-example set="home_query_vec" step="embedder" lang_filter="Go" >}}
 {{< /clients-example >}}
@@ -134,12 +117,12 @@ You can now supply the data objects, which will be indexed automatically
 when you add them with [`HSet()`]({{< relref "/commands/hset" >}}), as long as
 you use the `doc:` prefix specified in the index definition.
 
-Use the `Embed()` method of `huggingfacetransformer`
+Use the `RunPipeline()` method of `FeatureExtractionPipeline`
 as shown below to create the embeddings that represent the `content` fields.
 This method takes an array of strings and outputs a corresponding
-array of `Embedding` objects.
-Use the `ToFloat32()` method of `Embedding` to produce the array of float
-values that we need, and use the `floatsToBytes()` function we defined
+array of `FeatureExtractionOutput` objects.
+The `Embeddings` field of `FeatureExtractionOutput` contains the array of float
+values that you need for the index. Use the `floatsToBytes()` function defined
 above to convert this array to a `byte` string.
 
 {{< clients-example set="home_query_vec" step="add_data" lang_filter="Go" >}}
@@ -153,7 +136,7 @@ text. Redis calculates the similarity between the query vector and each
 embedding vector in the index as it runs the query. It then ranks the
 results in order of this numeric similarity value.
 
-The code below creates the query embedding using `Embed()`, as with
+The code below creates the query embedding using `RunPipeline()`, as with
 the indexing, and passes it as a parameter when the query executes
 (see
 [Vector search]({{< relref "/develop/ai/search-and-query/query/vector-search" >}})
@@ -163,14 +146,14 @@ for more information about using query parameters with embeddings).
 {{< /clients-example >}}
 
 The code is now ready to run, but note that it may take a while to complete when
-you run it for the first time (which happens because `huggingfacetransformer`
+you run it for the first time (which happens because `Hugot`
 must download the `all-MiniLM-L6-v2` model data before it can
 generate the embeddings). When you run the code, it outputs the following text:
 
 ```
-ID: doc:0, Distance:0.114169843495, Content:'That is a very happy person'
-ID: doc:1, Distance:0.610845327377, Content:'That is a happy dog'
-ID: doc:2, Distance:1.48624765873, Content:'Today is a sunny day'
+ID: doc:0, Distance:2.96992516518, Content:'That is a very happy person'
+ID: doc:1, Distance:17.3678302765, Content:'That is a happy dog'
+ID: doc:2, Distance:43.7771987915, Content:'Today is a sunny day'
 ```
 
 The results are ordered according to the value of the `vector_distance`
@@ -220,9 +203,9 @@ Apart from the `jdoc:` prefixes for the keys, the result from the JSON
 query is the same as for hash:
 
 ```
-ID: jdoc:0, Distance:0.114169843495, Content:'That is a very happy person'
-ID: jdoc:1, Distance:0.610845327377, Content:'That is a happy dog'
-ID: jdoc:2, Distance:1.48624765873, Content:'Today is a sunny day'
+ID: jdoc:0, Distance:2.96992516518, Content:'That is a very happy person'
+ID: jdoc:1, Distance:17.3678302765, Content:'That is a happy dog'
+ID: jdoc:2, Distance:43.7771987915, Content:'Today is a sunny day'
 ```
 
 ## Learn more

local_examples/client-specific/home_query_vec.go

@@ -8,7 +8,7 @@ import (
 	"fmt"
 	"math"
 
-	huggingfaceembedder "github.com/henomis/lingoose/embedder/huggingface"
+	"github.com/knights-analytics/hugot"
 	"github.com/redis/go-redis/v9"
 )
 
@@ -79,9 +79,37 @@ func main() {
 	// STEP_END
 
 	// STEP_START embedder
-	hf := huggingfaceembedder.New().
-		WithToken("<your-access-token>").
-		WithModel("sentence-transformers/all-MiniLM-L6-v2")
+	// Create a Hugot session
+	session, err := hugot.NewGoSession()
+	if err != nil {
+		panic(err)
+	}
+	defer func() {
+		err := session.Destroy()
+		if err != nil {
+			panic(err)
+		}
+	}()
+
+	// Download the model
+	downloadOptions := hugot.NewDownloadOptions()
+	downloadOptions.OnnxFilePath = "onnx/model.onnx" // Specify which ONNX file to use
+	modelPath, err := hugot.DownloadModel("sentence-transformers/all-MiniLM-L6-v2", "./models/", downloadOptions)
+	if err != nil {
+		panic(err)
+	}
+
+	// Create feature extraction pipeline configuration
+	config := hugot.FeatureExtractionConfig{
+		ModelPath: modelPath,
+		Name:      "embeddingPipeline",
+	}
+
+	// Create the feature extraction pipeline
+	embeddingPipeline, err := hugot.NewPipeline(session, config)
+	if err != nil {
+		panic(err)
+	}
 	// STEP_END
 
 	// STEP_START add_data
@@ -95,16 +123,17 @@ func main() {
 		"persons", "pets", "weather",
 	}
 
-	embeddings, err := hf.Embed(ctx, sentences)
+	// Generate embeddings using Hugot
+	embeddingResult, err := embeddingPipeline.RunPipeline(sentences)
 	if err != nil {
 		panic(err)
 	}
 
+	// Extract the embeddings from the result
+	embeddings := embeddingResult.Embeddings
+
 	for i, emb := range embeddings {
-		buffer := floatsToBytes(emb.ToFloat32())
-		if err != nil {
-			panic(err)
-		}
+		buffer := floatsToBytes(emb)
 
 		_, err = rdb.HSet(ctx,
 			fmt.Sprintf("doc:%v", i),
@@ -114,25 +143,25 @@ func main() {
 				"embedding": buffer,
 			},
 		).Result()
+
 		if err != nil {
 			panic(err)
 		}
 	}
 	// STEP_END
 
 	// STEP_START query
-	queryEmbedding, err := hf.Embed(ctx, []string{
+	// Generate query embedding using Hugot
+	queryResult, err := embeddingPipeline.RunPipeline([]string{
 		"That is a happy person",
 	})
-	if err != nil {
-		panic(err)
-	}
 
-	buffer := floatsToBytes(queryEmbedding[0].ToFloat32())
 	if err != nil {
 		panic(err)
 	}
 
+	buffer := floatsToBytes(queryResult.Embeddings[0])
+
 	results, err := rdb.FTSearchWithArgs(ctx,
 		"vector_idx",
 		"*=>[KNN 3 @embedding $vec AS vector_distance]",
@@ -147,6 +176,7 @@ func main() {
 			},
 		},
 	).Result()
+
 	if err != nil {
 		panic(err)
 	}
@@ -160,6 +190,13 @@ func main() {
 	// STEP_END
 
 	// STEP_START json_index
+	rdb.FTDropIndexWithArgs(ctx,
+		"vector_json_idx",
+		&redis.FTDropIndexOptions{
+			DeleteDocs: true,
+		},
+	)
+
 	_, err = rdb.FTCreate(ctx,
 		"vector_json_idx",
 		&redis.FTCreateOptions{
@@ -202,24 +239,27 @@ func main() {
 			map[string]any{
 				"content":   sentences[i],
 				"genre":     tags[i],
-				"embedding": emb.ToFloat32(),
+				"embedding": emb,
 			},
 		).Result()
+
 		if err != nil {
 			panic(err)
 		}
 	}
 	// STEP_END
 
 	// STEP_START json_query
-	jsonQueryEmbedding, err := hf.Embed(ctx, []string{
+	// Generate query embedding for JSON search using Hugot
+	jsonQueryResult, err := embeddingPipeline.RunPipeline([]string{
 		"That is a happy person",
 	})
+
 	if err != nil {
 		panic(err)
 	}
 
-	jsonBuffer := floatsToBytes(jsonQueryEmbedding[0].ToFloat32())
+	jsonBuffer := floatsToBytes(jsonQueryResult.Embeddings[0])
 
 	jsonResults, err := rdb.FTSearchWithArgs(ctx,
 		"vector_json_idx",
@@ -235,6 +275,7 @@ func main() {
 			},
 		},
 	).Result()
+
 	if err != nil {
 		panic(err)
 	}