From c81a2955909c29a3be6784712e4ab0c5bd7122bd Mon Sep 17 00:00:00 2001 From: paoloredis Date: Thu, 20 Mar 2025 10:57:30 +0100 Subject: [PATCH 01/13] Port redisvl documentation --- content/integrate/redisvl/api/_index.md | 54 + content/integrate/redisvl/api/cache.md | 320 +++++ content/integrate/redisvl/api/filter.md | 350 ++++++ content/integrate/redisvl/api/query.md | 920 +++++++++++++++ content/integrate/redisvl/api/reranker.md | 301 +++++ content/integrate/redisvl/api/router.md | 323 +++++ content/integrate/redisvl/api/schema.md | 343 ++++++ content/integrate/redisvl/api/searchindex.md | 853 ++++++++++++++ .../integrate/redisvl/api/session_manager.md | 278 +++++ content/integrate/redisvl/api/vectorizer.md | 1035 +++++++++++++++++ content/integrate/redisvl/overview/_index.md | 18 + content/integrate/redisvl/overview/cli.md | 236 ++++ .../redisvl/overview/installation.md | 80 ++ .../redisvl/user_guide/01_getting_started.md | 482 ++++++++ .../redisvl/user_guide/02_hybrid_queries.md | 727 ++++++++++++ .../redisvl/user_guide/03_llmcache.md | 474 ++++++++ .../redisvl/user_guide/04_vectorizers.md | 496 ++++++++ .../redisvl/user_guide/05_hash_vs_json.md | 345 ++++++ .../redisvl/user_guide/06_rerankers.md | 223 ++++ .../redisvl/user_guide/07_session_manager.md | 198 ++++ .../redisvl/user_guide/08_semantic_router.md | 315 +++++ .../integrate/redisvl/user_guide/_index.md | 56 + 22 files changed, 8427 insertions(+) create mode 100644 content/integrate/redisvl/api/_index.md create mode 100644 content/integrate/redisvl/api/cache.md create mode 100644 content/integrate/redisvl/api/filter.md create mode 100644 content/integrate/redisvl/api/query.md create mode 100644 content/integrate/redisvl/api/reranker.md create mode 100644 content/integrate/redisvl/api/router.md create mode 100644 content/integrate/redisvl/api/schema.md create mode 100644 content/integrate/redisvl/api/searchindex.md create mode 100644 content/integrate/redisvl/api/session_manager.md create mode 100644 content/integrate/redisvl/api/vectorizer.md create mode 100644 content/integrate/redisvl/overview/_index.md create mode 100644 content/integrate/redisvl/overview/cli.md create mode 100644 content/integrate/redisvl/overview/installation.md create mode 100644 content/integrate/redisvl/user_guide/01_getting_started.md create mode 100644 content/integrate/redisvl/user_guide/02_hybrid_queries.md create mode 100644 content/integrate/redisvl/user_guide/03_llmcache.md create mode 100644 content/integrate/redisvl/user_guide/04_vectorizers.md create mode 100644 content/integrate/redisvl/user_guide/05_hash_vs_json.md create mode 100644 content/integrate/redisvl/user_guide/06_rerankers.md create mode 100644 content/integrate/redisvl/user_guide/07_session_manager.md create mode 100644 content/integrate/redisvl/user_guide/08_semantic_router.md create mode 100644 content/integrate/redisvl/user_guide/_index.md diff --git a/content/integrate/redisvl/api/_index.md b/content/integrate/redisvl/api/_index.md new file mode 100644 index 0000000000..8a37bc7d01 --- /dev/null +++ b/content/integrate/redisvl/api/_index.md @@ -0,0 +1,54 @@ +--- +linkTitle: RedisVL API +title: RedisVL API +type: integration +--- + + +Reference documentation for the RedisVL API. + + + +* [Schema](schema/) + * [IndexSchema](schema/#indexschema) + * [Defining Fields](schema/#defining-fields) + * [Supported Field Types and Attributes](schema/#supported-field-types-and-attributes) +* [Search Index Classes](searchindex/) + * [SearchIndex](searchindex/#searchindex) + * [AsyncSearchIndex](searchindex/#asyncsearchindex) +* [Query](query/) + * [VectorQuery](query/#vectorquery) + * [VectorRangeQuery](query/#vectorrangequery) + * [FilterQuery](query/#filterquery) + * [CountQuery](query/#countquery) +* [Filter](filter/) + * [FilterExpression](filter/#filterexpression) + * [Tag](filter/#tag) + * [Text](filter/#text) + * [Num](filter/#num) + * [Geo](filter/#geo) + * [GeoRadius](filter/#georadius) +* [Vectorizers](vectorizer/) + * [HFTextVectorizer](vectorizer/#hftextvectorizer) + * [OpenAITextVectorizer](vectorizer/#openaitextvectorizer) + * [AzureOpenAITextVectorizer](vectorizer/#azureopenaitextvectorizer) + * [VertexAITextVectorizer](vectorizer/#vertexaitextvectorizer) + * [CohereTextVectorizer](vectorizer/#coheretextvectorizer) + * [BedrockTextVectorizer](vectorizer/#bedrocktextvectorizer) + * [CustomTextVectorizer](vectorizer/#customtextvectorizer) + * [VoyageAITextVectorizer](vectorizer/#voyageaitextvectorizer) +* [Rerankers](reranker/) + * [CohereReranker](reranker/#coherereranker) + * [HFCrossEncoderReranker](reranker/#hfcrossencoderreranker) + * [VoyageAIReranker](reranker/#voyageaireranker) +* [LLM Cache](cache/) + * [SemanticCache](cache/#semanticcache) +* [LLM Session Manager](session_manager/) + * [SemanticSessionManager](session_manager/#semanticsessionmanager) + * [StandardSessionManager](session_manager/#standardsessionmanager) +* [Semantic Router](router/) + * [Semantic Router](router/#semantic-router-api) + * [Routing Config](router/#routing-config) + * [Route](router/#route) + * [Route Match](router/#route-match) + * [Distance Aggregation Method](router/#distance-aggregation-method) diff --git a/content/integrate/redisvl/api/cache.md b/content/integrate/redisvl/api/cache.md new file mode 100644 index 0000000000..cbfc7169b0 --- /dev/null +++ b/content/integrate/redisvl/api/cache.md @@ -0,0 +1,320 @@ +--- +linkTitle: LLM Cache +title: LLM Cache +type: integration +--- + + +## SemanticCache + + + +### *class* SemanticCache(name='llmcache', distance_threshold=0.1, ttl=None, vectorizer=None, filterable_fields=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, \*\*kwargs) + +Bases: `BaseLLMCache` + +Semantic Cache for Large Language Models. + +Semantic Cache for Large Language Models. + +* **Parameters:** + * **name** (*str* *,* *optional*) – The name of the semantic cache search index. + Defaults to “llmcache”. + * **distance_threshold** (*float* *,* *optional*) – Semantic threshold for the + cache. Defaults to 0.1. + * **ttl** (*Optional* *[**int* *]* *,* *optional*) – The time-to-live for records cached + in Redis. Defaults to None. + * **vectorizer** (*Optional* *[**BaseVectorizer* *]* *,* *optional*) – The vectorizer for the cache. + Defaults to HFTextVectorizer. + * **filterable_fields** (*Optional* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *]*) – An optional list of RedisVL fields + that can be used to customize cache retrieval with filters. + * **redis_client** (*Optional* *[**Redis* *]* *,* *optional*) – A redis client connection instance. + Defaults to None. + * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. + * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments + for the redis client. Defaults to empty {}. + * **overwrite** (*bool*) – Whether or not to force overwrite the schema for + the semantic cache index. Defaults to false. +* **Raises:** + * **TypeError** – If an invalid vectorizer is provided. + * **TypeError** – If the TTL value is not an int. + * **ValueError** – If the threshold is not between 0 and 1. + * **ValueError** – If existing schema does not match new schema and overwrite is False. + +#### *async* acheck(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None) + +Async check the semantic cache for results similar to the specified prompt +or vector. + +This method searches the cache using vector similarity with +either a raw text prompt (converted to a vector) or a provided vector as +input. It checks for semantically similar prompts and fetches the cached +LLM responses. + +* **Parameters:** + * **prompt** (*Optional* *[**str* *]* *,* *optional*) – The text prompt to search for in + the cache. + * **vector** (*Optional* *[**List* *[**float* *]* *]* *,* *optional*) – The vector representation + of the prompt to search for in the cache. + * **num_results** (*int* *,* *optional*) – The number of cached results to return. + Defaults to 1. + * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to include + in each returned result. If None, defaults to all available + fields in the cached entry. + * **filter_expression** (*Optional* *[*[*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]*) – Optional filter expression + that can be used to filter cache results. Defaults to None and + the full cache will be searched. + * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic + vector distance. +* **Returns:** + A list of dicts containing the requested + : return fields for each similar cached response. +* **Return type:** + List[Dict[str, Any]] +* **Raises:** + * **ValueError** – If neither a prompt nor a vector is specified. + * **ValueError** – if ‘vector’ has incorrect dimensions. + * **TypeError** – If return_fields is not a list when provided. + +```python +response = await cache.acheck( + prompt="What is the captial city of France?" +) +``` + +#### *async* adrop(ids=None, keys=None) + +Async expire specific entries from the cache by id or specific +Redis key. + +* **Parameters:** + * **ids** (*Optional* *[**str* *]*) – The document ID or IDs to remove from the cache. + * **keys** (*Optional* *[**str* *]*) – The Redis keys to remove from the cache. +* **Return type:** + None + +#### *async* astore(prompt, response, vector=None, metadata=None, filters=None, ttl=None) + +Async stores the specified key-value pair in the cache along with metadata. + +* **Parameters:** + * **prompt** (*str*) – The user prompt to cache. + * **response** (*str*) – The LLM response to cache. + * **vector** (*Optional* *[**List* *[**float* *]* *]* *,* *optional*) – The prompt vector to + cache. Defaults to None, and the prompt vector is generated on + demand. + * **metadata** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The optional metadata to cache + alongside the prompt and response. Defaults to None. + * **filters** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]*) – The optional tag to assign to the cache entry. + Defaults to None. + * **ttl** (*Optional* *[**int* *]*) – The optional TTL override to use on this individual cache + entry. Defaults to the global TTL setting. +* **Returns:** + The Redis key for the entries added to the semantic cache. +* **Return type:** + str +* **Raises:** + * **ValueError** – If neither prompt nor vector is specified. + * **ValueError** – if vector has incorrect dimensions. + * **TypeError** – If provided metadata is not a dictionary. + +```python +key = await cache.astore( + prompt="What is the captial city of France?", + response="Paris", + metadata={"city": "Paris", "country": "France"} +) +``` + +#### *async* aupdate(key, \*\*kwargs) + +Async update specific fields within an existing cache entry. If no fields +are passed, then only the document TTL is refreshed. + +* **Parameters:** + **key** (*str*) – the key of the document to update using kwargs. +* **Raises:** + * **ValueError if an incorrect mapping is provided as a kwarg.** – + * **TypeError if metadata is provided and not** **of** **type dict.** – +* **Return type:** + None + +```python +key = await cache.astore('this is a prompt', 'this is a response') +await cache.aupdate( + key, + metadata={"hit_count": 1, "model_name": "Llama-2-7b"} +) +``` + +#### check(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None) + +Checks the semantic cache for results similar to the specified prompt +or vector. + +This method searches the cache using vector similarity with +either a raw text prompt (converted to a vector) or a provided vector as +input. It checks for semantically similar prompts and fetches the cached +LLM responses. + +* **Parameters:** + * **prompt** (*Optional* *[**str* *]* *,* *optional*) – The text prompt to search for in + the cache. + * **vector** (*Optional* *[**List* *[**float* *]* *]* *,* *optional*) – The vector representation + of the prompt to search for in the cache. + * **num_results** (*int* *,* *optional*) – The number of cached results to return. + Defaults to 1. + * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to include + in each returned result. If None, defaults to all available + fields in the cached entry. + * **filter_expression** (*Optional* *[*[*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]*) – Optional filter expression + that can be used to filter cache results. Defaults to None and + the full cache will be searched. + * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic + vector distance. +* **Returns:** + A list of dicts containing the requested + : return fields for each similar cached response. +* **Return type:** + List[Dict[str, Any]] +* **Raises:** + * **ValueError** – If neither a prompt nor a vector is specified. + * **ValueError** – if ‘vector’ has incorrect dimensions. + * **TypeError** – If return_fields is not a list when provided. + +```python +response = cache.check( + prompt="What is the captial city of France?" +) +``` + +#### clear() + +Clear the cache of all keys while preserving the index. + +* **Return type:** + None + +#### delete() + +Clear the semantic cache of all keys and remove the underlying search +index. + +* **Return type:** + None + +#### drop(ids=None, keys=None) + +Manually expire specific entries from the cache by id or specific +Redis key. + +* **Parameters:** + * **ids** (*Optional* *[**str* *]*) – The document ID or IDs to remove from the cache. + * **keys** (*Optional* *[**str* *]*) – The Redis keys to remove from the cache. +* **Return type:** + None + +#### set_threshold(distance_threshold) + +Sets the semantic distance threshold for the cache. + +* **Parameters:** + **distance_threshold** (*float*) – The semantic distance threshold for + the cache. +* **Raises:** + **ValueError** – If the threshold is not between 0 and 1. +* **Return type:** + None + +#### set_ttl(ttl=None) + +Set the default TTL, in seconds, for entries in the cache. + +* **Parameters:** + **ttl** (*Optional* *[**int* *]* *,* *optional*) – The optional time-to-live expiration + for the cache, in seconds. +* **Raises:** + **ValueError** – If the time-to-live value is not an integer. + +#### store(prompt, response, vector=None, metadata=None, filters=None, ttl=None) + +Stores the specified key-value pair in the cache along with metadata. + +* **Parameters:** + * **prompt** (*str*) – The user prompt to cache. + * **response** (*str*) – The LLM response to cache. + * **vector** (*Optional* *[**List* *[**float* *]* *]* *,* *optional*) – The prompt vector to + cache. Defaults to None, and the prompt vector is generated on + demand. + * **metadata** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The optional metadata to cache + alongside the prompt and response. Defaults to None. + * **filters** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]*) – The optional tag to assign to the cache entry. + Defaults to None. + * **ttl** (*Optional* *[**int* *]*) – The optional TTL override to use on this individual cache + entry. Defaults to the global TTL setting. +* **Returns:** + The Redis key for the entries added to the semantic cache. +* **Return type:** + str +* **Raises:** + * **ValueError** – If neither prompt nor vector is specified. + * **ValueError** – if vector has incorrect dimensions. + * **TypeError** – If provided metadata is not a dictionary. + +```python +key = cache.store( + prompt="What is the captial city of France?", + response="Paris", + metadata={"city": "Paris", "country": "France"} +) +``` + +#### update(key, \*\*kwargs) + +Update specific fields within an existing cache entry. If no fields +are passed, then only the document TTL is refreshed. + +* **Parameters:** + **key** (*str*) – the key of the document to update using kwargs. +* **Raises:** + * **ValueError if an incorrect mapping is provided as a kwarg.** – + * **TypeError if metadata is provided and not** **of** **type dict.** – +* **Return type:** + None + +```python +key = cache.store('this is a prompt', 'this is a response') +cache.update(key, metadata={"hit_count": 1, "model_name": "Llama-2-7b"}) +) +``` + +#### *property* aindex *: [AsyncSearchIndex](searchindex.md#redisvl.index.AsyncSearchIndex) | None* + +The underlying AsyncSearchIndex for the cache. + +* **Returns:** + The async search index. +* **Return type:** + [AsyncSearchIndex](searchindex.md#redisvl.index.AsyncSearchIndex) + +#### *property* distance_threshold *: float* + +The semantic distance threshold for the cache. + +* **Returns:** + The semantic distance threshold. +* **Return type:** + float + +#### *property* index *: [SearchIndex](searchindex.md#redisvl.index.SearchIndex)* + +The underlying SearchIndex for the cache. + +* **Returns:** + The search index. +* **Return type:** + [SearchIndex](searchindex.md#redisvl.index.SearchIndex) + +#### *property* ttl *: int | None* + +The default TTL, in seconds, for entries in the cache. diff --git a/content/integrate/redisvl/api/filter.md b/content/integrate/redisvl/api/filter.md new file mode 100644 index 0000000000..b61699e6f1 --- /dev/null +++ b/content/integrate/redisvl/api/filter.md @@ -0,0 +1,350 @@ +--- +linkTitle: Filter +title: Filter +type: integration +--- + + + + +## FilterExpression + +### *class* FilterExpression(\_filter=None, operator=None, left=None, right=None) + +A FilterExpression is a logical combination of filters in RedisVL. + +FilterExpressions can be combined using the & and | operators to create +complex expressions that evaluate to the Redis Query language. + +This presents an interface by which users can create complex queries +without having to know the Redis Query language. + +```python +from redisvl.query.filter import Tag, Num + +brand_is_nike = Tag("brand") == "nike" +price_is_over_100 = Num("price") < 100 +f = brand_is_nike & price_is_over_100 + +print(str(f)) + +>> (@brand:{nike} @price:[-inf (100)]) +``` + +This can be combined with the VectorQuery class to create a query: + +```python +from redisvl.query import VectorQuery + +v = VectorQuery( + vector=[0.1, 0.1, 0.5, ...], + vector_field_name="product_embedding", + return_fields=["product_id", "brand", "price"], + filter_expression=f, +) +``` + +#### NOTE +Filter expressions are typically not called directly. Instead they are +built by combining filter statements using the & and | operators. + +* **Parameters:** + * **\_filter** (*str* *|* *None*) + * **operator** (*FilterOperator* *|* *None*) + * **left** ([*FilterExpression*](#redisvl.query.filter.FilterExpression) *|* *None*) + * **right** ([*FilterExpression*](#redisvl.query.filter.FilterExpression) *|* *None*) + +## Tag + +### *class* Tag(field) + +A Tag filter can be applied to Tag fields + +* **Parameters:** + **field** (*str*) + +#### \_\_eq_\_(other) + +Create a Tag equality filter expression. + +* **Parameters:** + **other** (*Union* *[**List* *[**str* *]* *,* *str* *]*) – The tag(s) to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Tag + +f = Tag("brand") == "nike" +``` + +#### \_\_ne_\_(other) + +Create a Tag inequality filter expression. + +* **Parameters:** + **other** (*Union* *[**List* *[**str* *]* *,* *str* *]*) – The tag(s) to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Tag +f = Tag("brand") != "nike" +``` + +#### \_\_str_\_() + +Return the Redis Query string for the Tag filter + +* **Return type:** + str + +## Text + +### *class* Text(field) + +A Text is a FilterField representing a text field in a Redis index. + +* **Parameters:** + **field** (*str*) + +#### \_\_eq_\_(other) + +Create a Text equality filter expression. These expressions yield +filters that enforce an exact match on the supplied term(s). + +* **Parameters:** + **other** (*str*) – The text value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Text + +f = Text("job") == "engineer" +``` + +#### \_\_mod_\_(other) + +Create a Text “LIKE” filter expression. A flexible expression that +yields filters that can use a variety of additional operators like +wildcards (\*), fuzzy matches (%%), or combinatorics (|) of the supplied +term(s). + +* **Parameters:** + **other** (*str*) – The text value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Text + +f = Text("job") % "engine*" # suffix wild card match +f = Text("job") % "%%engine%%" # fuzzy match w/ Levenshtein Distance +f = Text("job") % "engineer|doctor" # contains either term in field +f = Text("job") % "engineer doctor" # contains both terms in field +``` + +#### \_\_ne_\_(other) + +Create a Text inequality filter expression. These expressions yield +negated filters on exact matches on the supplied term(s). Opposite of an +equality filter expression. + +* **Parameters:** + **other** (*str*) – The text value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Text + +f = Text("job") != "engineer" +``` + +#### \_\_str_\_() + +Return the Redis Query string for the Text filter + +* **Return type:** + str + +## Num + +### *class* Num(field) + +A Num is a FilterField representing a numeric field in a Redis index. + +* **Parameters:** + **field** (*str*) + +#### \_\_eq_\_(other) + +Create a Numeric equality filter expression. + +* **Parameters:** + **other** (*int*) – The value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Num +f = Num("zipcode") == 90210 +``` + +#### \_\_ge_\_(other) + +Create a Numeric greater than or equal to filter expression. + +* **Parameters:** + **other** (*int*) – The value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Num + +f = Num("age") >= 18 +``` + +#### \_\_gt_\_(other) + +Create a Numeric greater than filter expression. + +* **Parameters:** + **other** (*int*) – The value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Num + +f = Num("age") > 18 +``` + +#### \_\_le_\_(other) + +Create a Numeric less than or equal to filter expression. + +* **Parameters:** + **other** (*int*) – The value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Num + +f = Num("age") <= 18 +``` + +#### \_\_lt_\_(other) + +Create a Numeric less than filter expression. + +* **Parameters:** + **other** (*int*) – The value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Num + +f = Num("age") < 18 +``` + +#### \_\_ne_\_(other) + +Create a Numeric inequality filter expression. + +* **Parameters:** + **other** (*int*) – The value to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Num + +f = Num("zipcode") != 90210 +``` + +#### \_\_str_\_() + +Return the Redis Query string for the Numeric filter + +* **Return type:** + str + +## Geo + +### *class* Geo(field) + +A Geo is a FilterField representing a geographic (lat/lon) field in a +Redis index. + +* **Parameters:** + **field** (*str*) + +#### \_\_eq_\_(other) + +Create a geographic filter within a specified GeoRadius. + +* **Parameters:** + **other** ([*GeoRadius*](#redisvl.query.filter.GeoRadius)) – The geographic spec to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Geo, GeoRadius + +f = Geo("location") == GeoRadius(-122.4194, 37.7749, 1, unit="m") +``` + +#### \_\_ne_\_(other) + +Create a geographic filter outside of a specified GeoRadius. + +* **Parameters:** + **other** ([*GeoRadius*](#redisvl.query.filter.GeoRadius)) – The geographic spec to filter on. +* **Return type:** + [*FilterExpression*](#redisvl.query.filter.FilterExpression) + +```python +from redisvl.query.filter import Geo, GeoRadius + +f = Geo("location") != GeoRadius(-122.4194, 37.7749, 1, unit="m") +``` + +#### \_\_str_\_() + +Return the Redis Query string for the Geo filter + +* **Return type:** + str + +## GeoRadius + +### *class* GeoRadius(longitude, latitude, radius=1, unit='km') + +A GeoRadius is a GeoSpec representing a geographic radius. + +Create a GeoRadius specification (GeoSpec) + +* **Parameters:** + * **longitude** (*float*) – The longitude of the center of the radius. + * **latitude** (*float*) – The latitude of the center of the radius. + * **radius** (*int* *,* *optional*) – The radius of the circle. Defaults to 1. + * **unit** (*str* *,* *optional*) – The unit of the radius. Defaults to “km”. +* **Raises:** + **ValueError** – If the unit is not one of “m”, “km”, “mi”, or “ft”. + +#### \_\_init_\_(longitude, latitude, radius=1, unit='km') + +Create a GeoRadius specification (GeoSpec) + +* **Parameters:** + * **longitude** (*float*) – The longitude of the center of the radius. + * **latitude** (*float*) – The latitude of the center of the radius. + * **radius** (*int* *,* *optional*) – The radius of the circle. Defaults to 1. + * **unit** (*str* *,* *optional*) – The unit of the radius. Defaults to “km”. +* **Raises:** + **ValueError** – If the unit is not one of “m”, “km”, “mi”, or “ft”. diff --git a/content/integrate/redisvl/api/query.md b/content/integrate/redisvl/api/query.md new file mode 100644 index 0000000000..b866c890dd --- /dev/null +++ b/content/integrate/redisvl/api/query.md @@ -0,0 +1,920 @@ +--- +linkTitle: Query +title: Query +type: integration +--- + + +Query classes in RedisVL provide a structured way to define simple or complex +queries for different use cases. Each query class wraps the `redis-py` Query module +[https://github.com/redis/redis-py/blob/master/redis/commands/search/query.py](https://github.com/redis/redis-py/blob/master/redis/commands/search/query.py) with extended functionality for ease-of-use. + +## VectorQuery + +### *class* VectorQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False) + +Bases: `BaseVectorQuery`, `BaseQuery` + +A query for running a vector search along with an optional filter +expression. + +* **Parameters:** + * **vector** (*List* *[**float* *]*) – The vector to perform the vector search with. + * **vector_field_name** (*str*) – The name of the vector field to search + against in the database. + * **return_fields** (*List* *[**str* *]*) – The declared fields to return with search + results. + * **filter_expression** (*Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *,* *optional*) – A filter to apply + along with the vector search. Defaults to None. + * **dtype** (*str* *,* *optional*) – The dtype of the vector. Defaults to + “float32”. + * **num_results** (*int* *,* *optional*) – The top k results to return from the + vector search. Defaults to 10. + * **return_score** (*bool* *,* *optional*) – Whether to return the vector + distance. Defaults to True. + * **dialect** (*int* *,* *optional*) – The RediSearch query dialect. + Defaults to 2. + * **sort_by** (*Optional* *[**str* *]*) – The field to order the results by. Defaults + to None. Results will be ordered by vector distance. + * **in_order** (*bool*) – Requires the terms in the field to have + the same order as the terms in the query filter, regardless of + the offsets between them. Defaults to False. +* **Raises:** + **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression + +#### NOTE +Learn more about vector queries in Redis: [https://redis.io/docs/interact/search-and-query/search/vectors/#knn-search](https://redis.io/docs/interact/search-and-query/search/vectors/#knn-search) + +#### dialect(dialect) + +Add a dialect field to the query. + +- **dialect** - dialect version to execute the query under + +* **Parameters:** + **dialect** (*int*) +* **Return type:** + *Query* + +#### expander(expander) + +Add a expander field to the query. + +- **expander** - the name of the expander + +* **Parameters:** + **expander** (*str*) +* **Return type:** + *Query* + +#### in_order() + +Match only documents where the query terms appear in +the same order in the document. +i.e. for the query “hello world”, we do not match “world hello” + +* **Return type:** + *Query* + +#### language(language) + +Analyze the query as being in the specified language. + +* **Parameters:** + **language** (*str*) – The language (e.g. chinese or english) +* **Return type:** + *Query* + +#### limit_fields(\*fields) + +Limit the search to specific TEXT fields only. + +- **fields**: A list of strings, case sensitive field names + +from the defined schema. + +* **Parameters:** + **fields** (*List* *[**str* *]*) +* **Return type:** + *Query* + +#### limit_ids(\*ids) + +Limit the results to a specific set of pre-known document +ids of any length. + +* **Return type:** + *Query* + +#### no_content() + +Set the query to only return ids and not the document content. + +* **Return type:** + *Query* + +#### no_stopwords() + +Prevent the query from being filtered for stopwords. +Only useful in very big queries that you are certain contain +no stopwords. + +* **Return type:** + *Query* + +#### paging(offset, num) + +Set the paging for the query (defaults to 0..10). + +- **offset**: Paging offset for the results. Defaults to 0 +- **num**: How many results do we want + +* **Parameters:** + * **offset** (*int*) + * **num** (*int*) +* **Return type:** + *Query* + +#### query_string() + +Return the query string of this query only. + +* **Return type:** + str + +#### return_fields(\*fields) + +Add fields to return fields. + +* **Return type:** + *Query* + +#### scorer(scorer) + +Use a different scoring function to evaluate document relevance. +Default is TFIDF. + +* **Parameters:** + **scorer** (*str*) – The scoring function to use + (e.g. TFIDF.DOCNORM or BM25) +* **Return type:** + *Query* + +#### set_filter(filter_expression=None) + +Set the filter expression for the query. + +* **Parameters:** + **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]* *,* *optional*) – The filter + expression or query string to use on the query. +* **Raises:** + **TypeError** – If filter_expression is not a valid FilterExpression or string. + +#### slop(slop) + +Allow a maximum of N intervening non matched terms between +phrase terms (0 means exact phrase). + +* **Parameters:** + **slop** (*int*) +* **Return type:** + *Query* + +#### sort_by(field, asc=True) + +Add a sortby field to the query. + +- **field** - the name of the field to sort by +- **asc** - when True, sorting will be done in asceding order + +* **Parameters:** + * **field** (*str*) + * **asc** (*bool*) +* **Return type:** + *Query* + +#### timeout(timeout) + +overrides the timeout parameter of the module + +* **Parameters:** + **timeout** (*float*) +* **Return type:** + *Query* + +#### verbatim() + +Set the query to be verbatim, i.e. use no query expansion +or stemming. + +* **Return type:** + *Query* + +#### with_payloads() + +Ask the engine to return document payloads. + +* **Return type:** + *Query* + +#### with_scores() + +Ask the engine to return document search scores. + +* **Return type:** + *Query* + +#### *property* filter *: str | [FilterExpression](filter.md#redisvl.query.filter.FilterExpression)* + +The filter expression for the query. + +#### *property* params *: Dict[str, Any]* + +Return the parameters for the query. + +* **Returns:** + The parameters for the query. +* **Return type:** + Dict[str, Any] + +#### *property* query *: BaseQuery* + +Return self as the query object. + +## VectorRangeQuery + +### *class* VectorRangeQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', distance_threshold=0.2, num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False) + +Bases: `BaseVectorQuery`, `BaseQuery` + +A query for running a filtered vector search based on semantic +distance threshold. + +* **Parameters:** + * **vector** (*List* *[**float* *]*) – The vector to perform the range query with. + * **vector_field_name** (*str*) – The name of the vector field to search + against in the database. + * **return_fields** (*List* *[**str* *]*) – The declared fields to return with search + results. + * **filter_expression** (*Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *,* *optional*) – A filter to apply + along with the range query. Defaults to None. + * **dtype** (*str* *,* *optional*) – The dtype of the vector. Defaults to + “float32”. + * **distance_threshold** (*str* *,* *float*) – The threshold for vector distance. + A smaller threshold indicates a stricter semantic search. + Defaults to 0.2. + * **num_results** (*int*) – The MAX number of results to return. + Defaults to 10. + * **return_score** (*bool* *,* *optional*) – Whether to return the vector + distance. Defaults to True. + * **dialect** (*int* *,* *optional*) – The RediSearch query dialect. + Defaults to 2. + * **sort_by** (*Optional* *[**str* *]*) – The field to order the results by. Defaults + to None. Results will be ordered by vector distance. + * **in_order** (*bool*) – Requires the terms in the field to have + the same order as the terms in the query filter, regardless of + the offsets between them. Defaults to False. +* **Raises:** + **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression + +#### NOTE +Learn more about vector range queries: [https://redis.io/docs/interact/search-and-query/search/vectors/#range-query](https://redis.io/docs/interact/search-and-query/search/vectors/#range-query) + +#### dialect(dialect) + +Add a dialect field to the query. + +- **dialect** - dialect version to execute the query under + +* **Parameters:** + **dialect** (*int*) +* **Return type:** + *Query* + +#### expander(expander) + +Add a expander field to the query. + +- **expander** - the name of the expander + +* **Parameters:** + **expander** (*str*) +* **Return type:** + *Query* + +#### in_order() + +Match only documents where the query terms appear in +the same order in the document. +i.e. for the query “hello world”, we do not match “world hello” + +* **Return type:** + *Query* + +#### language(language) + +Analyze the query as being in the specified language. + +* **Parameters:** + **language** (*str*) – The language (e.g. chinese or english) +* **Return type:** + *Query* + +#### limit_fields(\*fields) + +Limit the search to specific TEXT fields only. + +- **fields**: A list of strings, case sensitive field names + +from the defined schema. + +* **Parameters:** + **fields** (*List* *[**str* *]*) +* **Return type:** + *Query* + +#### limit_ids(\*ids) + +Limit the results to a specific set of pre-known document +ids of any length. + +* **Return type:** + *Query* + +#### no_content() + +Set the query to only return ids and not the document content. + +* **Return type:** + *Query* + +#### no_stopwords() + +Prevent the query from being filtered for stopwords. +Only useful in very big queries that you are certain contain +no stopwords. + +* **Return type:** + *Query* + +#### paging(offset, num) + +Set the paging for the query (defaults to 0..10). + +- **offset**: Paging offset for the results. Defaults to 0 +- **num**: How many results do we want + +* **Parameters:** + * **offset** (*int*) + * **num** (*int*) +* **Return type:** + *Query* + +#### query_string() + +Return the query string of this query only. + +* **Return type:** + str + +#### return_fields(\*fields) + +Add fields to return fields. + +* **Return type:** + *Query* + +#### scorer(scorer) + +Use a different scoring function to evaluate document relevance. +Default is TFIDF. + +* **Parameters:** + **scorer** (*str*) – The scoring function to use + (e.g. TFIDF.DOCNORM or BM25) +* **Return type:** + *Query* + +#### set_distance_threshold(distance_threshold) + +Set the distance threshold for the query. + +* **Parameters:** + **distance_threshold** (*float*) – vector distance + +#### set_filter(filter_expression=None) + +Set the filter expression for the query. + +* **Parameters:** + **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]* *,* *optional*) – The filter + expression or query string to use on the query. +* **Raises:** + **TypeError** – If filter_expression is not a valid FilterExpression or string. + +#### slop(slop) + +Allow a maximum of N intervening non matched terms between +phrase terms (0 means exact phrase). + +* **Parameters:** + **slop** (*int*) +* **Return type:** + *Query* + +#### sort_by(field, asc=True) + +Add a sortby field to the query. + +- **field** - the name of the field to sort by +- **asc** - when True, sorting will be done in asceding order + +* **Parameters:** + * **field** (*str*) + * **asc** (*bool*) +* **Return type:** + *Query* + +#### timeout(timeout) + +overrides the timeout parameter of the module + +* **Parameters:** + **timeout** (*float*) +* **Return type:** + *Query* + +#### verbatim() + +Set the query to be verbatim, i.e. use no query expansion +or stemming. + +* **Return type:** + *Query* + +#### with_payloads() + +Ask the engine to return document payloads. + +* **Return type:** + *Query* + +#### with_scores() + +Ask the engine to return document search scores. + +* **Return type:** + *Query* + +#### *property* distance_threshold *: float* + +Return the distance threshold for the query. + +* **Returns:** + The distance threshold for the query. +* **Return type:** + float + +#### *property* filter *: str | [FilterExpression](filter.md#redisvl.query.filter.FilterExpression)* + +The filter expression for the query. + +#### *property* params *: Dict[str, Any]* + +Return the parameters for the query. + +* **Returns:** + The parameters for the query. +* **Return type:** + Dict[str, Any] + +#### *property* query *: BaseQuery* + +Return self as the query object. + +## FilterQuery + +### *class* FilterQuery(filter_expression=None, return_fields=None, num_results=10, dialect=2, sort_by=None, in_order=False, params=None) + +Bases: `BaseQuery` + +A query for running a filtered search with a filter expression. + +* **Parameters:** + * **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]*) – The optional filter + expression to query with. Defaults to ‘\*’. + * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to return. + * **num_results** (*Optional* *[**int* *]* *,* *optional*) – The number of results to return. Defaults to 10. + * **dialect** (*int* *,* *optional*) – The query dialect. Defaults to 2. + * **sort_by** (*Optional* *[**str* *]* *,* *optional*) – The field to order the results by. Defaults to None. + * **in_order** (*bool* *,* *optional*) – Requires the terms in the field to have the same order as the terms in the query filter. Defaults to False. + * **params** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The parameters for the query. Defaults to None. +* **Raises:** + **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression + +#### dialect(dialect) + +Add a dialect field to the query. + +- **dialect** - dialect version to execute the query under + +* **Parameters:** + **dialect** (*int*) +* **Return type:** + *Query* + +#### expander(expander) + +Add a expander field to the query. + +- **expander** - the name of the expander + +* **Parameters:** + **expander** (*str*) +* **Return type:** + *Query* + +#### in_order() + +Match only documents where the query terms appear in +the same order in the document. +i.e. for the query “hello world”, we do not match “world hello” + +* **Return type:** + *Query* + +#### language(language) + +Analyze the query as being in the specified language. + +* **Parameters:** + **language** (*str*) – The language (e.g. chinese or english) +* **Return type:** + *Query* + +#### limit_fields(\*fields) + +Limit the search to specific TEXT fields only. + +- **fields**: A list of strings, case sensitive field names + +from the defined schema. + +* **Parameters:** + **fields** (*List* *[**str* *]*) +* **Return type:** + *Query* + +#### limit_ids(\*ids) + +Limit the results to a specific set of pre-known document +ids of any length. + +* **Return type:** + *Query* + +#### no_content() + +Set the query to only return ids and not the document content. + +* **Return type:** + *Query* + +#### no_stopwords() + +Prevent the query from being filtered for stopwords. +Only useful in very big queries that you are certain contain +no stopwords. + +* **Return type:** + *Query* + +#### paging(offset, num) + +Set the paging for the query (defaults to 0..10). + +- **offset**: Paging offset for the results. Defaults to 0 +- **num**: How many results do we want + +* **Parameters:** + * **offset** (*int*) + * **num** (*int*) +* **Return type:** + *Query* + +#### query_string() + +Return the query string of this query only. + +* **Return type:** + str + +#### return_fields(\*fields) + +Add fields to return fields. + +* **Return type:** + *Query* + +#### scorer(scorer) + +Use a different scoring function to evaluate document relevance. +Default is TFIDF. + +* **Parameters:** + **scorer** (*str*) – The scoring function to use + (e.g. TFIDF.DOCNORM or BM25) +* **Return type:** + *Query* + +#### set_filter(filter_expression=None) + +Set the filter expression for the query. + +* **Parameters:** + **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]* *,* *optional*) – The filter + expression or query string to use on the query. +* **Raises:** + **TypeError** – If filter_expression is not a valid FilterExpression or string. + +#### slop(slop) + +Allow a maximum of N intervening non matched terms between +phrase terms (0 means exact phrase). + +* **Parameters:** + **slop** (*int*) +* **Return type:** + *Query* + +#### sort_by(field, asc=True) + +Add a sortby field to the query. + +- **field** - the name of the field to sort by +- **asc** - when True, sorting will be done in asceding order + +* **Parameters:** + * **field** (*str*) + * **asc** (*bool*) +* **Return type:** + *Query* + +#### timeout(timeout) + +overrides the timeout parameter of the module + +* **Parameters:** + **timeout** (*float*) +* **Return type:** + *Query* + +#### verbatim() + +Set the query to be verbatim, i.e. use no query expansion +or stemming. + +* **Return type:** + *Query* + +#### with_payloads() + +Ask the engine to return document payloads. + +* **Return type:** + *Query* + +#### with_scores() + +Ask the engine to return document search scores. + +* **Return type:** + *Query* + +#### *property* filter *: str | [FilterExpression](filter.md#redisvl.query.filter.FilterExpression)* + +The filter expression for the query. + +#### *property* params *: Dict[str, Any]* + +Return the query parameters. + +#### *property* query *: BaseQuery* + +Return self as the query object. + +## CountQuery + +### *class* CountQuery(filter_expression=None, dialect=2, params=None) + +Bases: `BaseQuery` + +A query for a simple count operation provided some filter expression. + +* **Parameters:** + * **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]*) – The filter expression to query with. Defaults to None. + * **params** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The parameters for the query. Defaults to None. + * **dialect** (*int*) +* **Raises:** + **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression + +```python +from redisvl.query import CountQuery +from redisvl.query.filter import Tag + +t = Tag("brand") == "Nike" +query = CountQuery(filter_expression=t) + +count = index.query(query) +``` + +#### dialect(dialect) + +Add a dialect field to the query. + +- **dialect** - dialect version to execute the query under + +* **Parameters:** + **dialect** (*int*) +* **Return type:** + *Query* + +#### expander(expander) + +Add a expander field to the query. + +- **expander** - the name of the expander + +* **Parameters:** + **expander** (*str*) +* **Return type:** + *Query* + +#### in_order() + +Match only documents where the query terms appear in +the same order in the document. +i.e. for the query “hello world”, we do not match “world hello” + +* **Return type:** + *Query* + +#### language(language) + +Analyze the query as being in the specified language. + +* **Parameters:** + **language** (*str*) – The language (e.g. chinese or english) +* **Return type:** + *Query* + +#### limit_fields(\*fields) + +Limit the search to specific TEXT fields only. + +- **fields**: A list of strings, case sensitive field names + +from the defined schema. + +* **Parameters:** + **fields** (*List* *[**str* *]*) +* **Return type:** + *Query* + +#### limit_ids(\*ids) + +Limit the results to a specific set of pre-known document +ids of any length. + +* **Return type:** + *Query* + +#### no_content() + +Set the query to only return ids and not the document content. + +* **Return type:** + *Query* + +#### no_stopwords() + +Prevent the query from being filtered for stopwords. +Only useful in very big queries that you are certain contain +no stopwords. + +* **Return type:** + *Query* + +#### paging(offset, num) + +Set the paging for the query (defaults to 0..10). + +- **offset**: Paging offset for the results. Defaults to 0 +- **num**: How many results do we want + +* **Parameters:** + * **offset** (*int*) + * **num** (*int*) +* **Return type:** + *Query* + +#### query_string() + +Return the query string of this query only. + +* **Return type:** + str + +#### return_fields(\*fields) + +Add fields to return fields. + +* **Return type:** + *Query* + +#### scorer(scorer) + +Use a different scoring function to evaluate document relevance. +Default is TFIDF. + +* **Parameters:** + **scorer** (*str*) – The scoring function to use + (e.g. TFIDF.DOCNORM or BM25) +* **Return type:** + *Query* + +#### set_filter(filter_expression=None) + +Set the filter expression for the query. + +* **Parameters:** + **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]* *,* *optional*) – The filter + expression or query string to use on the query. +* **Raises:** + **TypeError** – If filter_expression is not a valid FilterExpression or string. + +#### slop(slop) + +Allow a maximum of N intervening non matched terms between +phrase terms (0 means exact phrase). + +* **Parameters:** + **slop** (*int*) +* **Return type:** + *Query* + +#### sort_by(field, asc=True) + +Add a sortby field to the query. + +- **field** - the name of the field to sort by +- **asc** - when True, sorting will be done in asceding order + +* **Parameters:** + * **field** (*str*) + * **asc** (*bool*) +* **Return type:** + *Query* + +#### timeout(timeout) + +overrides the timeout parameter of the module + +* **Parameters:** + **timeout** (*float*) +* **Return type:** + *Query* + +#### verbatim() + +Set the query to be verbatim, i.e. use no query expansion +or stemming. + +* **Return type:** + *Query* + +#### with_payloads() + +Ask the engine to return document payloads. + +* **Return type:** + *Query* + +#### with_scores() + +Ask the engine to return document search scores. + +* **Return type:** + *Query* + +#### *property* filter *: str | [FilterExpression](filter.md#redisvl.query.filter.FilterExpression)* + +The filter expression for the query. + +#### *property* params *: Dict[str, Any]* + +Return the query parameters. + +#### *property* query *: BaseQuery* + +Return self as the query object. diff --git a/content/integrate/redisvl/api/reranker.md b/content/integrate/redisvl/api/reranker.md new file mode 100644 index 0000000000..235434877a --- /dev/null +++ b/content/integrate/redisvl/api/reranker.md @@ -0,0 +1,301 @@ +--- +linkTitle: Rerankers +title: Rerankers +type: integration +--- + + +## CohereReranker + + + +### *class* CohereReranker(model='rerank-english-v3.0', rank_by=None, limit=5, return_score=True, api_config=None) + +Bases: `BaseReranker` + +The CohereReranker class uses Cohere’s API to rerank documents based on an +input query. + +This reranker is designed to interact with Cohere’s /rerank API, +requiring an API key for authentication. The key can be provided +directly in the api_config dictionary or through the COHERE_API_KEY +environment variable. User must obtain an API key from Cohere’s website +([https://dashboard.cohere.com/](https://dashboard.cohere.com/)). Additionally, the cohere python +client must be installed with pip install cohere. + +```python +from redisvl.utils.rerank import CohereReranker + +# set up the Cohere reranker with some configuration +reranker = CohereReranker(rank_by=["content"], limit=2) +# rerank raw search results based on user input/query +results = reranker.rank( + query="your input query text here", + docs=[ + {"content": "document 1"}, + {"content": "document 2"}, + {"content": "document 3"} + ] +) +``` + +Initialize the CohereReranker with specified model, ranking criteria, +and API configuration. + +* **Parameters:** + * **model** (*str*) – The identifier for the Cohere model used for reranking. + Defaults to ‘rerank-english-v3.0’. + * **rank_by** (*Optional* *[**List* *[**str* *]* *]*) – Optional list of keys specifying the + attributes in the documents that should be considered for + ranking. None means ranking will rely on the model’s default + behavior. + * **limit** (*int*) – The maximum number of results to return after + reranking. Must be a positive integer. + * **return_score** (*bool*) – Whether to return scores alongside the + reranked results. + * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the API key. + Defaults to None. +* **Raises:** + * **ImportError** – If the cohere library is not installed. + * **ValueError** – If the API key is not provided. + +#### *async* arank(query, docs, \*\*kwargs) + +Rerank documents based on the provided query using the Cohere rerank API. + +This method processes the user’s query and the provided documents to +rerank them in a manner that is potentially more relevant to the +query’s context. + +* **Parameters:** + * **query** (*str*) – The user’s search query. + * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents + to be ranked, either as dictionaries or strings. +* **Returns:** + The reranked list of documents and optionally associated scores. +* **Return type:** + Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### rank(query, docs, \*\*kwargs) + +Rerank documents based on the provided query using the Cohere rerank API. + +This method processes the user’s query and the provided documents to +rerank them in a manner that is potentially more relevant to the +query’s context. + +* **Parameters:** + * **query** (*str*) – The user’s search query. + * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents + to be ranked, either as dictionaries or strings. +* **Returns:** + The reranked list of documents and optionally associated scores. +* **Return type:** + Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## HFCrossEncoderReranker + + + +### *class* HFCrossEncoderReranker(model='cross-encoder/ms-marco-MiniLM-L-6-v2', limit=3, return_score=True, \*, rank_by=None) + +Bases: `BaseReranker` + +The HFCrossEncoderReranker class uses a cross-encoder models from Hugging Face +to rerank documents based on an input query. + +This reranker loads a cross-encoder model using the CrossEncoder class +from the sentence_transformers library. It requires the +sentence_transformers library to be installed. + +```python +from redisvl.utils.rerank import HFCrossEncoderReranker + +# set up the HFCrossEncoderReranker with a specific model +reranker = HFCrossEncoderReranker(model_name="cross-encoder/ms-marco-MiniLM-L-6-v2", limit=3) +# rerank raw search results based on user input/query +results = reranker.rank( + query="your input query text here", + docs=[ + {"content": "document 1"}, + {"content": "document 2"}, + {"content": "document 3"} + ] +) +``` + +Initialize the HFCrossEncoderReranker with a specified model and ranking criteria. + +* **Parameters:** + * **model** (*str*) – The name or path of the cross-encoder model to use for reranking. + Defaults to ‘cross-encoder/ms-marco-MiniLM-L-6-v2’. + * **limit** (*int*) – The maximum number of results to return after reranking. Must be a positive integer. + * **return_score** (*bool*) – Whether to return scores alongside the reranked results. + * **rank_by** (*List* *[**str* *]* *|* *None*) + +#### *async* arank(query, docs, \*\*kwargs) + +Asynchronously rerank documents based on the provided query using the loaded cross-encoder model. + +This method processes the user’s query and the provided documents to rerank them +in a manner that is potentially more relevant to the query’s context. + +* **Parameters:** + * **query** (*str*) – The user’s search query. + * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents to be ranked, + either as dictionaries or strings. +* **Returns:** + The reranked list of documents and optionally associated scores. +* **Return type:** + Union[Tuple[List[Dict[str, Any]], List[float]], List[Dict[str, Any]]] + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### rank(query, docs, \*\*kwargs) + +Rerank documents based on the provided query using the loaded cross-encoder model. + +This method processes the user’s query and the provided documents to rerank them +in a manner that is potentially more relevant to the query’s context. + +* **Parameters:** + * **query** (*str*) – The user’s search query. + * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents to be ranked, + either as dictionaries or strings. +* **Returns:** + The reranked list of documents and optionally associated scores. +* **Return type:** + Union[Tuple[List[Dict[str, Any]], List[float]], List[Dict[str, Any]]] + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## VoyageAIReranker + + + +### *class* VoyageAIReranker(model, rank_by=None, limit=5, return_score=True, api_config=None) + +Bases: `BaseReranker` + +The VoyageAIReranker class uses VoyageAI’s API to rerank documents based on an +input query. + +This reranker is designed to interact with VoyageAI’s /rerank API, +requiring an API key for authentication. The key can be provided +directly in the api_config dictionary or through the VOYAGE_API_KEY +environment variable. User must obtain an API key from VoyageAI’s website +([https://dash.voyageai.com/](https://dash.voyageai.com/)). Additionally, the voyageai python +client must be installed with pip install voyageai. + +```python +from redisvl.utils.rerank import VoyageAIReranker + +# set up the VoyageAI reranker with some configuration +reranker = VoyageAIReranker(rank_by=["content"], limit=2) +# rerank raw search results based on user input/query +results = reranker.rank( + query="your input query text here", + docs=[ + {"content": "document 1"}, + {"content": "document 2"}, + {"content": "document 3"} + ] +) +``` + +Initialize the VoyageAIReranker with specified model, ranking criteria, +and API configuration. + +* **Parameters:** + * **model** (*str*) – The identifier for the VoyageAI model used for reranking. + * **rank_by** (*Optional* *[**List* *[**str* *]* *]*) – Optional list of keys specifying the + attributes in the documents that should be considered for + ranking. None means ranking will rely on the model’s default + behavior. + * **limit** (*int*) – The maximum number of results to return after + reranking. Must be a positive integer. + * **return_score** (*bool*) – Whether to return scores alongside the + reranked results. + * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the API key. + Defaults to None. +* **Raises:** + * **ImportError** – If the voyageai library is not installed. + * **ValueError** – If the API key is not provided. + +#### *async* arank(query, docs, \*\*kwargs) + +Rerank documents based on the provided query using the VoyageAI rerank API. + +This method processes the user’s query and the provided documents to +rerank them in a manner that is potentially more relevant to the +query’s context. + +* **Parameters:** + * **query** (*str*) – The user’s search query. + * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents + to be ranked, either as dictionaries or strings. +* **Returns:** + The reranked list of documents and optionally associated scores. +* **Return type:** + Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### rank(query, docs, \*\*kwargs) + +Rerank documents based on the provided query using the VoyageAI rerank API. + +This method processes the user’s query and the provided documents to +rerank them in a manner that is potentially more relevant to the +query’s context. + +* **Parameters:** + * **query** (*str*) – The user’s search query. + * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents + to be ranked, either as dictionaries or strings. +* **Returns:** + The reranked list of documents and optionally associated scores. +* **Return type:** + Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. diff --git a/content/integrate/redisvl/api/router.md b/content/integrate/redisvl/api/router.md new file mode 100644 index 0000000000..6ee03e5204 --- /dev/null +++ b/content/integrate/redisvl/api/router.md @@ -0,0 +1,323 @@ +--- +linkTitle: Semantic Router +title: Semantic Router +type: integration +--- + + + + +## Semantic Router + +### *class* SemanticRouter(name, routes, vectorizer=None, routing_config=None, redis_client=None, redis_url='redis://localhost:6379', overwrite=False, connection_kwargs={}) + +Semantic Router for managing and querying route vectors. + +Initialize the SemanticRouter. + +* **Parameters:** + * **name** (*str*) – The name of the semantic router. + * **routes** (*List* *[*[*Route*](#redisvl.extensions.router.Route) *]*) – List of Route objects. + * **vectorizer** (*BaseVectorizer* *,* *optional*) – The vectorizer used to embed route references. Defaults to default HFTextVectorizer. + * **routing_config** ([*RoutingConfig*](#redisvl.extensions.router.RoutingConfig) *,* *optional*) – Configuration for routing behavior. Defaults to the default RoutingConfig. + * **redis_client** (*Optional* *[**Redis* *]* *,* *optional*) – Redis client for connection. Defaults to None. + * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. + * **overwrite** (*bool* *,* *optional*) – Whether to overwrite existing index. Defaults to False. + * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments + for the redis client. Defaults to empty {}. + +#### clear() + +Flush all routes from the semantic router index. + +* **Return type:** + None + +#### delete() + +Delete the semantic router index. + +* **Return type:** + None + +#### *classmethod* from_dict(data, \*\*kwargs) + +Create a SemanticRouter from a dictionary. + +* **Parameters:** + **data** (*Dict* *[**str* *,* *Any* *]*) – The dictionary containing the semantic router data. +* **Returns:** + The semantic router instance. +* **Return type:** + [SemanticRouter](#redisvl.extensions.router.SemanticRouter) +* **Raises:** + **ValueError** – If required data is missing or invalid. + +```python +from redisvl.extensions.router import SemanticRouter +router_data = { + "name": "example_router", + "routes": [{"name": "route1", "references": ["ref1"], "distance_threshold": 0.5}], + "vectorizer": {"type": "openai", "model": "text-embedding-ada-002"}, +} +router = SemanticRouter.from_dict(router_data) +``` + +#### *classmethod* from_yaml(file_path, \*\*kwargs) + +Create a SemanticRouter from a YAML file. + +* **Parameters:** + **file_path** (*str*) – The path to the YAML file. +* **Returns:** + The semantic router instance. +* **Return type:** + [SemanticRouter](#redisvl.extensions.router.SemanticRouter) +* **Raises:** + * **ValueError** – If the file path is invalid. + * **FileNotFoundError** – If the file does not exist. + +```python +from redisvl.extensions.router import SemanticRouter +router = SemanticRouter.from_yaml("router.yaml", redis_url="redis://localhost:6379") +``` + +#### get(route_name) + +Get a route by its name. + +* **Parameters:** + **route_name** (*str*) – Name of the route. +* **Returns:** + The selected Route object or None if not found. +* **Return type:** + Optional[[Route](#redisvl.extensions.router.Route)] + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### remove_route(route_name) + +Remove a route and all references from the semantic router. + +* **Parameters:** + **route_name** (*str*) – Name of the route to remove. +* **Return type:** + None + +#### route_many(statement=None, vector=None, max_k=None, distance_threshold=None, aggregation_method=None) + +Query the semantic router with a given statement or vector for multiple matches. + +* **Parameters:** + * **statement** (*Optional* *[**str* *]*) – The input statement to be queried. + * **vector** (*Optional* *[**List* *[**float* *]* *]*) – The input vector to be queried. + * **max_k** (*Optional* *[**int* *]*) – The maximum number of top matches to return. + * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic distance. + * **aggregation_method** (*Optional* *[*[*DistanceAggregationMethod*](#redisvl.extensions.router.schema.DistanceAggregationMethod) *]*) – The aggregation method used for vector distances. +* **Returns:** + The matching routes and their details. +* **Return type:** + List[[RouteMatch](#redisvl.extensions.router.schema.RouteMatch)] + +#### to_dict() + +Convert the SemanticRouter instance to a dictionary. + +* **Returns:** + The dictionary representation of the SemanticRouter. +* **Return type:** + Dict[str, Any] + +```python +from redisvl.extensions.router import SemanticRouter +router = SemanticRouter(name="example_router", routes=[], redis_url="redis://localhost:6379") +router_dict = router.to_dict() +``` + +#### to_yaml(file_path, overwrite=True) + +Write the semantic router to a YAML file. + +* **Parameters:** + * **file_path** (*str*) – The path to the YAML file. + * **overwrite** (*bool*) – Whether to overwrite the file if it already exists. +* **Raises:** + **FileExistsError** – If the file already exists and overwrite is False. +* **Return type:** + None + +```python +from redisvl.extensions.router import SemanticRouter +router = SemanticRouter( + name="example_router", + routes=[], + redis_url="redis://localhost:6379" +) +router.to_yaml("router.yaml") +``` + +#### update_routing_config(routing_config) + +Update the routing configuration. + +* **Parameters:** + **routing_config** ([*RoutingConfig*](#redisvl.extensions.router.RoutingConfig)) – The new routing configuration. + +#### model_config *: ClassVar[ConfigDict]* *= {'arbitrary_types_allowed': True}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +#### name *: str* + +The name of the semantic router. + +#### *property* route_names *: List[str]* + +Get the list of route names. + +* **Returns:** + List of route names. +* **Return type:** + List[str] + +#### *property* route_thresholds *: Dict[str, float | None]* + +Get the distance thresholds for each route. + +* **Returns:** + Dictionary of route names and their distance thresholds. +* **Return type:** + Dict[str, float] + +#### routes *: List[[Route](#redisvl.extensions.router.Route)]* + +List of Route objects. + +#### routing_config *: [RoutingConfig](#redisvl.extensions.router.RoutingConfig)* + +Configuration for routing behavior. + +#### vectorizer *: BaseVectorizer* + +The vectorizer used to embed route references. + +## Routing Config + +### *class* RoutingConfig(\*, max_k=1, aggregation_method=DistanceAggregationMethod.avg) + +Configuration for routing behavior. + +Create a new model by parsing and validating input data from keyword arguments. + +Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be +validated to form a valid model. + +self is explicitly positional-only to allow self as a field name. + +* **Parameters:** + * **max_k** (*Annotated* *[**int* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=False* *,* *default=1* *,* *metadata=* *[**Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *]* *)* *]*) + * **aggregation_method** ([*DistanceAggregationMethod*](#redisvl.extensions.router.schema.DistanceAggregationMethod)) + +#### max_k *: Annotated[int, FieldInfo(annotation=NoneType, required=False, default=1, metadata=[Strict(strict=True), Gt(gt=0)])]* + +Aggregation method to use to classify queries. + +#### model_config *: ClassVar[ConfigDict]* *= {'extra': 'ignore'}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## Route + +### *class* Route(\*, name, references, metadata={}, distance_threshold=0.5) + +Model representing a routing path with associated metadata and thresholds. + +Create a new model by parsing and validating input data from keyword arguments. + +Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be +validated to form a valid model. + +self is explicitly positional-only to allow self as a field name. + +* **Parameters:** + * **name** (*str*) + * **references** (*List* *[**str* *]*) + * **metadata** (*Dict* *[**str* *,* *Any* *]*) + * **distance_threshold** (*Annotated* *[**float* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=True* *,* *metadata=* *[**Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *,* *Le* *(**le=1* *)* *]* *)* *]*) + +#### distance_threshold *: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Strict(strict=True), Gt(gt=0), Le(le=1)])]* + +Distance threshold for matching the route. + +#### metadata *: Dict[str, Any]* + +Metadata associated with the route. + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +#### name *: str* + +The name of the route. + +#### references *: List[str]* + +List of reference phrases for the route. + +## Route Match + +### *class* RouteMatch(\*, name=None, distance=None) + +Model representing a matched route with distance information. + +Create a new model by parsing and validating input data from keyword arguments. + +Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be +validated to form a valid model. + +self is explicitly positional-only to allow self as a field name. + +* **Parameters:** + * **name** (*str* *|* *None*) + * **distance** (*float* *|* *None*) + +#### distance *: float | None* + +The vector distance between the statement and the matched route. + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +#### name *: str | None* + +The matched route name. + +## Distance Aggregation Method + +### *class* DistanceAggregationMethod(value, names=, \*values, module=None, qualname=None, type=None, start=1, boundary=None) + +Enumeration for distance aggregation methods. + +#### avg *= 'avg'* + +Compute the average of the vector distances. + +#### min *= 'min'* + +Compute the minimum of the vector distances. + +#### sum *= 'sum'* + +Compute the sum of the vector distances. diff --git a/content/integrate/redisvl/api/schema.md b/content/integrate/redisvl/api/schema.md new file mode 100644 index 0000000000..dd52f67269 --- /dev/null +++ b/content/integrate/redisvl/api/schema.md @@ -0,0 +1,343 @@ +--- +linkTitle: Schema +title: Schema +type: integration +--- + + +Schema in RedisVL provides a structured format to define index settings and +field configurations using the following three components: + +| Component | Description | +|-------------|------------------------------------------------------------------------------------| +| version | The version of the schema spec. Current supported version is 0.1.0. | +| index | Index specific settings like name, key prefix, key separator, and storage type. | +| fields | Subset of fields within your data to include in the index and any custom settings. | + +## IndexSchema + + + +### *class* IndexSchema(\*, index, fields={}, version='0.1.0') + +A schema definition for a search index in Redis, used in RedisVL for +configuring index settings and organizing vector and metadata fields. + +The class offers methods to create an index schema from a YAML file or a +Python dictionary, supporting flexible schema definitions and easy +integration into various workflows. + +An example schema.yaml file might look like this: + +```yaml +version: '0.1.0' + +index: + name: user-index + prefix: user + key_separator: ":" + storage_type: json + +fields: + - name: user + type: tag + - name: credit_score + type: tag + - name: embedding + type: vector + attrs: + algorithm: flat + dims: 3 + distance_metric: cosine + datatype: float32 +``` + +Loading the schema for RedisVL from yaml is as simple as: + +```python +from redisvl.schema import IndexSchema + +schema = IndexSchema.from_yaml("schema.yaml") +``` + +Loading the schema for RedisVL from dict is as simple as: + +```python +from redisvl.schema import IndexSchema + +schema = IndexSchema.from_dict({ + "index": { + "name": "user-index", + "prefix": "user", + "key_separator": ":", + "storage_type": "json", + }, + "fields": [ + {"name": "user", "type": "tag"}, + {"name": "credit_score", "type": "tag"}, + { + "name": "embedding", + "type": "vector", + "attrs": { + "algorithm": "flat", + "dims": 3, + "distance_metric": "cosine", + "datatype": "float32" + } + } + ] +}) +``` + +#### NOTE +The fields attribute in the schema must contain unique field names to ensure +correct and unambiguous field references. + +Create a new model by parsing and validating input data from keyword arguments. + +Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be +validated to form a valid model. + +self is explicitly positional-only to allow self as a field name. + +* **Parameters:** + * **index** (*IndexInfo*) + * **fields** (*Dict* *[**str* *,* *BaseField* *]*) + * **version** (*Literal* *[* *'0.1.0'* *]*) + +#### add_field(field_inputs) + +Adds a single field to the index schema based on the specified field +type and attributes. + +This method allows for the addition of individual fields to the schema, +providing flexibility in defining the structure of the index. + +* **Parameters:** + **field_inputs** (*Dict* *[**str* *,* *Any* *]*) – A field to add. +* **Raises:** + **ValueError** – If the field name or type are not provided or if the name + already exists within the schema. + +```python +# Add a tag field +schema.add_field({"name": "user", "type": "tag}) + +# Add a vector field +schema.add_field({ + "name": "user-embedding", + "type": "vector", + "attrs": { + "dims": 1024, + "algorithm": "flat", + "datatype": "float32" + } +}) +``` + +#### add_fields(fields) + +Extends the schema with additional fields. + +This method allows dynamically adding new fields to the index schema. It +processes a list of field definitions. + +* **Parameters:** + **fields** (*List* *[**Dict* *[**str* *,* *Any* *]* *]*) – A list of fields to add. +* **Raises:** + **ValueError** – If a field with the same name already exists in the + schema. + +```python +schema.add_fields([ + {"name": "user", "type": "tag"}, + {"name": "bio", "type": "text"}, + { + "name": "user-embedding", + "type": "vector", + "attrs": { + "dims": 1024, + "algorithm": "flat", + "datatype": "float32" + } + } +]) +``` + +#### *classmethod* from_dict(data) + +Create an IndexSchema from a dictionary. + +* **Parameters:** + **data** (*Dict* *[**str* *,* *Any* *]*) – The index schema data. +* **Returns:** + The index schema. +* **Return type:** + [IndexSchema](#redisvl.schema.IndexSchema) + +```python +from redisvl.schema import IndexSchema + +schema = IndexSchema.from_dict({ + "index": { + "name": "docs-index", + "prefix": "docs", + "storage_type": "hash", + }, + "fields": [ + { + "name": "doc-id", + "type": "tag" + }, + { + "name": "doc-embedding", + "type": "vector", + "attrs": { + "algorithm": "flat", + "dims": 1536 + } + } + ] +}) +``` + +#### *classmethod* from_yaml(file_path) + +Create an IndexSchema from a YAML file. + +* **Parameters:** + **file_path** (*str*) – The path to the YAML file. +* **Returns:** + The index schema. +* **Return type:** + [IndexSchema](#redisvl.schema.IndexSchema) + +```python +from redisvl.schema import IndexSchema +schema = IndexSchema.from_yaml("schema.yaml") +``` + +#### remove_field(field_name) + +Removes a field from the schema based on the specified name. + +This method is useful for dynamically altering the schema by removing +existing fields. + +* **Parameters:** + **field_name** (*str*) – The name of the field to be removed. + +#### to_dict() + +Serialize the index schema model to a dictionary, handling Enums +and other special cases properly. + +* **Returns:** + The index schema as a dictionary. +* **Return type:** + Dict[str, Any] + +#### to_yaml(file_path, overwrite=True) + +Write the index schema to a YAML file. + +* **Parameters:** + * **file_path** (*str*) – The path to the YAML file. + * **overwrite** (*bool*) – Whether to overwrite the file if it already exists. +* **Raises:** + **FileExistsError** – If the file already exists and overwrite is False. +* **Return type:** + None + +#### *property* field_names *: List[str]* + +A list of field names associated with the index schema. + +* **Returns:** + A list of field names from the schema. +* **Return type:** + List[str] + +#### fields *: Dict[str, BaseField]* + +Fields associated with the search index and their properties + +#### index *: IndexInfo* + +Details of the basic index configurations. + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +#### version *: Literal['0.1.0']* + +Version of the underlying index schema. + +## Defining Fields + +Fields in the schema can be defined in YAML format or as a Python dictionary, specifying a name, type, an optional path, and attributes for customization. + +**YAML Example**: + +```yaml +- name: title + type: text + path: $.document.title + attrs: + weight: 1.0 + no_stem: false + withsuffixtrie: true +``` + +**Python Dictionary Example**: + +```python +{ + "name": "location", + "type": "geo", + "attrs": { + "sortable": true + } +} +``` + +## Supported Field Types and Attributes + +Each field type supports specific attributes that customize its behavior. Below are the field types and their available attributes: + +**Text Field Attributes**: + +- weight: Importance of the field in result calculation. +- no_stem: Disables stemming during indexing. +- withsuffixtrie: Optimizes queries by maintaining a suffix trie. +- phonetic_matcher: Enables phonetic matching. +- sortable: Allows sorting on this field. + +**Tag Field Attributes**: + +- separator: Character for splitting text into individual tags. +- case_sensitive: Case sensitivity in tag matching. +- withsuffixtrie: Suffix trie optimization for queries. +- sortable: Enables sorting based on the tag field. + +**Numeric and Geo Field Attributes**: + +- Both numeric and geo fields support the sortable attribute, enabling sorting on these fields. + +**Common Vector Field Attributes**: + +- dims: Dimensionality of the vector. +- algorithm: Indexing algorithm (flat or hnsw). +- datatype: Float datatype of the vector (bfloat16, float16, float32, float64). +- distance_metric: Metric for measuring query relevance (COSINE, L2, IP). + +**HNSW Vector Field Specific Attributes**: + +- m: Max outgoing edges per node in each layer. +- ef_construction: Max edge candidates during build time. +- ef_runtime: Max top candidates during search. +- epsilon: Range search boundary factor. + +Note: +: See fully documented Redis-supported fields and options here: [https://redis.io/commands/ft.create/](https://redis.io/commands/ft.create/) diff --git a/content/integrate/redisvl/api/searchindex.md b/content/integrate/redisvl/api/searchindex.md new file mode 100644 index 0000000000..edac140c40 --- /dev/null +++ b/content/integrate/redisvl/api/searchindex.md @@ -0,0 +1,853 @@ +--- +linkTitle: Search Index Classes +title: Search Index Classes +type: integration +--- + + +| Class | Description | +|-------------------------------------------|----------------------------------------------------------------------------------------------| +| [SearchIndex](#searchindex-api) | Primary class to write, read, and search across data structures in Redis. | +| [AsyncSearchIndex](#asyncsearchindex-api) | Async version of the SearchIndex to write, read, and search across data structures in Redis. | + + + +## SearchIndex + +### *class* SearchIndex(schema, redis_client=None, redis_url=None, connection_args={}, \*\*kwargs) + +A search index class for interacting with Redis as a vector database. + +The SearchIndex is instantiated with a reference to a Redis database and an +IndexSchema (YAML path or dictionary object) that describes the various +settings and field configurations. + +```python +from redisvl.index import SearchIndex + +# initialize the index object with schema from file +index = SearchIndex.from_yaml("schemas/schema.yaml") +index.connect(redis_url="redis://localhost:6379") + +# create the index +index.create(overwrite=True) + +# data is an iterable of dictionaries +index.load(data) + +# delete index and data +index.delete(drop=True) +``` + +Initialize the RedisVL search index with a schema, Redis client +(or URL string with other connection args), connection_args, and other +kwargs. + +* **Parameters:** + * **schema** ([*IndexSchema*](schema.md#redisvl.schema.IndexSchema)) – Index schema object. + * **redis_client** (*Optional* *[**redis.Redis* *]*) – An + instantiated redis client. + * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to + connect to. + * **connection_args** (*Dict* *[**str* *,* *Any* *]* *,* *optional*) – Redis client connection + args. + +#### aggregate(\*args, \*\*kwargs) + +Perform an aggregation operation against the index. + +Wrapper around the aggregation API that adds the index name +to the query and passes along the rest of the arguments +to the redis-py ft().aggregate() method. + +* **Returns:** + Raw Redis aggregation results. +* **Return type:** + Result + +#### clear() + +Clear all keys in Redis associated with the index, leaving the index +available and in-place for future insertions or updates. + +* **Returns:** + Count of records deleted from Redis. +* **Return type:** + int + +#### connect(redis_url=None, \*\*kwargs) + +Connect to a Redis instance using the provided redis_url, falling +back to the REDIS_URL environment variable (if available). + +Note: Additional keyword arguments (\*\*kwargs) can be used to provide +extra options specific to the Redis connection. + +* **Parameters:** + **redis_url** (*Optional* *[**str* *]* *,* *optional*) – The URL of the Redis server to + connect to. If not provided, the method defaults to using the + REDIS_URL environment variable. +* **Raises:** + * **redis.exceptions.ConnectionError** – If the connection to the Redis + server fails. + * **ValueError** – If the Redis URL is not provided nor accessible + through the REDIS_URL environment variable. + +```python +index.connect(redis_url="redis://localhost:6379") +``` + +#### create(overwrite=False, drop=False) + +Create an index in Redis with the current schema and properties. + +* **Parameters:** + * **overwrite** (*bool* *,* *optional*) – Whether to overwrite the index if it + already exists. Defaults to False. + * **drop** (*bool* *,* *optional*) – Whether to drop all keys associated with the + index in the case of overwriting. Defaults to False. +* **Raises:** + * **RuntimeError** – If the index already exists and ‘overwrite’ is False. + * **ValueError** – If no fields are defined for the index. +* **Return type:** + None + +```python +# create an index in Redis; only if one does not exist with given name +index.create() + +# overwrite an index in Redis without dropping associated data +index.create(overwrite=True) + +# overwrite an index in Redis; drop associated data (clean slate) +index.create(overwrite=True, drop=True) +``` + +#### delete(drop=True) + +Delete the search index while optionally dropping all keys associated +with the index. + +* **Parameters:** + **drop** (*bool* *,* *optional*) – Delete the key / documents pairs in the + index. Defaults to True. +* **Raises:** + **redis.exceptions.ResponseError** – If the index does not exist. + +#### disconnect() + +Disconnect from the Redis database. + +#### drop_keys(keys) + +Remove a specific entry or entries from the index by it’s key ID. + +* **Parameters:** + **keys** (*Union* *[**str* *,* *List* *[**str* *]* *]*) – The document ID or IDs to remove from the index. +* **Returns:** + Count of records deleted from Redis. +* **Return type:** + int + +#### exists() + +Check if the index exists in Redis. + +* **Returns:** + True if the index exists, False otherwise. +* **Return type:** + bool + +#### fetch(id) + +Fetch an object from Redis by id. + +The id is typically either a unique identifier, +or derived from some domain-specific metadata combination +(like a document id or chunk id). + +* **Parameters:** + **id** (*str*) – The specified unique identifier for a particular + document indexed in Redis. +* **Returns:** + The fetched object. +* **Return type:** + Dict[str, Any] + +#### *classmethod* from_dict(schema_dict, \*\*kwargs) + +Create a SearchIndex from a dictionary. + +* **Parameters:** + **schema_dict** (*Dict* *[**str* *,* *Any* *]*) – A dictionary containing the schema. +* **Returns:** + A RedisVL SearchIndex object. +* **Return type:** + [SearchIndex](#redisvl.index.SearchIndex) + +```python +from redisvl.index import SearchIndex + +index = SearchIndex.from_dict({ + "index": { + "name": "my-index", + "prefix": "rvl", + "storage_type": "hash", + }, + "fields": [ + {"name": "doc-id", "type": "tag"} + ] +}) +``` + +#### *classmethod* from_existing(name, redis_client=None, redis_url=None, \*\*kwargs) + +Initialize from an existing search index in Redis by index name. + +* **Parameters:** + * **name** (*str*) – Name of the search index in Redis. + * **redis_client** (*Optional* *[**redis.Redis* *]*) – An + instantiated redis client. + * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to + connect to. + +#### *classmethod* from_yaml(schema_path, \*\*kwargs) + +Create a SearchIndex from a YAML schema file. + +* **Parameters:** + **schema_path** (*str*) – Path to the YAML schema file. +* **Returns:** + A RedisVL SearchIndex object. +* **Return type:** + [SearchIndex](#redisvl.index.SearchIndex) + +```python +from redisvl.index import SearchIndex + +index = SearchIndex.from_yaml("schemas/schema.yaml") +``` + +#### info(name=None) + +Get information about the index. + +* **Parameters:** + **name** (*str* *,* *optional*) – Index name to fetch info about. + Defaults to None. +* **Returns:** + A dictionary containing the information about the index. +* **Return type:** + dict + +#### key(id) + +Construct a redis key as a combination of an index key prefix (optional) +and specified id. + +The id is typically either a unique identifier, or +derived from some domain-specific metadata combination (like a document +id or chunk id). + +* **Parameters:** + **id** (*str*) – The specified unique identifier for a particular + document indexed in Redis. +* **Returns:** + The full Redis key including key prefix and value as a string. +* **Return type:** + str + +#### listall() + +List all search indices in Redis database. + +* **Returns:** + The list of indices in the database. +* **Return type:** + List[str] + +#### load(data, id_field=None, keys=None, ttl=None, preprocess=None, batch_size=None) + +Load objects to the Redis database. Returns the list of keys loaded +to Redis. + +RedisVL automatically handles constructing the object keys, batching, +optional preprocessing steps, and setting optional expiration +(TTL policies) on keys. + +* **Parameters:** + * **data** (*Iterable* *[**Any* *]*) – An iterable of objects to store. + * **id_field** (*Optional* *[**str* *]* *,* *optional*) – Specified field used as the id + portion of the redis key (after the prefix) for each + object. Defaults to None. + * **keys** (*Optional* *[**Iterable* *[**str* *]* *]* *,* *optional*) – Optional iterable of keys. + Must match the length of objects if provided. Defaults to None. + * **ttl** (*Optional* *[**int* *]* *,* *optional*) – Time-to-live in seconds for each key. + Defaults to None. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – A function to preprocess + objects before storage. Defaults to None. + * **batch_size** (*Optional* *[**int* *]* *,* *optional*) – Number of objects to write in + a single Redis pipeline execution. Defaults to class’s + default batch size. +* **Returns:** + List of keys loaded to Redis. +* **Return type:** + List[str] +* **Raises:** + **ValueError** – If the length of provided keys does not match the length + of objects. + +```python +data = [{"test": "foo"}, {"test": "bar"}] + +# simple case +keys = index.load(data) + +# set 360 second ttl policy on data +keys = index.load(data, ttl=360) + +# load data with predefined keys +keys = index.load(data, keys=["rvl:foo", "rvl:bar"]) + +# load data with preprocessing step +def add_field(d): + d["new_field"] = 123 + return d +keys = index.load(data, preprocess=add_field) +``` + +#### paginate(query, page_size=30) + +Execute a given query against the index and return results in +paginated batches. + +This method accepts a RedisVL query instance, enabling pagination of +results which allows for subsequent processing over each batch with a +generator. + +* **Parameters:** + * **query** (*BaseQuery*) – The search query to be executed. + * **page_size** (*int* *,* *optional*) – The number of results to return in each + batch. Defaults to 30. +* **Yields:** + A generator yielding batches of search results. +* **Raises:** + * **TypeError** – If the page_size argument is not of type int. + * **ValueError** – If the page_size argument is less than or equal to zero. +* **Return type:** + *Generator* + +```python +# Iterate over paginated search results in batches of 10 +for result_batch in index.paginate(query, page_size=10): + # Process each batch of results + pass +``` + +#### NOTE +The page_size parameter controls the number of items each result +batch contains. Adjust this value based on performance +considerations and the expected volume of search results. + +#### query(query) + +Execute a query on the index. + +This method takes a BaseQuery object directly, runs the search, and +handles post-processing of the search. + +* **Parameters:** + **query** (*BaseQuery*) – The query to run. +* **Returns:** + A list of search results. +* **Return type:** + List[Result] + +```python +from redisvl.query import VectorQuery + +query = VectorQuery( + vector=[0.16, -0.34, 0.98, 0.23], + vector_field_name="embedding", + num_results=3 +) + +results = index.query(query) +``` + +#### search(\*args, \*\*kwargs) + +Perform a search against the index. + +Wrapper around the search API that adds the index name +to the query and passes along the rest of the arguments +to the redis-py ft().search() method. + +* **Returns:** + Raw Redis search results. +* **Return type:** + Result + +#### set_client(redis_client, \*\*kwargs) + +Manually set the Redis client to use with the search index. + +This method configures the search index to use a specific Redis or +Async Redis client. It is useful for cases where an external, +custom-configured client is preferred instead of creating a new one. + +* **Parameters:** + **redis_client** (*redis.Redis*) – A Redis or Async Redis + client instance to be used for the connection. +* **Raises:** + **TypeError** – If the provided client is not valid. + +```python +import redis +from redisvl.index import SearchIndex + +client = redis.Redis.from_url("redis://localhost:6379") +index = SearchIndex.from_yaml("schemas/schema.yaml") +index.set_client(client) +``` + +#### *property* client *: Redis | None* + +The underlying redis-py client object. + +#### *property* key_separator *: str* + +The optional separator between a defined prefix and key value in +forming a Redis key. + +#### *property* name *: str* + +The name of the Redis search index. + +#### *property* prefix *: str* + +The optional key prefix that comes before a unique key value in +forming a Redis key. + +#### *property* storage_type *: StorageType* + +The underlying storage type for the search index; either +hash or json. + + + +## AsyncSearchIndex + +### *class* AsyncSearchIndex(schema, \*\*kwargs) + +A search index class for interacting with Redis as a vector database in +async-mode. + +The AsyncSearchIndex is instantiated with a reference to a Redis database +and an IndexSchema (YAML path or dictionary object) that describes the +various settings and field configurations. + +```python +from redisvl.index import AsyncSearchIndex + +# initialize the index object with schema from file +index = AsyncSearchIndex.from_yaml("schemas/schema.yaml") +await index.connect(redis_url="redis://localhost:6379") + +# create the index +await index.create(overwrite=True) + +# data is an iterable of dictionaries +await index.load(data) + +# delete index and data +await index.delete(drop=True) +``` + +Initialize the RedisVL async search index with a schema. + +* **Parameters:** + * **schema** ([*IndexSchema*](schema.md#redisvl.schema.IndexSchema)) – Index schema object. + * **connection_args** (*Dict* *[**str* *,* *Any* *]* *,* *optional*) – Redis client connection + args. + +#### *async* aggregate(\*args, \*\*kwargs) + +Perform an aggregation operation against the index. + +Wrapper around the aggregation API that adds the index name +to the query and passes along the rest of the arguments +to the redis-py ft().aggregate() method. + +* **Returns:** + Raw Redis aggregation results. +* **Return type:** + Result + +#### *async* clear() + +Clear all keys in Redis associated with the index, leaving the index +available and in-place for future insertions or updates. + +* **Returns:** + Count of records deleted from Redis. +* **Return type:** + int + +#### *async* connect(redis_url=None, \*\*kwargs) + +Connect to a Redis instance using the provided redis_url, falling +back to the REDIS_URL environment variable (if available). + +Note: Additional keyword arguments (\*\*kwargs) can be used to provide +extra options specific to the Redis connection. + +* **Parameters:** + **redis_url** (*Optional* *[**str* *]* *,* *optional*) – The URL of the Redis server to + connect to. If not provided, the method defaults to using the + REDIS_URL environment variable. +* **Raises:** + * **redis.exceptions.ConnectionError** – If the connection to the Redis + server fails. + * **ValueError** – If the Redis URL is not provided nor accessible + through the REDIS_URL environment variable. + +```python +index.connect(redis_url="redis://localhost:6379") +``` + +#### *async* create(overwrite=False, drop=False) + +Asynchronously create an index in Redis with the current schema +: and properties. + +* **Parameters:** + * **overwrite** (*bool* *,* *optional*) – Whether to overwrite the index if it + already exists. Defaults to False. + * **drop** (*bool* *,* *optional*) – Whether to drop all keys associated with the + index in the case of overwriting. Defaults to False. +* **Raises:** + * **RuntimeError** – If the index already exists and ‘overwrite’ is False. + * **ValueError** – If no fields are defined for the index. +* **Return type:** + None + +```python +# create an index in Redis; only if one does not exist with given name +await index.create() + +# overwrite an index in Redis without dropping associated data +await index.create(overwrite=True) + +# overwrite an index in Redis; drop associated data (clean slate) +await index.create(overwrite=True, drop=True) +``` + +#### *async* delete(drop=True) + +Delete the search index. + +* **Parameters:** + **drop** (*bool* *,* *optional*) – Delete the documents in the index. + Defaults to True. +* **Raises:** + **redis.exceptions.ResponseError** – If the index does not exist. + +#### disconnect() + +Disconnect and cleanup the underlying async redis connection. + +#### *async* drop_keys(keys) + +Remove a specific entry or entries from the index by it’s key ID. + +* **Parameters:** + **keys** (*Union* *[**str* *,* *List* *[**str* *]* *]*) – The document ID or IDs to remove from the index. +* **Returns:** + Count of records deleted from Redis. +* **Return type:** + int + +#### *async* exists() + +Check if the index exists in Redis. + +* **Returns:** + True if the index exists, False otherwise. +* **Return type:** + bool + +#### *async* fetch(id) + +Asynchronously etch an object from Redis by id. The id is typically +either a unique identifier, or derived from some domain-specific +metadata combination (like a document id or chunk id). + +* **Parameters:** + **id** (*str*) – The specified unique identifier for a particular + document indexed in Redis. +* **Returns:** + The fetched object. +* **Return type:** + Dict[str, Any] + +#### *classmethod* from_dict(schema_dict, \*\*kwargs) + +Create a SearchIndex from a dictionary. + +* **Parameters:** + **schema_dict** (*Dict* *[**str* *,* *Any* *]*) – A dictionary containing the schema. +* **Returns:** + A RedisVL SearchIndex object. +* **Return type:** + [SearchIndex](#redisvl.index.SearchIndex) + +```python +from redisvl.index import SearchIndex + +index = SearchIndex.from_dict({ + "index": { + "name": "my-index", + "prefix": "rvl", + "storage_type": "hash", + }, + "fields": [ + {"name": "doc-id", "type": "tag"} + ] +}) +``` + +#### *async classmethod* from_existing(name, redis_client=None, redis_url=None, \*\*kwargs) + +Initialize from an existing search index in Redis by index name. + +* **Parameters:** + * **name** (*str*) – Name of the search index in Redis. + * **redis_client** (*Optional* *[**redis.Redis* *]*) – An + instantiated redis client. + * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to + connect to. + +#### *classmethod* from_yaml(schema_path, \*\*kwargs) + +Create a SearchIndex from a YAML schema file. + +* **Parameters:** + **schema_path** (*str*) – Path to the YAML schema file. +* **Returns:** + A RedisVL SearchIndex object. +* **Return type:** + [SearchIndex](#redisvl.index.SearchIndex) + +```python +from redisvl.index import SearchIndex + +index = SearchIndex.from_yaml("schemas/schema.yaml") +``` + +#### *async* info(name=None) + +Get information about the index. + +* **Parameters:** + **name** (*str* *,* *optional*) – Index name to fetch info about. + Defaults to None. +* **Returns:** + A dictionary containing the information about the index. +* **Return type:** + dict + +#### key(id) + +Construct a redis key as a combination of an index key prefix (optional) +and specified id. + +The id is typically either a unique identifier, or +derived from some domain-specific metadata combination (like a document +id or chunk id). + +* **Parameters:** + **id** (*str*) – The specified unique identifier for a particular + document indexed in Redis. +* **Returns:** + The full Redis key including key prefix and value as a string. +* **Return type:** + str + +#### *async* listall() + +List all search indices in Redis database. + +* **Returns:** + The list of indices in the database. +* **Return type:** + List[str] + +#### *async* load(data, id_field=None, keys=None, ttl=None, preprocess=None, concurrency=None) + +Asynchronously load objects to Redis with concurrency control. +Returns the list of keys loaded to Redis. + +RedisVL automatically handles constructing the object keys, batching, +optional preprocessing steps, and setting optional expiration +(TTL policies) on keys. + +* **Parameters:** + * **data** (*Iterable* *[**Any* *]*) – An iterable of objects to store. + * **id_field** (*Optional* *[**str* *]* *,* *optional*) – Specified field used as the id + portion of the redis key (after the prefix) for each + object. Defaults to None. + * **keys** (*Optional* *[**Iterable* *[**str* *]* *]* *,* *optional*) – Optional iterable of keys. + Must match the length of objects if provided. Defaults to None. + * **ttl** (*Optional* *[**int* *]* *,* *optional*) – Time-to-live in seconds for each key. + Defaults to None. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – An async function to + preprocess objects before storage. Defaults to None. + * **concurrency** (*Optional* *[**int* *]* *,* *optional*) – The maximum number of + concurrent write operations. Defaults to class’s default + concurrency level. +* **Returns:** + List of keys loaded to Redis. +* **Return type:** + List[str] +* **Raises:** + **ValueError** – If the length of provided keys does not match the + length of objects. + +```python +data = [{"test": "foo"}, {"test": "bar"}] + +# simple case +keys = await index.load(data) + +# set 360 second ttl policy on data +keys = await index.load(data, ttl=360) + +# load data with predefined keys +keys = await index.load(data, keys=["rvl:foo", "rvl:bar"]) + +# load data with preprocessing step +async def add_field(d): + d["new_field"] = 123 + return d +keys = await index.load(data, preprocess=add_field) +``` + +#### *async* paginate(query, page_size=30) + +Execute a given query against the index and return results in +paginated batches. + +This method accepts a RedisVL query instance, enabling async pagination +of results which allows for subsequent processing over each batch with a +generator. + +* **Parameters:** + * **query** (*BaseQuery*) – The search query to be executed. + * **page_size** (*int* *,* *optional*) – The number of results to return in each + batch. Defaults to 30. +* **Yields:** + An async generator yielding batches of search results. +* **Raises:** + * **TypeError** – If the page_size argument is not of type int. + * **ValueError** – If the page_size argument is less than or equal to zero. +* **Return type:** + *AsyncGenerator* + +```python +# Iterate over paginated search results in batches of 10 +async for result_batch in index.paginate(query, page_size=10): + # Process each batch of results + pass +``` + +#### NOTE +The page_size parameter controls the number of items each result +batch contains. Adjust this value based on performance +considerations and the expected volume of search results. + +#### *async* query(query) + +Asynchronously execute a query on the index. + +This method takes a BaseQuery object directly, runs the search, and +handles post-processing of the search. + +* **Parameters:** + **query** (*BaseQuery*) – The query to run. +* **Returns:** + A list of search results. +* **Return type:** + List[Result] + +```python +from redisvl.query import VectorQuery + +query = VectorQuery( + vector=[0.16, -0.34, 0.98, 0.23], + vector_field_name="embedding", + num_results=3 +) + +results = await index.query(query) +``` + +#### *async* search(\*args, \*\*kwargs) + +Perform a search on this index. + +Wrapper around redis.search.Search that adds the index name +to the search query and passes along the rest of the arguments +to the redis-py ft.search() method. + +* **Returns:** + Raw Redis search results. +* **Return type:** + Result + +#### *async* set_client(redis_client) + +Manually set the Redis client to use with the search index. + +This method configures the search index to use a specific +Async Redis client. It is useful for cases where an external, +custom-configured client is preferred instead of creating a new one. + +* **Parameters:** + **redis_client** (*aredis.Redis*) – An Async Redis + client instance to be used for the connection. +* **Raises:** + **TypeError** – If the provided client is not valid. + +```python +import redis.asyncio as aredis +from redisvl.index import AsyncSearchIndex + +# async Redis client and index +client = aredis.Redis.from_url("redis://localhost:6379") +index = AsyncSearchIndex.from_yaml("schemas/schema.yaml") +await index.set_client(client) +``` + +#### *property* client *: Redis | None* + +The underlying redis-py client object. + +#### *property* key_separator *: str* + +The optional separator between a defined prefix and key value in +forming a Redis key. + +#### *property* name *: str* + +The name of the Redis search index. + +#### *property* prefix *: str* + +The optional key prefix that comes before a unique key value in +forming a Redis key. + +#### *property* storage_type *: StorageType* + +The underlying storage type for the search index; either +hash or json. diff --git a/content/integrate/redisvl/api/session_manager.md b/content/integrate/redisvl/api/session_manager.md new file mode 100644 index 0000000000..7b668e8701 --- /dev/null +++ b/content/integrate/redisvl/api/session_manager.md @@ -0,0 +1,278 @@ +--- +linkTitle: LLM Session Manager +title: LLM Session Manager +type: integration +--- + + +## SemanticSessionManager + + + +### *class* SemanticSessionManager(name, session_tag=None, prefix=None, vectorizer=None, distance_threshold=0.3, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, \*\*kwargs) + +Bases: `BaseSessionManager` + +Initialize session memory with index + +Session Manager stores the current and previous user text prompts and +LLM responses to allow for enriching future prompts with session +context. Session history is stored in individual user or LLM prompts and +responses. + +* **Parameters:** + * **name** (*str*) – The name of the session manager index. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. + * **prefix** (*Optional* *[**str* *]*) – Prefix for the keys for this session data. + Defaults to None and will be replaced with the index name. + * **vectorizer** (*Optional* *[**BaseVectorizer* *]*) – The vectorizer used to create embeddings. + * **distance_threshold** (*float*) – The maximum semantic distance to be + included in the context. Defaults to 0.3. + * **redis_client** (*Optional* *[**Redis* *]*) – A Redis client instance. Defaults to + None. + * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. + * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments + for the redis client. Defaults to empty {}. + * **overwrite** (*bool*) – Whether or not to force overwrite the schema for + the semantic session index. Defaults to false. + +The proposed schema will support a single vector embedding constructed +from either the prompt or response in a single string. + +#### add_message(message, session_tag=None) + +Insert a single prompt or response into the session memory. +A timestamp is associated with it so that it can be later sorted +in sequential ordering after retrieval. + +* **Parameters:** + * **message** (*Dict* *[**str* *,**str* *]*) – The user prompt or LLM response. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. +* **Return type:** + None + +#### add_messages(messages, session_tag=None) + +Insert a list of prompts and responses into the session memory. +A timestamp is associated with each so that they can be later sorted +in sequential ordering after retrieval. + +* **Parameters:** + * **messages** (*List* *[**Dict* *[**str* *,* *str* *]* *]*) – The list of user prompts and LLM responses. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. +* **Return type:** + None + +#### clear() + +Clears the chat session history. + +* **Return type:** + None + +#### delete() + +Clear all conversation keys and remove the search index. + +* **Return type:** + None + +#### drop(id=None) + +Remove a specific exchange from the conversation history. + +* **Parameters:** + **id** (*Optional* *[**str* *]*) – The id of the session entry to delete. + If None then the last entry is deleted. +* **Return type:** + None + +#### get_recent(top_k=5, as_text=False, raw=False, session_tag=None) + +Retreive the recent conversation history in sequential order. + +* **Parameters:** + * **top_k** (*int*) – The number of previous exchanges to return. Default is 5. + * **as_text** (*bool*) – Whether to return the conversation as a single string, + or list of alternating prompts and responses. + * **raw** (*bool*) – Whether to return the full Redis hash entry or just the + prompt and response + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. +* **Returns:** + A single string transcription of the session + : or list of strings if as_text is false. +* **Return type:** + Union[str, List[str]] +* **Raises:** + **ValueError** – if top_k is not an integer greater than or equal to 0. + +#### get_relevant(prompt, as_text=False, top_k=5, fall_back=False, session_tag=None, raw=False, distance_threshold=None) + +Searches the chat history for information semantically related to +the specified prompt. + +This method uses vector similarity search with a text prompt as input. +It checks for semantically similar prompts and responses and gets +the top k most relevant previous prompts or responses to include as +context to the next LLM call. + +* **Parameters:** + * **prompt** (*str*) – The message text to search for in session memory + * **as_text** (*bool*) – Whether to return the prompts and responses as text + * **JSON** (*or as*) + * **top_k** (*int*) – The number of previous messages to return. Default is 5. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. + * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic + vector distance. + * **fall_back** (*bool*) – Whether to drop back to recent conversation history + if no relevant context is found. + * **raw** (*bool*) – Whether to return the full Redis hash entry or just the + message. +* **Returns:** + Either a list of strings, or a + list of prompts and responses in JSON containing the most relevant. +* **Return type:** + Union[List[str], List[Dict[str,str]] + +Raises ValueError: if top_k is not an integer greater or equal to 0. + +#### store(prompt, response, session_tag=None) + +Insert a prompt:response pair into the session memory. A timestamp +is associated with each message so that they can be later sorted +in sequential ordering after retrieval. + +* **Parameters:** + * **prompt** (*str*) – The user prompt to the LLM. + * **response** (*str*) – The corresponding LLM response. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. +* **Return type:** + None + +#### *property* messages *: List[str] | List[Dict[str, str]]* + +Returns the full chat history. + +## StandardSessionManager + + + +### *class* StandardSessionManager(name, session_tag=None, prefix=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, \*\*kwargs) + +Bases: `BaseSessionManager` + +Initialize session memory + +Session Manager stores the current and previous user text prompts and +LLM responses to allow for enriching future prompts with session +context.Session history is stored in individual user or LLM prompts and +responses. + +* **Parameters:** + * **name** (*str*) – The name of the session manager index. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. + * **prefix** (*Optional* *[**str* *]*) – Prefix for the keys for this session data. + Defaults to None and will be replaced with the index name. + * **redis_client** (*Optional* *[**Redis* *]*) – A Redis client instance. Defaults to + None. + * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. + * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments + for the redis client. Defaults to empty {}. + +The proposed schema will support a single combined vector embedding +constructed from the prompt & response in a single string. + +#### add_message(message, session_tag=None) + +Insert a single prompt or response into the session memory. +A timestamp is associated with it so that it can be later sorted +in sequential ordering after retrieval. + +* **Parameters:** + * **message** (*Dict* *[**str* *,**str* *]*) – The user prompt or LLM response. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. +* **Return type:** + None + +#### add_messages(messages, session_tag=None) + +Insert a list of prompts and responses into the session memory. +A timestamp is associated with each so that they can be later sorted +in sequential ordering after retrieval. + +* **Parameters:** + * **messages** (*List* *[**Dict* *[**str* *,* *str* *]* *]*) – The list of user prompts and LLM responses. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. +* **Return type:** + None + +#### clear() + +Clears the chat session history. + +* **Return type:** + None + +#### delete() + +Clear all conversation keys and remove the search index. + +* **Return type:** + None + +#### drop(id=None) + +Remove a specific exchange from the conversation history. + +* **Parameters:** + **id** (*Optional* *[**str* *]*) – The id of the session entry to delete. + If None then the last entry is deleted. +* **Return type:** + None + +#### get_recent(top_k=5, as_text=False, raw=False, session_tag=None) + +Retrieve the recent conversation history in sequential order. + +* **Parameters:** + * **top_k** (*int*) – The number of previous messages to return. Default is 5. + * **as_text** (*bool*) – Whether to return the conversation as a single string, + or list of alternating prompts and responses. + * **raw** (*bool*) – Whether to return the full Redis hash entry or just the + prompt and response + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. +* **Returns:** + A single string transcription of the session + : or list of strings if as_text is false. +* **Return type:** + Union[str, List[str]] +* **Raises:** + **ValueError** – if top_k is not an integer greater than or equal to 0. + +#### store(prompt, response, session_tag=None) + +Insert a prompt:response pair into the session memory. A timestamp +is associated with each exchange so that they can be later sorted +in sequential ordering after retrieval. + +* **Parameters:** + * **prompt** (*str*) – The user prompt to the LLM. + * **response** (*str*) – The corresponding LLM response. + * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + session. Defaults to instance ULID. +* **Return type:** + None + +#### *property* messages *: List[str] | List[Dict[str, str]]* + +Returns the full chat history. diff --git a/content/integrate/redisvl/api/vectorizer.md b/content/integrate/redisvl/api/vectorizer.md new file mode 100644 index 0000000000..cf54d3a2be --- /dev/null +++ b/content/integrate/redisvl/api/vectorizer.md @@ -0,0 +1,1035 @@ +--- +linkTitle: Vectorizers +title: Vectorizers +type: integration +--- + + +## HFTextVectorizer + + + +### *class* HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2', dtype='float32', \*, dims=None) + +Bases: `BaseVectorizer` + +The HFTextVectorizer class is designed to leverage the power of Hugging +Face’s Sentence Transformers for generating text embeddings. This vectorizer +is particularly useful in scenarios where advanced natural language +processing and understanding are required, and ideal for running on your own +hardware (for free). + +Utilizing this vectorizer involves specifying a pre-trained model from +Hugging Face’s vast collection of Sentence Transformers. These models are +trained on a variety of datasets and tasks, ensuring versatility and +robust performance across different text embedding needs. Additionally, +make sure the sentence-transformers library is installed with +pip install sentence-transformers==2.2.2. + +```python +# Embedding a single text +vectorizer = HFTextVectorizer(model="sentence-transformers/all-mpnet-base-v2") +embedding = vectorizer.embed("Hello, world!") + +# Embedding a batch of texts +embeddings = vectorizer.embed_many(["Hello, world!", "How are you?"], batch_size=2) +``` + +Initialize the Hugging Face text vectorizer. + +* **Parameters:** + * **model** (*str*) – The pre-trained model from Hugging Face’s Sentence + Transformers to be used for embedding. Defaults to + ‘sentence-transformers/all-mpnet-base-v2’. + * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. + Used when setting as_buffer=True in calls to embed() and embed_many(). + Defaults to ‘float32’. + * **dims** (*int* *|* *None*) +* **Raises:** + * **ImportError** – If the sentence-transformers library is not installed. + * **ValueError** – If there is an error setting the embedding model dimensions. + * **ValueError** – If an invalid dtype is provided. + +#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Embed a chunk of text using the Hugging Face sentence transformer. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing + callable to perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the text. + +#### embed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, \*\*kwargs) + +Asynchronously embed many chunks of texts using the Hugging Face +sentence transformer. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing + callable to perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. Defaults to 10. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the test. + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## OpenAITextVectorizer + + + +### *class* OpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', \*, dims=None) + +Bases: `BaseVectorizer` + +The OpenAITextVectorizer class utilizes OpenAI’s API to generate +embeddings for text data. + +This vectorizer is designed to interact with OpenAI’s embeddings API, +requiring an API key for authentication. The key can be provided directly +in the api_config dictionary or through the OPENAI_API_KEY environment +variable. Users must obtain an API key from OpenAI’s website +([https://api.openai.com/](https://api.openai.com/)). Additionally, the openai python client must be +installed with pip install openai>=1.13.0. + +The vectorizer supports both synchronous and asynchronous operations, +allowing for batch processing of texts and flexibility in handling +preprocessing tasks. + +```python +# Synchronous embedding of a single text +vectorizer = OpenAITextVectorizer( + model="text-embedding-ada-002", + api_config={"api_key": "your_api_key"} # OR set OPENAI_API_KEY in your env +) +embedding = vectorizer.embed("Hello, world!") + +# Asynchronous batch embedding of multiple texts +embeddings = await vectorizer.aembed_many( + ["Hello, world!", "How are you?"], + batch_size=2 +) +``` + +Initialize the OpenAI vectorizer. + +* **Parameters:** + * **model** (*str*) – Model to use for embedding. Defaults to + ‘text-embedding-ada-002’. + * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the + API key and any additional OpenAI API options. Defaults to None. + * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. + Used when setting as_buffer=True in calls to embed() and embed_many(). + Defaults to ‘float32’. + * **dims** (*int* *|* *None*) +* **Raises:** + * **ImportError** – If the openai library is not installed. + * **ValueError** – If the OpenAI API key is not provided. + * **ValueError** – If an invalid dtype is provided. + +#### aembed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Asynchronously embed a chunk of text using the OpenAI API. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the text. + +#### aembed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, \*\*kwargs) + +Asynchronously embed many chunks of texts using the OpenAI API. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. Defaults to 10. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the text. + +#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Embed a chunk of text using the OpenAI API. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the text. + +#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) + +Embed many chunks of texts using the OpenAI API. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing + callable to perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. Defaults to 10. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the text. + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## AzureOpenAITextVectorizer + + + +### *class* AzureOpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', \*, dims=None) + +Bases: `BaseVectorizer` + +The AzureOpenAITextVectorizer class utilizes AzureOpenAI’s API to generate +embeddings for text data. + +This vectorizer is designed to interact with AzureOpenAI’s embeddings API, +requiring an API key, an AzureOpenAI deployment endpoint and API version. +These values can be provided directly in the api_config dictionary with +the parameters ‘azure_endpoint’, ‘api_version’ and ‘api_key’ or through the +environment variables ‘AZURE_OPENAI_ENDPOINT’, ‘OPENAI_API_VERSION’, and ‘AZURE_OPENAI_API_KEY’. +Users must obtain these values from the ‘Keys and Endpoints’ section in their Azure OpenAI service. +Additionally, the openai python client must be installed with pip install openai>=1.13.0. + +The vectorizer supports both synchronous and asynchronous operations, +allowing for batch processing of texts and flexibility in handling +preprocessing tasks. + +```python +# Synchronous embedding of a single text +vectorizer = AzureOpenAITextVectorizer( + model="text-embedding-ada-002", + api_config={ + "api_key": "your_api_key", # OR set AZURE_OPENAI_API_KEY in your env + "api_version": "your_api_version", # OR set OPENAI_API_VERSION in your env + "azure_endpoint": "your_azure_endpoint", # OR set AZURE_OPENAI_ENDPOINT in your env + } +) +embedding = vectorizer.embed("Hello, world!") + +# Asynchronous batch embedding of multiple texts +embeddings = await vectorizer.aembed_many( + ["Hello, world!", "How are you?"], + batch_size=2 +) +``` + +Initialize the AzureOpenAI vectorizer. + +* **Parameters:** + * **model** (*str*) – Deployment to use for embedding. Must be the + ‘Deployment name’ not the ‘Model name’. Defaults to + ‘text-embedding-ada-002’. + * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the + API key, API version, Azure endpoint, and any other API options. + Defaults to None. + * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. + Used when setting as_buffer=True in calls to embed() and embed_many(). + Defaults to ‘float32’. + * **dims** (*int* *|* *None*) +* **Raises:** + * **ImportError** – If the openai library is not installed. + * **ValueError** – If the AzureOpenAI API key, version, or endpoint are not provided. + * **ValueError** – If an invalid dtype is provided. + +#### aembed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Asynchronously embed a chunk of text using the OpenAI API. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the test. + +#### aembed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, \*\*kwargs) + +Asynchronously embed many chunks of texts using the AzureOpenAI API. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. Defaults to 10. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the test. + +#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Embed a chunk of text using the AzureOpenAI API. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the test. + +#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) + +Embed many chunks of texts using the AzureOpenAI API. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing + callable to perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. Defaults to 10. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the test. + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## VertexAITextVectorizer + + + +### *class* VertexAITextVectorizer(model='textembedding-gecko', api_config=None, dtype='float32', \*, dims=None) + +Bases: `BaseVectorizer` + +The VertexAITextVectorizer uses Google’s VertexAI Palm 2 embedding model +API to create text embeddings. + +This vectorizer is tailored for use in +environments where integration with Google Cloud Platform (GCP) services is +a key requirement. + +Utilizing this vectorizer requires an active GCP project and location +(region), along with appropriate application credentials. These can be +provided through the api_config dictionary or set the GOOGLE_APPLICATION_CREDENTIALS +env var. Additionally, the vertexai python client must be +installed with pip install google-cloud-aiplatform>=1.26. + +```python +# Synchronous embedding of a single text +vectorizer = VertexAITextVectorizer( + model="textembedding-gecko", + api_config={ + "project_id": "your_gcp_project_id", # OR set GCP_PROJECT_ID + "location": "your_gcp_location", # OR set GCP_LOCATION + }) +embedding = vectorizer.embed("Hello, world!") + +# Asynchronous batch embedding of multiple texts +embeddings = await vectorizer.embed_many( + ["Hello, world!", "Goodbye, world!"], + batch_size=2 +) +``` + +Initialize the VertexAI vectorizer. + +* **Parameters:** + * **model** (*str*) – Model to use for embedding. Defaults to + ‘textembedding-gecko’. + * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the + API config details. Defaults to None. + * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. + Used when setting as_buffer=True in calls to embed() and embed_many(). + Defaults to ‘float32’. + * **dims** (*int* *|* *None*) +* **Raises:** + * **ImportError** – If the google-cloud-aiplatform library is not installed. + * **ValueError** – If the API key is not provided. + * **ValueError** – If an invalid dtype is provided. + +#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Embed a chunk of text using the VertexAI API. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the test. + +#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) + +Embed many chunks of texts using the VertexAI API. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. Defaults to 10. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – If the wrong input type is passed in for the test. + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## CohereTextVectorizer + + + +### *class* CohereTextVectorizer(model='embed-english-v3.0', api_config=None, dtype='float32', \*, dims=None) + +Bases: `BaseVectorizer` + +The CohereTextVectorizer class utilizes Cohere’s API to generate +embeddings for text data. + +This vectorizer is designed to interact with Cohere’s /embed API, +requiring an API key for authentication. The key can be provided +directly in the api_config dictionary or through the COHERE_API_KEY +environment variable. User must obtain an API key from Cohere’s website +([https://dashboard.cohere.com/](https://dashboard.cohere.com/)). Additionally, the cohere python +client must be installed with pip install cohere. + +The vectorizer supports only synchronous operations, allows for batch +processing of texts and flexibility in handling preprocessing tasks. + +```python +from redisvl.utils.vectorize import CohereTextVectorizer + +vectorizer = CohereTextVectorizer( + model="embed-english-v3.0", + api_config={"api_key": "your-cohere-api-key"} # OR set COHERE_API_KEY in your env +) +query_embedding = vectorizer.embed( + text="your input query text here", + input_type="search_query" +) +doc_embeddings = cohere.embed_many( + texts=["your document text", "more document text"], + input_type="search_document" +) +``` + +Initialize the Cohere vectorizer. + +Visit [https://cohere.ai/embed](https://cohere.ai/embed) to learn about embeddings. + +* **Parameters:** + * **model** (*str*) – Model to use for embedding. Defaults to ‘embed-english-v3.0’. + * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the API key. + Defaults to None. + * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. + Used when setting as_buffer=True in calls to embed() and embed_many(). + Defaults to ‘float32’. + * **dims** (*int* *|* *None*) +* **Raises:** + * **ImportError** – If the cohere library is not installed. + * **ValueError** – If the API key is not provided. + * **ValueError** – If an invalid dtype is provided. + +#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Embed a chunk of text using the Cohere Embeddings API. + +Must provide the embedding input_type as a kwarg to this method +that specifies the type of input you’re giving to the model. + +Supported input types: +: - `search_document`: Used for embeddings stored in a vector database for search use-cases. + - `search_query`: Used for embeddings of search queries run against a vector DB to find relevant documents. + - `classification`: Used for embeddings passed through a text classifier + - `clustering`: Used for the embeddings run through a clustering algorithm. + +When hydrating your Redis DB, the documents you want to search over +should be embedded with input_type= “search_document” and when you are +querying the database, you should set the input_type = “search query”. +If you want to use the embeddings for a classification or clustering +task downstream, you should set input_type= “classification” or +“clustering”. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. + * **input_type** (*str*) – Specifies the type of input passed to the model. + Required for embedding models v3 and higher. +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – In an invalid input_type is provided. + +#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) + +Embed many chunks of text using the Cohere Embeddings API. + +Must provide the embedding input_type as a kwarg to this method +that specifies the type of input you’re giving to the model. + +Supported input types: +: - `search_document`: Used for embeddings stored in a vector database for search use-cases. + - `search_query`: Used for embeddings of search queries run against a vector DB to find relevant documents. + - `classification`: Used for embeddings passed through a text classifier + - `clustering`: Used for the embeddings run through a clustering algorithm. + +When hydrating your Redis DB, the documents you want to search over +should be embedded with input_type= “search_document” and when you are +querying the database, you should set the input_type = “search query”. +If you want to use the embeddings for a classification or clustering +task downstream, you should set input_type= “classification” or +“clustering”. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. Defaults to 10. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. + * **input_type** (*str*) – Specifies the type of input passed to the model. + Required for embedding models v3 and higher. +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – In an invalid input_type is provided. + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## BedrockTextVectorizer + + + +### *class* BedrockTextVectorizer(model='amazon.titan-embed-text-v2:0', api_config=None, dtype='float32', \*, dims=None) + +Bases: `BaseVectorizer` + +The AmazonBedrockTextVectorizer class utilizes Amazon Bedrock’s API to generate +embeddings for text data. + +This vectorizer is designed to interact with Amazon Bedrock API, +requiring AWS credentials for authentication. The credentials can be provided +directly in the api_config dictionary or through environment variables: +- AWS_ACCESS_KEY_ID +- AWS_SECRET_ACCESS_KEY +- AWS_REGION (defaults to us-east-1) + +The vectorizer supports synchronous operations with batch processing and +preprocessing capabilities. + +```python +# Initialize with explicit credentials +vectorizer = AmazonBedrockTextVectorizer( + model="amazon.titan-embed-text-v2:0", + api_config={ + "aws_access_key_id": "your_access_key", + "aws_secret_access_key": "your_secret_key", + "aws_region": "us-east-1" + } +) + +# Initialize using environment variables +vectorizer = AmazonBedrockTextVectorizer() + +# Generate embeddings +embedding = vectorizer.embed("Hello, world!") +embeddings = vectorizer.embed_many(["Hello", "World"], batch_size=2) +``` + +Initialize the AWS Bedrock Vectorizer. + +* **Parameters:** + * **model** (*str*) – The Bedrock model ID to use. Defaults to amazon.titan-embed-text-v2:0 + * **api_config** (*Optional* *[**Dict* *[**str* *,* *str* *]* *]*) – AWS credentials and config. + Can include: aws_access_key_id, aws_secret_access_key, aws_region + If not provided, will use environment variables. + * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. + Used when setting as_buffer=True in calls to embed() and embed_many(). + Defaults to ‘float32’. + * **dims** (*int* *|* *None*) +* **Raises:** + * **ValueError** – If credentials are not provided in config or environment. + * **ImportError** – If boto3 is not installed. + * **ValueError** – If an invalid dtype is provided. + +#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Embed a chunk of text using Amazon Bedrock. + +* **Parameters:** + * **text** (*str*) – Text to embed. + * **preprocess** (*Optional* *[**Callable* *]*) – Optional preprocessing function. + * **as_buffer** (*bool*) – Whether to return as byte buffer. +* **Returns:** + The embedding vector. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If text is not a string. + +#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) + +Embed multiple texts using Amazon Bedrock. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of texts to embed. + * **preprocess** (*Optional* *[**Callable* *]*) – Optional preprocessing function. + * **batch_size** (*int*) – Size of batches for processing. + * **as_buffer** (*bool*) – Whether to return as byte buffers. +* **Returns:** + List of embedding vectors. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – If texts is not a list of strings. + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## CustomTextVectorizer + + + +### *class* CustomTextVectorizer(embed, embed_many=None, aembed=None, aembed_many=None, dtype='float32') + +Bases: `BaseVectorizer` + +The CustomTextVectorizer class wraps user-defined embedding methods to create +embeddings for text data. + +This vectorizer is designed to accept a provided callable text vectorizer and +provides a class definition to allow for compatibility with RedisVL. +The vectorizer may support both synchronous and asynchronous operations which +allows for batch processing of texts, but at a minimum only syncronous embedding +is required to satisfy the ‘embed()’ method. + +```python +# Synchronous embedding of a single text +vectorizer = CustomTextVectorizer( + embed = my_vectorizer.generate_embedding +) +embedding = vectorizer.embed("Hello, world!") + +# Asynchronous batch embedding of multiple texts +embeddings = await vectorizer.aembed_many( + ["Hello, world!", "How are you?"], + batch_size=2 +) +``` + +Initialize the Custom vectorizer. + +* **Parameters:** + * **embed** (*Callable*) – a Callable function that accepts a string object and returns a list of floats. + * **embed_many** (*Optional* *[**Callable*) – a Callable function that accepts a list of string objects and returns a list containing lists of floats. Defaults to None. + * **aembed** (*Optional* *[**Callable* *]*) – an asyncronous Callable function that accepts a string object and returns a lists of floats. Defaults to None. + * **aembed_many** (*Optional* *[**Callable* *]*) – an asyncronous Callable function that accepts a list of string objects and returns a list containing lists of floats. Defaults to None. + * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. + Used when setting as_buffer=True in calls to embed() and embed_many(). + Defaults to ‘float32’. +* **Raises:** + **ValueError** – if embedding validation fails. + +#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Generate an embedding for a single piece of text using your sync embed function. + +* **Parameters:** + * **text** (*str*) – The text to embed. + * **preprocess** (*Optional* *[**Callable* *]*) – An optional callable to preprocess the text. + * **as_buffer** (*bool*) – If True, return the embedding as a byte buffer. +* **Returns:** + The embedding of the input text. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If the input is not a string. + +#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) + +Generate embeddings for multiple pieces of text in batches using your sync embed_many function. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – A list of texts to embed. + * **preprocess** (*Optional* *[**Callable* *]*) – Optional preprocessing for each text. + * **batch_size** (*int*) – Number of texts per batch. + * **as_buffer** (*bool*) – If True, convert each embedding to a byte buffer. +* **Returns:** + A list of embeddings, where each embedding is a list of floats. +* **Return type:** + List[List[float]] +* **Raises:** + * **TypeError** – If the input is not a list of strings. + * **NotImplementedError** – If no embed_many function was provided. + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. + +## VoyageAITextVectorizer + + + +### *class* VoyageAITextVectorizer(model='voyage-large-2', api_config=None, dtype='float32', \*, dims=None) + +Bases: `BaseVectorizer` + +The VoyageAITextVectorizer class utilizes VoyageAI’s API to generate +embeddings for text data. + +This vectorizer is designed to interact with VoyageAI’s /embed API, +requiring an API key for authentication. The key can be provided +directly in the api_config dictionary or through the VOYAGE_API_KEY +environment variable. User must obtain an API key from VoyageAI’s website +([https://dash.voyageai.com/](https://dash.voyageai.com/)). Additionally, the voyageai python +client must be installed with pip install voyageai. + +The vectorizer supports both synchronous and asynchronous operations, allows for batch +processing of texts and flexibility in handling preprocessing tasks. + +```python +from redisvl.utils.vectorize import VoyageAITextVectorizer + +vectorizer = VoyageAITextVectorizer( + model="voyage-large-2", + api_config={"api_key": "your-voyageai-api-key"} # OR set VOYAGE_API_KEY in your env +) +query_embedding = vectorizer.embed( + text="your input query text here", + input_type="query" +) +doc_embeddings = vectorizer.embed_many( + texts=["your document text", "more document text"], + input_type="document" +) +``` + +Initialize the VoyageAI vectorizer. + +Visit [https://docs.voyageai.com/docs/embeddings](https://docs.voyageai.com/docs/embeddings) to learn about embeddings and check the available models. + +* **Parameters:** + * **model** (*str*) – Model to use for embedding. Defaults to “voyage-large-2”. + * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the API key. + Defaults to None. + * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. + Used when setting as_buffer=True in calls to embed() and embed_many(). + Defaults to ‘float32’. + * **dims** (*int* *|* *None*) +* **Raises:** + * **ImportError** – If the voyageai library is not installed. + * **ValueError** – If the API key is not provided. + +#### aembed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Embed a chunk of text using the VoyageAI Embeddings API. + +Can provide the embedding input_type as a kwarg to this method +that specifies the type of input you’re giving to the model. For retrieval/search use cases, +we recommend specifying this argument when encoding queries or documents to enhance retrieval quality. +Embeddings generated with and without the input_type argument are compatible. + +Supported input types are `document` and `query` + +When hydrating your Redis DB, the documents you want to search over +should be embedded with input_type=”document” and when you are +querying the database, you should set the input_type=”query”. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. + * **input_type** (*str*) – Specifies the type of input passed to the model. + * **truncation** (*bool*) – Whether to truncate the input texts to fit within the context length. + Check [https://docs.voyageai.com/docs/embeddings](https://docs.voyageai.com/docs/embeddings) +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – In an invalid input_type is provided. + +#### aembed_many(texts, preprocess=None, batch_size=None, as_buffer=False, \*\*kwargs) + +Embed many chunks of text using the VoyageAI Embeddings API. + +Can provide the embedding input_type as a kwarg to this method +that specifies the type of input you’re giving to the model. For retrieval/search use cases, +we recommend specifying this argument when encoding queries or documents to enhance retrieval quality. +Embeddings generated with and without the input_type argument are compatible. + +Supported input types are `document` and `query` + +When hydrating your Redis DB, the documents you want to search over +should be embedded with input_type=”document” and when you are +querying the database, you should set the input_type=”query”. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. . + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. + * **input_type** (*str*) – Specifies the type of input passed to the model. + * **truncation** (*bool*) – Whether to truncate the input texts to fit within the context length. + Check [https://docs.voyageai.com/docs/embeddings](https://docs.voyageai.com/docs/embeddings) +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – In an invalid input_type is provided. + +#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) + +Embed a chunk of text using the VoyageAI Embeddings API. + +Can provide the embedding input_type as a kwarg to this method +that specifies the type of input you’re giving to the model. For retrieval/search use cases, +we recommend specifying this argument when encoding queries or documents to enhance retrieval quality. +Embeddings generated with and without the input_type argument are compatible. + +Supported input types are `document` and `query` + +When hydrating your Redis DB, the documents you want to search over +should be embedded with input_type=”document” and when you are +querying the database, you should set the input_type=”query”. + +* **Parameters:** + * **text** (*str*) – Chunk of text to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. + * **input_type** (*str*) – Specifies the type of input passed to the model. + * **truncation** (*bool*) – Whether to truncate the input texts to fit within the context length. + Check [https://docs.voyageai.com/docs/embeddings](https://docs.voyageai.com/docs/embeddings) +* **Returns:** + Embedding. +* **Return type:** + List[float] +* **Raises:** + **TypeError** – If an invalid input_type is provided. + +#### embed_many(texts, preprocess=None, batch_size=None, as_buffer=False, \*\*kwargs) + +Embed many chunks of text using the VoyageAI Embeddings API. + +Can provide the embedding input_type as a kwarg to this method +that specifies the type of input you’re giving to the model. For retrieval/search use cases, +we recommend specifying this argument when encoding queries or documents to enhance retrieval quality. +Embeddings generated with and without the input_type argument are compatible. + +Supported input types are `document` and `query` + +When hydrating your Redis DB, the documents you want to search over +should be embedded with input_type=”document” and when you are +querying the database, you should set the input_type=”query”. + +* **Parameters:** + * **texts** (*List* *[**str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + perform before vectorization. Defaults to None. + * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating + embeddings. . + * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding + to a byte string. Defaults to False. + * **input_type** (*str*) – Specifies the type of input passed to the model. + * **truncation** (*bool*) – Whether to truncate the input texts to fit within the context length. + Check [https://docs.voyageai.com/docs/embeddings](https://docs.voyageai.com/docs/embeddings) +* **Returns:** + List of embeddings. +* **Return type:** + List[List[float]] +* **Raises:** + **TypeError** – If an invalid input_type is provided. + +#### model_post_init(context, /) + +This function is meant to behave like a BaseModel method to initialise private attributes. + +It takes context as an argument since that’s what pydantic-core passes when calling it. + +* **Parameters:** + * **self** (*BaseModel*) – The BaseModel instance. + * **context** (*Any*) – The context. +* **Return type:** + None + +#### model_config *: ClassVar[ConfigDict]* *= {}* + +Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. diff --git a/content/integrate/redisvl/overview/_index.md b/content/integrate/redisvl/overview/_index.md new file mode 100644 index 0000000000..0e63a665d7 --- /dev/null +++ b/content/integrate/redisvl/overview/_index.md @@ -0,0 +1,18 @@ +--- +linkTitle: Overview +title: Overview +type: integration +--- + + + + +* [Install RedisVL](installation/) + * [Install RedisVL with Pip](installation/#install-redisvl-with-pip) + * [Install RedisVL from Source](installation/#install-redisvl-from-source) + * [Installing Redis](installation/#installing-redis) +* [The RedisVL CLI](cli/) + * [Commands](cli/#commands) + * [Index](cli/#index) + * [Stats](cli/#stats) + * [Optional arguments](cli/#optional-arguments) diff --git a/content/integrate/redisvl/overview/cli.md b/content/integrate/redisvl/overview/cli.md new file mode 100644 index 0000000000..1d1653aeda --- /dev/null +++ b/content/integrate/redisvl/overview/cli.md @@ -0,0 +1,236 @@ +--- +linkTitle: The RedisVL CLI +title: The RedisVL CLI +type: integration +--- + + +RedisVL is a Python library with a dedicated CLI to help load and create vector search indices within Redis. + +This notebook will walk through how to use the Redis Vector Library CLI (``rvl``). + +Before running this notebook, be sure to +1. Have installed ``redisvl`` and have that environment active for this notebook. +2. Have a running Redis instance with the Search and Query capability + + +```python +# First, see if the rvl tool is installed +!rvl version +``` + + 18:12:25 [RedisVL] INFO RedisVL version 0.3.9 + + +## Commands +Here's a table of all the rvl commands and options. We'll go into each one in detail below. + +| Command | Options | Description | +|---------------|--------------------------|-------------| +| `rvl version` | | display the redisvl library version| +| `rvl index` | `create --schema` or `-s `| create a redis index from the specified schema file| +| `rvl index` | `listall` | list all the existing search indices| +| `rvl index` | `info --index` or ` -i ` | display the index definition in tabular format| +| `rvl index` | `delete --index` or `-i ` | remove the specified index, leaving the data still in Redis| +| `rvl index` | `destroy --index` or `-i `| remove the specified index, as well as the associated data| +| `rvl stats` | `--index` or `-i ` | display the index statistics, including number of docs, average bytes per record, indexing time, etc| +| `rvl stats` | `--schema` or `-s ` | display the index statistics of a schema defined in . The index must have already been created within Redis| + +## Index + +The ``rvl index`` command can be used for a number of tasks related to creating and managing indices. Whether you are working in Python or another language, this cli tool can still be useful for managing and inspecting your indices. + +First, we will create an index from a yaml schema that looks like the following: + + + +```python +%%writefile schema.yaml + +version: '0.1.0' + +index: + name: vectorizers + prefix: doc + storage_type: hash + +fields: + - name: sentence + type: text + - name: embedding + type: vector + attrs: + dims: 768 + algorithm: flat + distance_metric: cosine +``` + + Overwriting schema.yaml + + + +```python +# Create an index from a yaml schema +!rvl index create -s schema.yaml +``` + + 18:12:32 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + 18:12:32 [RedisVL] INFO Index created successfully + + + +```python +# list the indices that are available +!rvl index listall +``` + + 18:12:35 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + 18:12:35 [RedisVL] INFO Indices: + 18:12:35 [RedisVL] INFO 1. vectorizers + + + +```python +# inspect the index fields +!rvl index info -i vectorizers +``` + + 18:12:37 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + + + Index Information: + ╭──────────────┬────────────────┬────────────┬─────────────────┬────────────╮ + │ Index Name │ Storage Type │ Prefixes │ Index Options │ Indexing │ + ├──────────────┼────────────────┼────────────┼─────────────────┼────────────┤ + │ vectorizers │ HASH │ ['doc'] │ [] │ 0 │ + ╰──────────────┴────────────────┴────────────┴─────────────────┴────────────╯ + Index Fields: + ╭───────────┬─────────────┬────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬─────────────────┬────────────────╮ + │ Name │ Attribute │ Type │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ + ├───────────┼─────────────┼────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼─────────────────┼────────────────┤ + │ sentence │ sentence │ TEXT │ WEIGHT │ 1 │ │ │ │ │ │ │ + │ embedding │ embedding │ VECTOR │ algorithm │ FLAT │ data_type │ FLOAT32 │ dim │ 768 │ distance_metric │ COSINE │ + ╰───────────┴─────────────┴────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴─────────────────┴────────────────╯ + + + +```python +# delete an index without deleting the data within it +!rvl index delete -i vectorizers +``` + + 18:12:40 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + 18:12:40 [RedisVL] INFO Index deleted successfully + + + +```python +# see the indices that still exist +!rvl index listall +``` + + 18:12:43 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + 18:12:43 [RedisVL] INFO Indices: + + +## Stats + +The ``rvl stats`` command will return some basic information about the index. This is useful for checking the status of an index, or for getting information about the index to use in other commands. + + +```python +# create a new index with the same schema +# recreating the index will reindex the documents +!rvl index create -s schema.yaml +``` + + 18:13:21 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + 18:13:21 redisvl.index.index INFO Index already exists, not overwriting. + 18:13:21 [RedisVL] INFO Index created successfully + + + +```python +# list the indices that are available +!rvl index listall +``` + + 18:13:25 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + 18:13:25 [RedisVL] INFO Indices: + 18:13:25 [RedisVL] INFO 1. vectorizers + + + +```python +# see all the stats for the index +!rvl stats -i vectorizers +``` + + 18:13:31 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + + Statistics: + ╭─────────────────────────────┬─────────╮ + │ Stat Key │ Value │ + ├─────────────────────────────┼─────────┤ + │ num_docs │ 0 │ + │ num_terms │ 0 │ + │ max_doc_id │ 0 │ + │ num_records │ 0 │ + │ percent_indexed │ 1 │ + │ hash_indexing_failures │ 3 │ + │ number_of_uses │ 2 │ + │ bytes_per_record_avg │ nan │ + │ doc_table_size_mb │ 0 │ + │ inverted_sz_mb │ 0 │ + │ key_table_size_mb │ 0 │ + │ offset_bits_per_record_avg │ nan │ + │ offset_vectors_sz_mb │ 0 │ + │ offsets_per_term_avg │ nan │ + │ records_per_doc_avg │ nan │ + │ sortable_values_size_mb │ 0 │ + │ total_indexing_time │ 0.02 │ + │ total_inverted_index_blocks │ 0 │ + │ vector_index_sz_mb │ 0 │ + ╰─────────────────────────────┴─────────╯ + + +## Optional arguments +You can modify these commands with the below optional arguments + +| Argument | Description | Default | +|----------------|-------------|---------| +| `-u --url` | The full Redis URL to connec to | `redis://localhost:6379` | +| `--host` | Redis host to connect to | `localhost` | +| `-p --port` | Redis port to connect to. Must be an integer | `6379` | +| `--user` | Redis username, if one is required | `default` | +| `--ssl` | Boolean flag indicating if ssl is required. If set the Redis base url changes to `rediss://` | None | +| `-a --password`| Redis password, if one is required| `""` | + +### Choosing your Redis instance +By default rvl first checks if you have `REDIS_URL` environment variable defined and tries to connect to that. If not, it then falls back to `localhost:6379`, unless you pass the `--host` or `--port` arguments + + +```python +# specify your Redis instance to connect to +!rvl index listall --host localhost --port 6379 +``` + + 18:13:36 [RedisVL] INFO Using Redis address from environment variable, REDIS_URL + 18:13:36 [RedisVL] INFO Indices: + 18:13:36 [RedisVL] INFO 1. vectorizers + + +### Using SSL encription +If your Redis instance is configured to use SSL encription then set the `--ssl` flag. +You can similarly specify the username and password to construct the full Redis URL + + +```python +# connect to rediss://jane_doe:password123@localhost:6379 +!rvl index listall --user jane_doe -a password123 --ssl +``` + + +```python +!rvl index destroy -i vectorizers +``` diff --git a/content/integrate/redisvl/overview/installation.md b/content/integrate/redisvl/overview/installation.md new file mode 100644 index 0000000000..271d294c3a --- /dev/null +++ b/content/integrate/redisvl/overview/installation.md @@ -0,0 +1,80 @@ +--- +linkTitle: Install RedisVL +title: Install RedisVL +type: integration +--- + + +There are a few ways to install RedisVL. The easiest way is to use pip. + +## Install RedisVL with Pip + +Install `redisvl` into your Python (>=3.8) environment using `pip`: + +```bash +$ pip install -U redisvl +``` + +RedisVL comes with a few dependencies that are automatically installed, however, a few dependencies +are optional and can be installed separately if needed: + +```bash +$ pip install redisvl[all] # install vectorizer dependencies +$ pip install redisvl[dev] # install dev dependencies +``` + +If you use ZSH, remember to escape the brackets: + +```bash +$ pip install redisvl\[all\] +``` + +This library supports the use of hiredis, so you can also install by running: + +```bash +pip install redisvl[hiredis] +``` + +## Install RedisVL from Source + +To install RedisVL from source, clone the repository and install the package using `pip`: + +```bash +$ git clone https://github.com/redis/redis-vl-python.git && cd redisvl +$ pip install . + +# or for an editable installation (for developers of RedisVL) +$ pip install -e . +``` + +## Installing Redis + +RedisVL requires a distribution of Redis that supports the [Search and Query](https://redis.com/modules/redis-search/) capability of which there are 3: + +offering + +1. [Redis Cloud](https://redis.io/cloud), a fully managed cloud offering +2. [Redis Stack](https://redis.io/docs/getting-started/install-stack/docker/), a local docker image for testing and development +3. [Redis Enterprise](https://redis.com/redis-enterprise/), a commercial self-hosted + +### Redis Cloud + +Redis Cloud is the easiest way to get started with RedisVL. You can sign up for a free account [here](https://redis.io/cloud). Make sure to have the `Search and Query` +capability enabled when creating your database. + +### Redis Stack (local development) + +For local development and testing, Redis-Stack can be used. We recommend running Redis +in a docker container. To do so, run the following command: + +```bash +docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest +``` + +This will also spin up the [Redis Insight GUI](https://redis.io/insight/) at `http://localhost:8001`. + +### Redis Enterprise (self-hosted) + +Redis Enterprise is a commercial offering that can be self-hosted. You can download the latest version [here](https://redis.io/downloads/). + +If you are considering a self-hosted Redis Enterprise deployment on Kubernetes, there is the [Redis Enterprise Operator](https://docs.redis.com/latest/kubernetes/) for Kubernetes. This will allow you to easily deploy and manage a Redis Enterprise cluster on Kubernetes. diff --git a/content/integrate/redisvl/user_guide/01_getting_started.md b/content/integrate/redisvl/user_guide/01_getting_started.md new file mode 100644 index 0000000000..739cf87890 --- /dev/null +++ b/content/integrate/redisvl/user_guide/01_getting_started.md @@ -0,0 +1,482 @@ +--- +linkTitle: Getting Started with RedisVL +title: Getting Started with RedisVL +type: integration +--- + +`redisvl` is a versatile Python library with an integrated CLI, designed to enhance AI applications using Redis. This guide will walk you through the following steps: + +1. Defining an `IndexSchema` +2. Preparing a sample dataset +3. Creating a `SearchIndex` object +4. Testing `rvl` CLI functionality +5. Loading the sample data +6. Building `VectorQuery` objects and executing searches +7. Updating a `SearchIndex` object + +...and more! + +Prerequisites: +- Ensure `redisvl` is installed in your Python environment. +- Have a running instance of [Redis Stack](https://redis.io/docs/install/install-stack/) or [Redis Cloud](https://redis.io/cloud). + +_____ + +## Define an `IndexSchema` + +The `IndexSchema` maintains crucial **index configuration** and **field definitions** to +enable search with Redis. For ease of use, the schema can be constructed from a +python dictionary or yaml file. + +### Example Schema Creation +Consider a dataset with user information, including `job`, `age`, `credit_score`, +and a 3-dimensional `user_embedding` vector. + +You must also decide on a Redis index name and key prefix to use for this +dataset. Below are example schema definitions in both YAML and Dict format. + +**YAML Definition:** + +```yaml +version: '0.1.0' + +index: + name: user_simple + prefix: user_simple_docs + +fields: + - name: user + type: tag + - name: credit_score + type: tag + - name: job + type: text + - name: age + type: numeric + - name: user_embedding + type: vector + attrs: + algorithm: flat + dims: 3 + distance_metric: cosine + datatype: float32 +``` +Store this in a local file, such as `schema.yaml`, for RedisVL usage. + +**Python Dictionary:** + + +```python +schema = { + "index": { + "name": "user_simple", + "prefix": "user_simple_docs", + }, + "fields": [ + {"name": "user", "type": "tag"}, + {"name": "credit_score", "type": "tag"}, + {"name": "job", "type": "text"}, + {"name": "age", "type": "numeric"}, + { + "name": "user_embedding", + "type": "vector", + "attrs": { + "dims": 3, + "distance_metric": "cosine", + "algorithm": "flat", + "datatype": "float32" + } + } + ] +} +``` + +## Sample Dataset Preparation + +Below, create a mock dataset with `user`, `job`, `age`, `credit_score`, and +`user_embedding` fields. The `user_embedding` vectors are synthetic examples +for demonstration purposes. + +For more information on creating real-world embeddings, refer to this +[article](https://mlops.community/vector-similarity-search-from-basics-to-production/). + + +```python +import numpy as np + + +data = [ + { + 'user': 'john', + 'age': 1, + 'job': 'engineer', + 'credit_score': 'high', + 'user_embedding': np.array([0.1, 0.1, 0.5], dtype=np.float32).tobytes() + }, + { + 'user': 'mary', + 'age': 2, + 'job': 'doctor', + 'credit_score': 'low', + 'user_embedding': np.array([0.1, 0.1, 0.5], dtype=np.float32).tobytes() + }, + { + 'user': 'joe', + 'age': 3, + 'job': 'dentist', + 'credit_score': 'medium', + 'user_embedding': np.array([0.9, 0.9, 0.1], dtype=np.float32).tobytes() + } +] +``` + +As seen above, the sample `user_embedding` vectors are converted into bytes. Using the `NumPy`, this is fairly trivial. + +## Create a `SearchIndex` + +With the schema and sample dataset ready, instantiate a `SearchIndex`: + + +```python +from redisvl.index import SearchIndex + +index = SearchIndex.from_dict(schema) +# or use .from_yaml('schema_file.yaml') +``` + +Now we also need to facilitate a Redis connection. There are a few ways to do this: + +- Create & manage your own client connection (recommended) +- Provide a simple Redis URL and let RedisVL connect on your behalf + +### Bring your own Redis connection instance + +This is ideal in scenarios where you have custom settings on the connection instance or if your application will share a connection pool: + + +```python +from redis import Redis + +client = Redis.from_url("redis://localhost:6379") + +index.set_client(client) +# optionally provide an async Redis client object to enable async index operations +``` + + + + + + + + +### Let the index manage the connection instance + +This is ideal for simple cases: + + +```python +index.connect("redis://localhost:6379") +# optionally use an async client by passing use_async=True +``` + + + + + + + + +### Create the underlying index + +Now that we are connected to Redis, we need to run the create command. + + +```python +index.create(overwrite=True) +``` + +Note that at this point, the index has no entries. Data loading follows. + +## Inspect with the `rvl` CLI +Use the `rvl` CLI to inspect the created index and its fields: + + +```python +!rvl index listall +``` + + 11:50:15 [RedisVL] INFO Indices: + 11:50:15 [RedisVL] INFO 1. user_simple + + + +```python +!rvl index info -i user_simple +``` + + + + Index Information: + ╭──────────────┬────────────────┬──────────────────────┬─────────────────┬────────────╮ + │ Index Name │ Storage Type │ Prefixes │ Index Options │ Indexing │ + ├──────────────┼────────────────┼──────────────────────┼─────────────────┼────────────┤ + │ user_simple │ HASH │ ['user_simple_docs'] │ [] │ 0 │ + ╰──────────────┴────────────────┴──────────────────────┴─────────────────┴────────────╯ + Index Fields: + ╭────────────────┬────────────────┬─────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬─────────────────┬────────────────╮ + │ Name │ Attribute │ Type │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ + ├────────────────┼────────────────┼─────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼─────────────────┼────────────────┤ + │ user │ user │ TAG │ SEPARATOR │ , │ │ │ │ │ │ │ + │ credit_score │ credit_score │ TAG │ SEPARATOR │ , │ │ │ │ │ │ │ + │ job │ job │ TEXT │ WEIGHT │ 1 │ │ │ │ │ │ │ + │ age │ age │ NUMERIC │ │ │ │ │ │ │ │ │ + │ user_embedding │ user_embedding │ VECTOR │ algorithm │ FLAT │ data_type │ FLOAT32 │ dim │ 3 │ distance_metric │ COSINE │ + ╰────────────────┴────────────────┴─────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴─────────────────┴────────────────╯ + + +## Load Data to `SearchIndex` + +Load the sample dataset to Redis: + + +```python +keys = index.load(data) + +print(keys) +``` + + ['user_simple_docs:01JM2NWFWNH0BNA640MT5DS8BD', 'user_simple_docs:01JM2NWFWNF4S2V4E4HYG25CVA', 'user_simple_docs:01JM2NWFWNBFXJJ4PV9F4KMJSE'] + + +By default, `load` will create a unique Redis key as a combination of the index key `prefix` and a random ULID. You can also customize the key by providing direct keys or pointing to a specified `id_field` on load. + +### Upsert the index with new data +Upsert data by using the `load` method again: + + +```python +# Add more data +new_data = [{ + 'user': 'tyler', + 'age': 9, + 'job': 'engineer', + 'credit_score': 'high', + 'user_embedding': np.array([0.1, 0.3, 0.5], dtype=np.float32).tobytes() +}] +keys = index.load(new_data) + +print(keys) +``` + + ['user_simple_docs:01JM2NWJGYMJ0QTR5YB4MB0BX9'] + + +## Creating `VectorQuery` Objects + +Next we will create a vector query object for our newly populated index. This example will use a simple vector to demonstrate how vector similarity works. Vectors in production will likely be much larger than 3 floats and often require Machine Learning models (i.e. Huggingface sentence transformers) or an embeddings API (Cohere, OpenAI). `redisvl` provides a set of [Vectorizers](https://docs.redisvl.com/en/latest/user_guide/vectorizers_04.html#openai) to assist in vector creation. + + +```python +from redisvl.query import VectorQuery +from jupyterutils import result_print + +query = VectorQuery( + vector=[0.1, 0.1, 0.5], + vector_field_name="user_embedding", + return_fields=["user", "age", "job", "credit_score", "vector_distance"], + num_results=3 +) +``` + +### Executing queries +With our `VectorQuery` object defined above, we can execute the query over the `SearchIndex` using the `query` method. + + +```python +results = index.query(query) +result_print(results) +``` + + +
vector_distanceuseragejobcredit_score
0john1engineerhigh
0mary2doctorlow
0.0566299557686tyler9engineerhigh
+ + +## Using an Asynchronous Redis Client + +The `AsyncSearchIndex` class along with an async Redis python client allows for queries, index creation, and data loading to be done asynchronously. This is the +recommended route for working with `redisvl` in production-like settings. + + +```python +schema +``` + + + + + {'index': {'name': 'user_simple', 'prefix': 'user_simple_docs'}, + 'fields': [{'name': 'user', 'type': 'tag'}, + {'name': 'credit_score', 'type': 'tag'}, + {'name': 'job', 'type': 'text'}, + {'name': 'age', 'type': 'numeric'}, + {'name': 'user_embedding', + 'type': 'vector', + 'attrs': {'dims': 3, + 'distance_metric': 'cosine', + 'algorithm': 'flat', + 'datatype': 'float32'}}]} + + + + +```python +from redisvl.index import AsyncSearchIndex +from redis.asyncio import Redis + +client = Redis.from_url("redis://localhost:6379") + +index = AsyncSearchIndex.from_dict(schema) +await index.set_client(client) +``` + + + + + + + + + +```python +# execute the vector query async +results = await index.query(query) +result_print(results) +``` + + +
vector_distanceuseragejobcredit_score
0john1engineerhigh
0mary2doctorlow
0.0566299557686tyler9engineerhigh
+ + +## Updating a schema +In some scenarios, it makes sense to update the index schema. With Redis and `redisvl`, this is easy because Redis can keep the underlying data in place while you change or make updates to the index configuration. + +So for our scenario, let's imagine we want to reindex this data in 2 ways: +- by using a `Tag` type for `job` field instead of `Text` +- by using an `hnsw` vector index for the `user_embedding` field instead of a `flat` vector index + + +```python +# Modify this schema to have what we want + +index.schema.remove_field("job") +index.schema.remove_field("user_embedding") +index.schema.add_fields([ + {"name": "job", "type": "tag"}, + { + "name": "user_embedding", + "type": "vector", + "attrs": { + "dims": 3, + "distance_metric": "cosine", + "algorithm": "hnsw", + "datatype": "float32" + } + } +]) +``` + + +```python +# Run the index update but keep underlying data in place +await index.create(overwrite=True, drop=False) +``` + + 11:53:25 redisvl.index.index INFO Index already exists, overwriting. + + + +```python +# Execute the vector query async +results = await index.query(query) +result_print(results) +``` + + +
vector_distanceuseragejobcredit_score
0john1engineerhigh
0mary2doctorlow
0.0566299557686tyler9engineerhigh
+ + +## Check Index Stats +Use the `rvl` CLI to check the stats for the index: + + +```python +!rvl stats -i user_simple +``` + + + Statistics: + ╭─────────────────────────────┬─────────────╮ + │ Stat Key │ Value │ + ├─────────────────────────────┼─────────────┤ + │ num_docs │ 4 │ + │ num_terms │ 4 │ + │ max_doc_id │ 4 │ + │ num_records │ 22 │ + │ percent_indexed │ 1 │ + │ hash_indexing_failures │ 0 │ + │ number_of_uses │ 5 │ + │ bytes_per_record_avg │ 50.9091 │ + │ doc_table_size_mb │ 0.000423431 │ + │ inverted_sz_mb │ 0.00106812 │ + │ key_table_size_mb │ 0.000165939 │ + │ offset_bits_per_record_avg │ 8 │ + │ offset_vectors_sz_mb │ 5.72205e-06 │ + │ offsets_per_term_avg │ 0.272727 │ + │ records_per_doc_avg │ 5.5 │ + │ sortable_values_size_mb │ 0 │ + │ total_indexing_time │ 0.197 │ + │ total_inverted_index_blocks │ 12 │ + │ vector_index_sz_mb │ 0.0201416 │ + ╰─────────────────────────────┴─────────────╯ + + +## Cleanup + +Below we will clean up after our work. First, you can flush all data from Redis associated with the index by +using the `.clear()` method. This will leave the secondary index in place for future insertions or updates. + +But if you want to clean up everything, including the index, just use `.delete()` +which will by default remove the index AND the underlying data. + + +```python +# Clear all data from Redis associated with the index +await index.clear() +``` + + + + + 4 + + + + +```python +# Butm the index is still in place +await index.exists() +``` + + + + + True + + + + +```python +# Remove / delete the index in its entirety +await index.delete() +``` diff --git a/content/integrate/redisvl/user_guide/02_hybrid_queries.md b/content/integrate/redisvl/user_guide/02_hybrid_queries.md new file mode 100644 index 0000000000..205e95cae7 --- /dev/null +++ b/content/integrate/redisvl/user_guide/02_hybrid_queries.md @@ -0,0 +1,727 @@ +--- +linkTitle: Querying with RedisVL +title: Querying with RedisVL +type: integration +--- + + +In this notebook, we will explore more complex queries that can be performed with ``redisvl`` + +Before running this notebook, be sure to +1. Have installed ``redisvl`` and have that environment active for this notebook. +2. Have a running Redis instance with RediSearch > 2.4 running. + + +```python +import pickle +from jupyterutils import table_print, result_print + +# load in the example data and printing utils +data = pickle.load(open("hybrid_example_data.pkl", "rb")) +table_print(data) +``` + + +
useragejobcredit_scoreoffice_locationuser_embedding
john18engineerhigh-122.4194,37.7749b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'
derrick14doctorlow-122.4194,37.7749b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'
nancy94doctorhigh-122.4194,37.7749b'333?\xcd\xcc\xcc=\x00\x00\x00?'
tyler100engineerhigh-122.0839,37.3861b'\xcd\xcc\xcc=\xcd\xcc\xcc>\x00\x00\x00?'
tim12dermatologisthigh-122.0839,37.3861b'\xcd\xcc\xcc>\xcd\xcc\xcc>\x00\x00\x00?'
taimur15CEOlow-122.0839,37.3861b'\x9a\x99\x19?\xcd\xcc\xcc=\x00\x00\x00?'
joe35dentistmedium-122.0839,37.3861b'fff?fff?\xcd\xcc\xcc='
+ + + +```python +schema = { + "index": { + "name": "user_queries", + "prefix": "user_queries_docs", + "storage_type": "hash", # default setting -- HASH + }, + "fields": [ + {"name": "user", "type": "tag"}, + {"name": "credit_score", "type": "tag"}, + {"name": "job", "type": "text"}, + {"name": "age", "type": "numeric"}, + {"name": "office_location", "type": "geo"}, + { + "name": "user_embedding", + "type": "vector", + "attrs": { + "dims": 3, + "distance_metric": "cosine", + "algorithm": "flat", + "datatype": "float32" + } + + } + ], +} +``` + + +```python +from redisvl.index import SearchIndex + +# construct a search index from the schema +index = SearchIndex.from_dict(schema) + +# connect to local redis instance +index.connect("redis://localhost:6379") + +# create the index (no data yet) +index.create(overwrite=True) +``` + + +```python +# use the CLI to see the created index +!rvl index listall +``` + + 14:16:51 [RedisVL] INFO Indices: + 14:16:51 [RedisVL] INFO 1. user_queries + + + +```python +# load data to redis +keys = index.load(data) +``` + +## Hybrid Queries + +Hybrid queries are queries that combine multiple types of filters. For example, you may want to search for a user that is a certain age, has a certain job, and is within a certain distance of a location. This is a hybrid query that combines numeric, tag, and geographic filters. + +### Tag Filters + +Tag filters are filters that are applied to tag fields. These are fields that are not tokenized and are used to store a single categorical value. + + +```python +from redisvl.query import VectorQuery +from redisvl.query.filter import Tag + +t = Tag("credit_score") == "high" + +v = VectorQuery( + vector=[0.1, 0.1, 0.5], + vector_field_name="user_embedding", + return_fields=["user", "credit_score", "age", "job", "office_location"], + filter_expression=t +) + +results = index.query(v) +result_print(results) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
+ + + +```python +# negation +t = Tag("credit_score") != "high" + +v.set_filter(t) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0derricklow14doctor-122.4194,37.7749
0.217882037163taimurlow15CEO-122.0839,37.3861
0.653301358223joemedium35dentist-122.0839,37.3861
+ + + +```python +# use multiple tags as a list +t = Tag("credit_score") == ["high", "medium"] + +v.set_filter(t) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
0.653301358223joemedium35dentist-122.0839,37.3861
+ + + +```python +# use multiple tags as a set (to enforce uniqueness) +t = Tag("credit_score") == set(["high", "high", "medium"]) + +v.set_filter(t) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
0.653301358223joemedium35dentist-122.0839,37.3861
+ + +What about scenarios where you might want to dynamically generate a list of tags? Have no fear. RedisVL allows you to do this gracefully without having to check for the **empty case**. The **empty case** is when you attempt to run a Tag filter on a field with no defined values to match: + +`Tag("credit_score") == []` + +An empty filter like the one above will yield a `*` Redis query filter which implies the base case -- there is no filter here to use. + + +```python +# gracefully fallback to "*" filter if empty case +empty_case = Tag("credit_score") == [] + +v.set_filter(empty_case) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0derricklow14doctor-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.217882037163taimurlow15CEO-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
0.653301358223joemedium35dentist-122.0839,37.3861
+ + +### Numeric Filters + +Numeric filters are filters that are applied to numeric fields and can be used to isolate a range of values for a given field. + + +```python +from redisvl.query.filter import Num + +numeric_filter = Num("age") > 15 + +v.set_filter(numeric_filter) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
0.653301358223joemedium35dentist-122.0839,37.3861
+ + + +```python +# exact match query +numeric_filter = Num("age") == 14 + +v.set_filter(numeric_filter) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0derricklow14doctor-122.4194,37.7749
+ + + +```python +# negation +numeric_filter = Num("age") != 14 + +v.set_filter(numeric_filter) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.217882037163taimurlow15CEO-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
0.653301358223joemedium35dentist-122.0839,37.3861
+ + +### Text Filters + +Text filters are filters that are applied to text fields. These filters are applied to the entire text field. For example, if you have a text field that contains the text "The quick brown fox jumps over the lazy dog", a text filter of "quick" will match this text field. + + +```python +from redisvl.query.filter import Text + +# exact match filter -- document must contain the exact word doctor +text_filter = Text("job") == "doctor" + +v.set_filter(text_filter) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0derricklow14doctor-122.4194,37.7749
0.266666650772nancyhigh94doctor-122.4194,37.7749
+ + + +```python +# negation -- document must not contain the exact word doctor +negate_text_filter = Text("job") != "doctor" + +v.set_filter(negate_text_filter) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.217882037163taimurlow15CEO-122.0839,37.3861
0.653301358223joemedium35dentist-122.0839,37.3861
+ + + +```python +# wildcard match filter +wildcard_filter = Text("job") % "doct*" + +v.set_filter(wildcard_filter) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0derricklow14doctor-122.4194,37.7749
0.266666650772nancyhigh94doctor-122.4194,37.7749
+ + + +```python +# fuzzy match filter +fuzzy_match = Text("job") % "%%engine%%" + +v.set_filter(fuzzy_match) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
+ + + +```python +# conditional -- match documents with job field containing engineer OR doctor +conditional = Text("job") % "engineer|doctor" + +v.set_filter(conditional) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0derricklow14doctor-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
+ + + +```python +# gracefully fallback to "*" filter if empty case +empty_case = Text("job") % "" + +v.set_filter(empty_case) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0derricklow14doctor-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.217882037163taimurlow15CEO-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
0.653301358223joemedium35dentist-122.0839,37.3861
+ + +Use raw query strings as input. Below we use the `~` flag to indicate that the full text query is optional. We also choose the BM25 scorer and return document scores along with the result. + + +```python +v.set_filter("(~(@job:engineer))") +v.scorer("BM25").with_scores() + +index.query(v) +``` + + + + + [{'id': 'user_queries_docs:409ff48274724984ba14865db0495fc5', + 'score': 0.9090908893868948, + 'vector_distance': '0', + 'user': 'john', + 'credit_score': 'high', + 'age': '18', + 'job': 'engineer', + 'office_location': '-122.4194,37.7749'}, + {'id': 'user_queries_docs:69cb262c303a4147b213dfdec8bd4b01', + 'score': 0.0, + 'vector_distance': '0', + 'user': 'derrick', + 'credit_score': 'low', + 'age': '14', + 'job': 'doctor', + 'office_location': '-122.4194,37.7749'}, + {'id': 'user_queries_docs:562263669ff74a0295c515018d151d7b', + 'score': 0.9090908893868948, + 'vector_distance': '0.109129190445', + 'user': 'tyler', + 'credit_score': 'high', + 'age': '100', + 'job': 'engineer', + 'office_location': '-122.0839,37.3861'}, + {'id': 'user_queries_docs:94176145f9de4e288ca2460cd5d1188e', + 'score': 0.0, + 'vector_distance': '0.158808946609', + 'user': 'tim', + 'credit_score': 'high', + 'age': '12', + 'job': 'dermatologist', + 'office_location': '-122.0839,37.3861'}, + {'id': 'user_queries_docs:d0bcf6842862410583901004b6b3aeba', + 'score': 0.0, + 'vector_distance': '0.217882037163', + 'user': 'taimur', + 'credit_score': 'low', + 'age': '15', + 'job': 'CEO', + 'office_location': '-122.0839,37.3861'}, + {'id': 'user_queries_docs:3dec0e9f2db04e19bff224c5a2a0ba3c', + 'score': 0.0, + 'vector_distance': '0.266666650772', + 'user': 'nancy', + 'credit_score': 'high', + 'age': '94', + 'job': 'doctor', + 'office_location': '-122.4194,37.7749'}, + {'id': 'user_queries_docs:93ee6c0e4ccb42f6b7af7858ea6a6408', + 'score': 0.0, + 'vector_distance': '0.653301358223', + 'user': 'joe', + 'credit_score': 'medium', + 'age': '35', + 'job': 'dentist', + 'office_location': '-122.0839,37.3861'}] + + + +### Geographic Filters + +Geographic filters are filters that are applied to geographic fields. These filters are used to find results that are within a certain distance of a given point. The distance is specified in kilometers, miles, meters, or feet. A radius can also be specified to find results within a certain radius of a given point. + + +```python +from redisvl.query.filter import Geo, GeoRadius + +# within 10 km of San Francisco office +geo_filter = Geo("office_location") == GeoRadius(-122.4194, 37.7749, 10, "km") + +v.set_filter(geo_filter) +result_print(index.query(v)) +``` + + +
scorevector_distanceusercredit_scoreagejoboffice_location
0.45454544469344740johnhigh18engineer-122.4194,37.7749
0.45454544469344740derricklow14doctor-122.4194,37.7749
0.45454544469344740.266666650772nancyhigh94doctor-122.4194,37.7749
+ + + +```python +# within 100 km Radius of San Francisco office +geo_filter = Geo("office_location") == GeoRadius(-122.4194, 37.7749, 100, "km") + +v.set_filter(geo_filter) +result_print(index.query(v)) +``` + + +
scorevector_distanceusercredit_scoreagejoboffice_location
0.45454544469344740johnhigh18engineer-122.4194,37.7749
0.45454544469344740derricklow14doctor-122.4194,37.7749
0.45454544469344740.109129190445tylerhigh100engineer-122.0839,37.3861
0.45454544469344740.158808946609timhigh12dermatologist-122.0839,37.3861
0.45454544469344740.217882037163taimurlow15CEO-122.0839,37.3861
0.45454544469344740.266666650772nancyhigh94doctor-122.4194,37.7749
0.45454544469344740.653301358223joemedium35dentist-122.0839,37.3861
+ + + +```python +# not within 10 km Radius of San Francisco office +geo_filter = Geo("office_location") != GeoRadius(-122.4194, 37.7749, 10, "km") + +v.set_filter(geo_filter) +result_print(index.query(v)) +``` + + +
scorevector_distanceusercredit_scoreagejoboffice_location
0.00.109129190445tylerhigh100engineer-122.0839,37.3861
0.00.158808946609timhigh12dermatologist-122.0839,37.3861
0.00.217882037163taimurlow15CEO-122.0839,37.3861
0.00.653301358223joemedium35dentist-122.0839,37.3861
+ + +## Combining Filters + +In this example, we will combine a numeric filter with a tag filter. We will search for users that are between the ages of 20 and 30 and have a job of "engineer". + +### Intersection ("and") + + +```python +t = Tag("credit_score") == "high" +low = Num("age") >= 18 +high = Num("age") <= 100 + +combined = t & low & high + +v = VectorQuery([0.1, 0.1, 0.5], + "user_embedding", + return_fields=["user", "credit_score", "age", "job", "office_location"], + filter_expression=combined) + + +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
+ + +### Union ("or") + +The union of two queries is the set of all results that are returned by either of the two queries. The union of two queries is performed using the `|` operator. + + +```python +low = Num("age") < 18 +high = Num("age") > 93 + +combined = low | high + +v.set_filter(combined) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0derricklow14doctor-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.217882037163taimurlow15CEO-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
+ + +### Dynamic Combination + +There are often situations where you may or may not want to use a filter in a +given query. As shown above, filters will except the ``None`` type and revert +to a wildcard filter essentially returning all results. + +The same goes for filter combinations which enables rapid reuse of filters in +requests with different parameters as shown below. This removes the need for +a number of "if-then" conditionals to test for the empty case. + + + + +```python +def make_filter(age=None, credit=None, job=None): + flexible_filter = ( + (Num("age") > age) & + (Tag("credit_score") == credit) & + (Text("job") % job) + ) + return flexible_filter + +``` + + +```python +# all parameters +combined = make_filter(age=18, credit="high", job="engineer") +v.set_filter(combined) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0.109129190445tylerhigh100engineer-122.0839,37.3861
+ + + +```python +# just age and credit_score +combined = make_filter(age=18, credit="high") +v.set_filter(combined) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
+ + + +```python +# just age +combined = make_filter(age=18) +v.set_filter(combined) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
0.653301358223joemedium35dentist-122.0839,37.3861
+ + + +```python +# no filters +combined = make_filter() +v.set_filter(combined) +result_print(index.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0derricklow14doctor-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
0.158808946609timhigh12dermatologist-122.0839,37.3861
0.217882037163taimurlow15CEO-122.0839,37.3861
0.266666650772nancyhigh94doctor-122.4194,37.7749
0.653301358223joemedium35dentist-122.0839,37.3861
+ + +## Non-vector Queries + +In some cases, you may not want to run a vector query, but just use a ``FilterExpression`` similar to a SQL query. The ``FilterQuery`` class enable this functionality. It is similar to the ``VectorQuery`` class but soley takes a ``FilterExpression``. + + +```python +from redisvl.query import FilterQuery + +has_low_credit = Tag("credit_score") == "low" + +filter_query = FilterQuery( + return_fields=["user", "credit_score", "age", "job", "location"], + filter_expression=has_low_credit +) + +results = index.query(filter_query) + +result_print(results) +``` + + +
usercredit_scoreagejob
derricklow14doctor
taimurlow15CEO
+ + +## Count Queries + +In some cases, you may need to use a ``FilterExpression`` to execute a ``CountQuery`` that simply returns the count of the number of entities in the pertaining set. It is similar to the ``FilterQuery`` class but does not return the values of the underlying data. + + +```python +from redisvl.query import CountQuery + +has_low_credit = Tag("credit_score") == "low" + +filter_query = CountQuery(filter_expression=has_low_credit) + +count = index.query(filter_query) + +print(f"{count} records match the filter expression {str(has_low_credit)} for the given index.") +``` + + 2 records match the filter expression @credit_score:{low} for the given index. + + +## Range Queries + +Range Queries are a useful method to perform a vector search where only results within a vector ``distance_threshold`` are returned. This enables the user to find all records within their dataset that are similar to a query vector where "similar" is defined by a quantitative value. + + +```python +from redisvl.query import RangeQuery + +range_query = RangeQuery( + vector=[0.1, 0.1, 0.5], + vector_field_name="user_embedding", + return_fields=["user", "credit_score", "age", "job", "location"], + distance_threshold=0.2 +) + +# same as the vector query or filter query +results = index.query(range_query) + +result_print(results) +``` + + +
vector_distanceusercredit_scoreagejob
0johnhigh18engineer
0derricklow14doctor
0.109129190445tylerhigh100engineer
0.158808946609timhigh12dermatologist
+ + +We can also change the distance threshold of the query object between uses if we like. Here we will set ``distance_threshold==0.1``. This means that the query object will return all matches that are within 0.1 of the query object. This is a small distance, so we expect to get fewer matches than before. + + +```python +range_query.set_distance_threshold(0.1) + +result_print(index.query(range_query)) +``` + + +
vector_distanceusercredit_scoreagejob
0johnhigh18engineer
0derricklow14doctor
+ + +Range queries can also be used with filters like any other query type. The following limits the results to only include records with a ``job`` of ``engineer`` while also being within the vector range (aka distance). + + +```python +is_engineer = Text("job") == "engineer" + +range_query.set_filter(is_engineer) + +result_print(index.query(range_query)) +``` + + +
vector_distanceusercredit_scoreagejob
0johnhigh18engineer
+ + +## Advanced Query Modifiers + +See all modifier options available on the query API docs: https://docs.redisvl.com/en/latest/api/query.html + + +```python +# Sort by a different field and change dialect +v = VectorQuery( + vector=[0.1, 0.1, 0.5], + vector_field_name="user_embedding", + return_fields=["user", "credit_score", "age", "job", "office_location"], + num_results=5, + filter_expression=is_engineer +).sort_by("age", asc=False).dialect(3) + +result = index.query(v) +result_print(result) +``` + + +
vector_distanceageusercredit_scorejoboffice_location
0.109129190445100tylerhighengineer-122.0839,37.3861
018johnhighengineer-122.4194,37.7749
+ + +### Raw Redis Query String + +Sometimes it's helpful to convert these classes into their raw Redis query strings. + + +```python +# check out the complex query from above +str(v) +``` + + + + + '@job:("engineer")=>[KNN 5 @user_embedding $vector AS vector_distance] RETURN 6 user credit_score age job office_location vector_distance SORTBY age DESC DIALECT 3 LIMIT 0 5' + + + + +```python +t = Tag("credit_score") == "high" + +str(t) +``` + + + + + '@credit_score:{high}' + + + + +```python +t = Tag("credit_score") == "high" +low = Num("age") >= 18 +high = Num("age") <= 100 + +combined = t & low & high + +str(combined) +``` + + + + + '((@credit_score:{high} @age:[18 +inf]) @age:[-inf 100])' + + + +The RedisVL `SearchIndex` class exposes a `search()` method which is a simple wrapper around the `FT.SEARCH` API. +Provide any valid Redis query string. + + +```python +results = index.search(str(t)) +for r in results.docs: + print(r.__dict__) +``` + + {'id': 'user_queries_docs:409ff48274724984ba14865db0495fc5', 'payload': None, 'user': 'john', 'age': '18', 'job': 'engineer', 'credit_score': 'high', 'office_location': '-122.4194,37.7749', 'user_embedding': '==\x00\x00\x00?'} + {'id': 'user_queries_docs:3dec0e9f2db04e19bff224c5a2a0ba3c', 'payload': None, 'user': 'nancy', 'age': '94', 'job': 'doctor', 'credit_score': 'high', 'office_location': '-122.4194,37.7749', 'user_embedding': '333?=\x00\x00\x00?'} + {'id': 'user_queries_docs:562263669ff74a0295c515018d151d7b', 'payload': None, 'user': 'tyler', 'age': '100', 'job': 'engineer', 'credit_score': 'high', 'office_location': '-122.0839,37.3861', 'user_embedding': '=>\x00\x00\x00?'} + {'id': 'user_queries_docs:94176145f9de4e288ca2460cd5d1188e', 'payload': None, 'user': 'tim', 'age': '12', 'job': 'dermatologist', 'credit_score': 'high', 'office_location': '-122.0839,37.3861', 'user_embedding': '>>\x00\x00\x00?'} + + + +```python +# Cleanup +index.delete() +``` diff --git a/content/integrate/redisvl/user_guide/03_llmcache.md b/content/integrate/redisvl/user_guide/03_llmcache.md new file mode 100644 index 0000000000..da54e52af3 --- /dev/null +++ b/content/integrate/redisvl/user_guide/03_llmcache.md @@ -0,0 +1,474 @@ +--- +linkTitle: Semantic Caching for LLMs +title: Semantic Caching for LLMs +type: integration +--- + + +RedisVL provides a ``SemanticCache`` interface to utilize Redis' built-in caching capabilities AND vector search in order to store responses from previously-answered questions. This reduces the number of requests and tokens sent to the Large Language Models (LLM) service, decreasing costs and enhancing application throughput (by reducing the time taken to generate responses). + +This notebook will go over how to use Redis as a Semantic Cache for your applications + +First, we will import [OpenAI](https://platform.openai.com) to use their API for responding to user prompts. We will also create a simple `ask_openai` helper method to assist. + + +```python +import os +import getpass +import time +import numpy as np + +from openai import OpenAI + + +os.environ["TOKENIZERS_PARALLELISM"] = "False" + +api_key = os.getenv("OPENAI_API_KEY") or getpass.getpass("Enter your OpenAI API key: ") + +client = OpenAI(api_key=api_key) + +def ask_openai(question: str) -> str: + response = client.completions.create( + model="gpt-3.5-turbo-instruct", + prompt=question, + max_tokens=200 + ) + return response.choices[0].text.strip() +``` + + +```python +# Test +print(ask_openai("What is the capital of France?")) +``` + + The capital of France is Paris. + + +## Initializing ``SemanticCache`` + +``SemanticCache`` will automatically create an index within Redis upon initialization for the semantic cache content. + + +```python +from redisvl.extensions.llmcache import SemanticCache + +llmcache = SemanticCache( + name="llmcache", # underlying search index name + redis_url="redis://localhost:6379", # redis connection url string + distance_threshold=0.1 # semantic cache distance threshold +) +``` + + 22:11:38 redisvl.index.index INFO Index already exists, not overwriting. + + + +```python +# look at the index specification created for the semantic cache lookup +!rvl index info -i llmcache +``` + + + + Index Information: + ╭──────────────┬────────────────┬──────────────┬─────────────────┬────────────╮ + │ Index Name │ Storage Type │ Prefixes │ Index Options │ Indexing │ + ├──────────────┼────────────────┼──────────────┼─────────────────┼────────────┤ + │ llmcache │ HASH │ ['llmcache'] │ [] │ 0 │ + ╰──────────────┴────────────────┴──────────────┴─────────────────┴────────────╯ + Index Fields: + ╭───────────────┬───────────────┬─────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬─────────────────┬────────────────╮ + │ Name │ Attribute │ Type │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ + ├───────────────┼───────────────┼─────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼─────────────────┼────────────────┤ + │ prompt │ prompt │ TEXT │ WEIGHT │ 1 │ │ │ │ │ │ │ + │ response │ response │ TEXT │ WEIGHT │ 1 │ │ │ │ │ │ │ + │ inserted_at │ inserted_at │ NUMERIC │ │ │ │ │ │ │ │ │ + │ updated_at │ updated_at │ NUMERIC │ │ │ │ │ │ │ │ │ + │ prompt_vector │ prompt_vector │ VECTOR │ algorithm │ FLAT │ data_type │ FLOAT32 │ dim │ 768 │ distance_metric │ COSINE │ + ╰───────────────┴───────────────┴─────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴─────────────────┴────────────────╯ + + +## Basic Cache Usage + + +```python +question = "What is the capital of France?" +``` + + +```python +# Check the semantic cache -- should be empty +if response := llmcache.check(prompt=question): + print(response) +else: + print("Empty cache") +``` + + Empty cache + + +Our initial cache check should be empty since we have not yet stored anything in the cache. Below, store the `question`, +proper `response`, and any arbitrary `metadata` (as a python dictionary object) in the cache. + + +```python +# Cache the question, answer, and arbitrary metadata +llmcache.store( + prompt=question, + response="Paris", + metadata={"city": "Paris", "country": "france"} +) +``` + + + + + 'llmcache:115049a298532be2f181edb03f766770c0db84c22aff39003fec340deaec7545' + + + +Now we will check the cache again with the same question and with a semantically similar question: + + +```python +# Check the cache again +if response := llmcache.check(prompt=question, return_fields=["prompt", "response", "metadata"]): + print(response) +else: + print("Empty cache") +``` + + [{'prompt': 'What is the capital of France?', 'response': 'Paris', 'metadata': {'city': 'Paris', 'country': 'france'}, 'key': 'llmcache:115049a298532be2f181edb03f766770c0db84c22aff39003fec340deaec7545'}] + + + +```python +# Check for a semantically similar result +question = "What actually is the capital of France?" +llmcache.check(prompt=question)[0]['response'] +``` + + + + + 'Paris' + + + +## Customize the Distance Threshhold + +For most use cases, the right semantic similarity threshhold is not a fixed quantity. Depending on the choice of embedding model, +the properties of the input query, and even business use case -- the threshhold might need to change. + +Fortunately, you can seamlessly adjust the threshhold at any point like below: + + +```python +# Widen the semantic distance threshold +llmcache.set_threshold(0.3) +``` + + +```python +# Really try to trick it by asking around the point +# But is able to slip just under our new threshold +question = "What is the capital city of the country in Europe that also has a city named Nice?" +llmcache.check(prompt=question)[0]['response'] +``` + + + + + 'Paris' + + + + +```python +# Invalidate the cache completely by clearing it out +llmcache.clear() + +# should be empty now +llmcache.check(prompt=question) +``` + + + + + [] + + + +## Utilize TTL + +Redis uses TTL policies (optional) to expire individual keys at points in time in the future. +This allows you to focus on your data flow and business logic without bothering with complex cleanup tasks. + +A TTL policy set on the `SemanticCache` allows you to temporarily hold onto cache entries. Below, we will set the TTL policy to 5 seconds. + + +```python +llmcache.set_ttl(5) # 5 seconds +``` + + +```python +llmcache.store("This is a TTL test", "This is a TTL test response") + +time.sleep(6) +``` + + +```python +# confirm that the cache has cleared by now on it's own +result = llmcache.check("This is a TTL test") + +print(result) +``` + + [] + + + +```python +# Reset the TTL to null (long lived data) +llmcache.set_ttl() +``` + +## Simple Performance Testing + +Next, we will measure the speedup obtained by using ``SemanticCache``. We will use the ``time`` module to measure the time taken to generate responses with and without ``SemanticCache``. + + +```python +def answer_question(question: str) -> str: + """Helper function to answer a simple question using OpenAI with a wrapper + check for the answer in the semantic cache first. + + Args: + question (str): User input question. + + Returns: + str: Response. + """ + results = llmcache.check(prompt=question) + if results: + return results[0]["response"] + else: + answer = ask_openai(question) + return answer +``` + + +```python +start = time.time() +# asking a question -- openai response time +question = "What was the name of the first US President?" +answer = answer_question(question) +end = time.time() + +print(f"Without caching, a call to openAI to answer this simple question took {end-start} seconds.") + +# add the entry to our LLM cache +llmcache.store(prompt=question, response="George Washington") +``` + + Without caching, a call to openAI to answer this simple question took 0.9034533500671387 seconds. + + + + + + 'llmcache:67e0f6e28fe2a61c0022fd42bf734bb8ffe49d3e375fd69d692574295a20fc1a' + + + + +```python +# Calculate the avg latency for caching over LLM usage +times = [] + +for _ in range(10): + cached_start = time.time() + cached_answer = answer_question(question) + cached_end = time.time() + times.append(cached_end-cached_start) + +avg_time_with_cache = np.mean(times) +print(f"Avg time taken with LLM cache enabled: {avg_time_with_cache}") +print(f"Percentage of time saved: {round(((end - start) - avg_time_with_cache) / (end - start) * 100, 2)}%") +``` + + Avg time taken with LLM cache enabled: 0.09753389358520508 + Percentage of time saved: 89.2% + + + +```python +# check the stats of the index +!rvl stats -i llmcache +``` + + + Statistics: + ╭─────────────────────────────┬─────────────╮ + │ Stat Key │ Value │ + ├─────────────────────────────┼─────────────┤ + │ num_docs │ 1 │ + │ num_terms │ 19 │ + │ max_doc_id │ 6 │ + │ num_records │ 53 │ + │ percent_indexed │ 1 │ + │ hash_indexing_failures │ 0 │ + │ number_of_uses │ 45 │ + │ bytes_per_record_avg │ 45.0566 │ + │ doc_table_size_mb │ 0.000134468 │ + │ inverted_sz_mb │ 0.00227737 │ + │ key_table_size_mb │ 2.76566e-05 │ + │ offset_bits_per_record_avg │ 8 │ + │ offset_vectors_sz_mb │ 3.91006e-05 │ + │ offsets_per_term_avg │ 0.773585 │ + │ records_per_doc_avg │ 53 │ + │ sortable_values_size_mb │ 0 │ + │ total_indexing_time │ 19.454 │ + │ total_inverted_index_blocks │ 21 │ + │ vector_index_sz_mb │ 3.0161 │ + ╰─────────────────────────────┴─────────────╯ + + + +```python +# Clear the cache AND delete the underlying index +llmcache.delete() +``` + +## Cache Access Controls, Tags & Filters +When running complex workflows with similar applications, or handling multiple users it's important to keep data segregated. Building on top of RedisVL's support for complex and hybrid queries we can tag and filter cache entries using custom-defined `filterable_fields`. + +Let's store multiple users' data in our cache with similar prompts and ensure we return only the correct user information: + + +```python +private_cache = SemanticCache( + name="private_cache", + filterable_fields=[{"name": "user_id", "type": "tag"}] +) + +private_cache.store( + prompt="What is the phone number linked to my account?", + response="The number on file is 123-555-0000", + filters={"user_id": "abc"}, +) + +private_cache.store( + prompt="What's the phone number linked in my account?", + response="The number on file is 123-555-1111", + filters={"user_id": "def"}, +) +``` + + + + + 'private_cache:5de9d651f802d9cc3f62b034ced3466bf886a542ce43fe1c2b4181726665bf9c' + + + + +```python +from redisvl.query.filter import Tag + +# define user id filter +user_id_filter = Tag("user_id") == "abc" + +response = private_cache.check( + prompt="What is the phone number linked to my account?", + filter_expression=user_id_filter, + num_results=2 +) + +print(f"found {len(response)} entry \n{response[0]['response']}") +``` + + found 1 entry + The number on file is 123-555-0000 + + + +```python +# Cleanup +private_cache.delete() +``` + +Multiple `filterable_fields` can be defined on a cache, and complex filter expressions can be constructed to filter on these fields, as well as the default fields already present. + + +```python + +complex_cache = SemanticCache( + name='account_data', + filterable_fields=[ + {"name": "user_id", "type": "tag"}, + {"name": "account_type", "type": "tag"}, + {"name": "account_balance", "type": "numeric"}, + {"name": "transaction_amount", "type": "numeric"} + ] +) +complex_cache.store( + prompt="what is my most recent checking account transaction under $100?", + response="Your most recent transaction was for $75", + filters={"user_id": "abc", "account_type": "checking", "transaction_amount": 75}, +) +complex_cache.store( + prompt="what is my most recent savings account transaction?", + response="Your most recent deposit was for $300", + filters={"user_id": "abc", "account_type": "savings", "transaction_amount": 300}, +) +complex_cache.store( + prompt="what is my most recent checking account transaction over $200?", + response="Your most recent transaction was for $350", + filters={"user_id": "abc", "account_type": "checking", "transaction_amount": 350}, +) +complex_cache.store( + prompt="what is my checking account balance?", + response="Your current checking account is $1850", + filters={"user_id": "abc", "account_type": "checking"}, +) +``` + + + + + 'account_data:d48ebb3a2efbdbc17930a8c7559c548a58b562b2572ef0be28f0bb4ece2382e1' + + + + +```python +from redisvl.query.filter import Num + +value_filter = Num("transaction_amount") > 100 +account_filter = Tag("account_type") == "checking" +complex_filter = value_filter & account_filter + +# check for checking account transactions over $100 +complex_cache.set_threshold(0.3) +response = complex_cache.check( + prompt="what is my most recent checking account transaction?", + filter_expression=complex_filter, + num_results=5 +) +print(f'found {len(response)} entry') +print(response[0]["response"]) +``` + + found 1 entry + Your most recent transaction was for $350 + + + +```python +# Cleanup +complex_cache.delete() +``` diff --git a/content/integrate/redisvl/user_guide/04_vectorizers.md b/content/integrate/redisvl/user_guide/04_vectorizers.md new file mode 100644 index 0000000000..0ceb56e6c1 --- /dev/null +++ b/content/integrate/redisvl/user_guide/04_vectorizers.md @@ -0,0 +1,496 @@ +--- +linkTitle: Vectorizers +title: Vectorizers +type: integration +--- + + +In this notebook, we will show how to use RedisVL to create embeddings using the built-in text embedding vectorizers. Today RedisVL supports: +1. OpenAI +2. HuggingFace +3. Vertex AI +4. Cohere +5. Mistral AI +6. Amazon Bedrock +7. Bringing your own vectorizer +8. VoyageAI + +Before running this notebook, be sure to +1. Have installed ``redisvl`` and have that environment active for this notebook. +2. Have a running Redis Stack instance with RediSearch > 2.4 active. + +For example, you can run Redis Stack locally with Docker: + +```bash +docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest +``` + +This will run Redis on port 6379 and RedisInsight at http://localhost:8001. + + +```python +# import necessary modules +import os +``` + +## Creating Text Embeddings + +This example will show how to create an embedding from 3 simple sentences with a number of different text vectorizers in RedisVL. + +- "That is a happy dog" +- "That is a happy person" +- "Today is a nice day" + + +### OpenAI + +The ``OpenAITextVectorizer`` makes it simple to use RedisVL with the embeddings models at OpenAI. For this you will need to install ``openai``. + +```bash +pip install openai +``` + + + +```python +import getpass + +# setup the API Key +api_key = os.environ.get("OPENAI_API_KEY") or getpass.getpass("Enter your OpenAI API key: ") +``` + + +```python +from redisvl.utils.vectorize import OpenAITextVectorizer + +# create a vectorizer +oai = OpenAITextVectorizer( + model="text-embedding-ada-002", + api_config={"api_key": api_key}, +) + +test = oai.embed("This is a test sentence.") +print("Vector dimensions: ", len(test)) +test[:10] +``` + + +```python +# Create many embeddings at once +sentences = [ + "That is a happy dog", + "That is a happy person", + "Today is a sunny day" +] + +embeddings = oai.embed_many(sentences) +embeddings[0][:10] +``` + + +```python +# openai also supports asyncronous requests, which we can use to speed up the vectorization process. +embeddings = await oai.aembed_many(sentences) +print("Number of Embeddings:", len(embeddings)) + +``` + +### Azure OpenAI + +The ``AzureOpenAITextVectorizer`` is a variation of the OpenAI vectorizer that calls OpenAI models within Azure. If you've already installed ``openai``, then you're ready to use Azure OpenAI. + +The only practical difference between OpenAI and Azure OpenAI is the variables required to call the API. + + +```python +# additionally to the API Key, setup the API endpoint and version +api_key = os.environ.get("AZURE_OPENAI_API_KEY") or getpass.getpass("Enter your AzureOpenAI API key: ") +api_version = os.environ.get("OPENAI_API_VERSION") or getpass.getpass("Enter your AzureOpenAI API version: ") +azure_endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT") or getpass.getpass("Enter your AzureOpenAI API endpoint: ") +deployment_name = os.environ.get("AZURE_OPENAI_DEPLOYMENT_NAME", "text-embedding-ada-002") + +``` + + +```python +from redisvl.utils.vectorize import AzureOpenAITextVectorizer + +# create a vectorizer +az_oai = AzureOpenAITextVectorizer( + model=deployment_name, # Must be your CUSTOM deployment name + api_config={ + "api_key": api_key, + "api_version": api_version, + "azure_endpoint": azure_endpoint + }, +) + +test = az_oai.embed("This is a test sentence.") +print("Vector dimensions: ", len(test)) +test[:10] +``` + + +```python +# Just like OpenAI, AzureOpenAI supports batching embeddings and asynchronous requests. +sentences = [ + "That is a happy dog", + "That is a happy person", + "Today is a sunny day" +] + +embeddings = await az_oai.aembed_many(sentences) +embeddings[0][:10] +``` + +### Huggingface + +[Huggingface](https://huggingface.co/models) is a popular NLP platform that has a number of pre-trained models you can use off the shelf. RedisVL supports using Huggingface "Sentence Transformers" to create embeddings from text. To use Huggingface, you will need to install the ``sentence-transformers`` library. + +```bash +pip install sentence-transformers +``` + + +```python +os.environ["TOKENIZERS_PARALLELISM"] = "false" +from redisvl.utils.vectorize import HFTextVectorizer + + +# create a vectorizer +# choose your model from the huggingface website +hf = HFTextVectorizer(model="sentence-transformers/all-mpnet-base-v2") + +# embed a sentence +test = hf.embed("This is a test sentence.") +test[:10] +``` + + +```python +# You can also create many embeddings at once +embeddings = hf.embed_many(sentences, as_buffer=True) + +``` + +### VertexAI + +[VertexAI](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings) is GCP's fully-featured AI platform including a number of pretrained LLMs. RedisVL supports using VertexAI to create embeddings from these models. To use VertexAI, you will first need to install the ``google-cloud-aiplatform`` library. + +```bash +pip install google-cloud-aiplatform>=1.26 +``` + +1. Then you need to gain access to a [Google Cloud Project](https://cloud.google.com/gcp?hl=en) and provide [access to credentials](https://cloud.google.com/docs/authentication/application-default-credentials). This is accomplished by setting the `GOOGLE_APPLICATION_CREDENTIALS` environment variable pointing to the path of a JSON key file downloaded from your service account on GCP. +2. Lastly, you need to find your [project ID](https://support.google.com/googleapi/answer/7014113?hl=en) and [geographic region for VertexAI](https://cloud.google.com/vertex-ai/docs/general/locations). + + +**Make sure the following env vars are set:** + +``` +GOOGLE_APPLICATION_CREDENTIALS= +GCP_PROJECT_ID= +GCP_LOCATION= +``` + + +```python +from redisvl.utils.vectorize import VertexAITextVectorizer + + +# create a vectorizer +vtx = VertexAITextVectorizer(api_config={ + "project_id": os.environ.get("GCP_PROJECT_ID") or getpass.getpass("Enter your GCP Project ID: "), + "location": os.environ.get("GCP_LOCATION") or getpass.getpass("Enter your GCP Location: "), + "google_application_credentials": os.environ.get("GOOGLE_APPLICATION_CREDENTIALS") or getpass.getpass("Enter your Google App Credentials path: ") +}) + +# embed a sentence +test = vtx.embed("This is a test sentence.") +test[:10] +``` + +### Cohere + +[Cohere](https://dashboard.cohere.ai/) allows you to implement language AI into your product. The `CohereTextVectorizer` makes it simple to use RedisVL with the embeddings models at Cohere. For this you will need to install `cohere`. + +```bash +pip install cohere +``` + + +```python +import getpass +# setup the API Key +api_key = os.environ.get("COHERE_API_KEY") or getpass.getpass("Enter your Cohere API key: ") +``` + + +Special attention needs to be paid to the `input_type` parameter for each `embed` call. For example, for embedding +queries, you should set `input_type='search_query'`; for embedding documents, set `input_type='search_document'`. See +more information [here](https://docs.cohere.com/reference/embed) + + +```python +from redisvl.utils.vectorize import CohereTextVectorizer + +# create a vectorizer +co = CohereTextVectorizer( + model="embed-english-v3.0", + api_config={"api_key": api_key}, +) + +# embed a search query +test = co.embed("This is a test sentence.", input_type='search_query') +print("Vector dimensions: ", len(test)) +print(test[:10]) + +# embed a document +test = co.embed("This is a test sentence.", input_type='search_document') +print("Vector dimensions: ", len(test)) +print(test[:10]) +``` + +Learn more about using RedisVL and Cohere together through [this dedicated user guide](https://docs.cohere.com/docs/redis-and-cohere). + +### VoyageAI + +[VoyageAI](https://dash.voyageai.com/) allows you to implement language AI into your product. The `VoyageAITextVectorizer` makes it simple to use RedisVL with the embeddings models at VoyageAI. For this you will need to install `voyageai`. + +```bash +pip install voyageai +``` + + +```python +import getpass +# setup the API Key +api_key = os.environ.get("VOYAGE_API_KEY") or getpass.getpass("Enter your VoyageAI API key: ") +``` + + +Special attention needs to be paid to the `input_type` parameter for each `embed` call. For example, for embedding +queries, you should set `input_type='query'`; for embedding documents, set `input_type='document'`. See +more information [here](https://docs.voyageai.com/docs/embeddings) + + +```python +from redisvl.utils.vectorize import VoyageAITextVectorizer + +# create a vectorizer +vo = VoyageAITextVectorizer( + model="voyage-law-2", # Please check the available models at https://docs.voyageai.com/docs/embeddings + api_config={"api_key": api_key}, +) + +# embed a search query +test = vo.embed("This is a test sentence.", input_type='query') +print("Vector dimensions: ", len(test)) +print(test[:10]) + +# embed a document +test = vo.embed("This is a test sentence.", input_type='document') +print("Vector dimensions: ", len(test)) +print(test[:10]) +``` + +### Mistral AI + +[Mistral](https://console.mistral.ai/) offers LLM and embedding APIs for you to implement into your product. The `MistralAITextVectorizer` makes it simple to use RedisVL with their embeddings model. +You will need to install `mistralai`. + +```bash +pip install mistralai +``` + + +```python +from redisvl.utils.vectorize import MistralAITextVectorizer + +mistral = MistralAITextVectorizer() + +# embed a sentence using their asyncronous method +test = await mistral.aembed("This is a test sentence.") +print("Vector dimensions: ", len(test)) +print(test[:10]) +``` + +### Amazon Bedrock + +Amazon Bedrock provides fully managed foundation models for text embeddings. Install the required dependencies: + +```bash +pip install 'redisvl[bedrock]' # Installs boto3 +``` + +#### Configure AWS credentials: + + +```python +import os +import getpass + +if "AWS_ACCESS_KEY_ID" not in os.environ: + os.environ["AWS_ACCESS_KEY_ID"] = getpass.getpass("Enter AWS Access Key ID: ") +if "AWS_SECRET_ACCESS_KEY" not in os.environ: + os.environ["AWS_SECRET_ACCESS_KEY"] = getpass.getpass("Enter AWS Secret Key: ") + +os.environ["AWS_REGION"] = "us-east-1" # Change as needed +``` + +#### Create embeddings: + + +```python +from redisvl.utils.vectorize import BedrockTextVectorizer + +bedrock = BedrockTextVectorizer( + model="amazon.titan-embed-text-v2:0" +) + +# Single embedding +text = "This is a test sentence." +embedding = bedrock.embed(text) +print(f"Vector dimensions: {len(embedding)}") + +# Multiple embeddings +sentences = [ + "That is a happy dog", + "That is a happy person", + "Today is a sunny day" +] +embeddings = bedrock.embed_many(sentences) +``` + +### Custom Vectorizers + +RedisVL supports the use of other vectorizers and provides a class to enable compatibility with any function that generates a vector or vectors from string data + + +```python +from redisvl.utils.vectorize import CustomTextVectorizer + +def generate_embeddings(text_input, **kwargs): + return [0.101] * 768 + +custom_vectorizer = CustomTextVectorizer(generate_embeddings) + +custom_vectorizer.embed("This is a test sentence.")[:10] +``` + +This enables the use of custom vectorizers with other RedisVL components + + +```python +from redisvl.extensions.llmcache import SemanticCache + +cache = SemanticCache(name="custom_cache", vectorizer=custom_vectorizer) + +cache.store("this is a test prompt", "this is a test response") +cache.check("this is also a test prompt") +``` + +## Search with Provider Embeddings + +Now that we've created our embeddings, we can use them to search for similar sentences. We will use the same 3 sentences from above and search for similar sentences. + +First, we need to create the schema for our index. + +Here's what the schema for the example looks like in yaml for the HuggingFace vectorizer: + +```yaml +version: '0.1.0' + +index: + name: vectorizers + prefix: doc + storage_type: hash + +fields: + - name: sentence + type: text + - name: embedding + type: vector + attrs: + dims: 768 + algorithm: flat + distance_metric: cosine +``` + + +```python +from redisvl.index import SearchIndex + +# construct a search index from the schema +index = SearchIndex.from_yaml("./schema.yaml") + +# connect to local redis instance +index.connect("redis://localhost:6379") + +# create the index (no data yet) +index.create(overwrite=True) +``` + + +```python +# use the CLI to see the created index +!rvl index listall +``` + +Loading data to RedisVL is easy. It expects a list of dictionaries. The vector is stored as bytes. + + +```python +from redisvl.redis.utils import array_to_buffer + +embeddings = hf.embed_many(sentences) + +data = [{"text": t, + "embedding": array_to_buffer(v, dtype="float32")} + for t, v in zip(sentences, embeddings)] + +index.load(data) +``` + + +```python +from redisvl.query import VectorQuery + +# use the HuggingFace vectorizer again to create a query embedding +query_embedding = hf.embed("That is a happy cat") + +query = VectorQuery( + vector=query_embedding, + vector_field_name="embedding", + return_fields=["text"], + num_results=3 +) + +results = index.query(query) +for doc in results: + print(doc["text"], doc["vector_distance"]) +``` + +## Selecting your float data type +When embedding text as byte arrays RedisVL supports 4 different floating point data types, `float16`, `float32`, `float64` and `bfloat16`, and 2 integer types, `int8` and `uint8`. +Your dtype set for your vectorizer must match what is defined in your search index. If one is not explicitly set the default is `float32`. + + +```python +vectorizer = HFTextVectorizer(dtype="float16") + +# subsequent calls to embed('', as_buffer=True) and embed_many('', as_buffer=True) will now encode as float16 +float16_bytes = vectorizer.embed('test sentence', as_buffer=True) + +# to generate embeddings with different dtype instantiate a new vectorizer +vectorizer_64 = HFTextVectorizer(dtype='float64') +float64_bytes = vectorizer_64.embed('test sentence', as_buffer=True) + +float16_bytes != float64_bytes +``` + + +```python +# cleanup +index.delete() +``` diff --git a/content/integrate/redisvl/user_guide/05_hash_vs_json.md b/content/integrate/redisvl/user_guide/05_hash_vs_json.md new file mode 100644 index 0000000000..0b3ba1bedd --- /dev/null +++ b/content/integrate/redisvl/user_guide/05_hash_vs_json.md @@ -0,0 +1,345 @@ +--- +linkTitle: Hash vs JSON Storage +title: Hash vs JSON Storage +type: integration +--- + + + +Out of the box, Redis provides a [variety of data structures](https://redis.com/redis-enterprise/data-structures/) that can adapt to your domain specific applications and use cases. +In this notebook, we will demonstrate how to use RedisVL with both [Hash](https://redis.io/docs/data-types/hashes/) and [JSON](https://redis.io/docs/data-types/json/) data. + + +Before running this notebook, be sure to +1. Have installed ``redisvl`` and have that environment active for this notebook. +2. Have a running Redis Stack or Redis Enterprise instance with RediSearch > 2.4 activated. + +For example, you can run [Redis Stack](https://redis.io/docs/install/install-stack/) locally with Docker: + +```bash +docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest +``` + +Or create a [FREE Redis Cloud](https://redis.io/cloud). + + +```python +# import necessary modules +import pickle + +from redisvl.redis.utils import buffer_to_array +from redisvl.index import SearchIndex + + +# load in the example data and printing utils +data = pickle.load(open("hybrid_example_data.pkl", "rb")) +``` + + +```python +from jupyterutils import result_print, table_print + +table_print(data) +``` + + +
useragejobcredit_scoreoffice_locationuser_embedding
john18engineerhigh-122.4194,37.7749b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'
derrick14doctorlow-122.4194,37.7749b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'
nancy94doctorhigh-122.4194,37.7749b'333?\xcd\xcc\xcc=\x00\x00\x00?'
tyler100engineerhigh-122.0839,37.3861b'\xcd\xcc\xcc=\xcd\xcc\xcc>\x00\x00\x00?'
tim12dermatologisthigh-122.0839,37.3861b'\xcd\xcc\xcc>\xcd\xcc\xcc>\x00\x00\x00?'
taimur15CEOlow-122.0839,37.3861b'\x9a\x99\x19?\xcd\xcc\xcc=\x00\x00\x00?'
joe35dentistmedium-122.0839,37.3861b'fff?fff?\xcd\xcc\xcc='
+ + +## Hash or JSON -- how to choose? +Both storage options offer a variety of features and tradeoffs. Below we will work through a dummy dataset to learn when and how to use both. + +### Working with Hashes +Hashes in Redis are simple collections of field-value pairs. Think of it like a mutable single-level dictionary contains multiple "rows": + + +```python +{ + "model": "Deimos", + "brand": "Ergonom", + "type": "Enduro bikes", + "price": 4972, +} +``` + +Hashes are best suited for use cases with the following characteristics: +- Performance (speed) and storage space (memory consumption) are top concerns +- Data can be easily normalized and modeled as a single-level dict + +Hashes are typically the default recommendation. + + +```python +# define the hash index schema +hash_schema = { + "index": { + "name": "user-hash", + "prefix": "user-hash-docs", + "storage_type": "hash", # default setting -- HASH + }, + "fields": [ + {"name": "user", "type": "tag"}, + {"name": "credit_score", "type": "tag"}, + {"name": "job", "type": "text"}, + {"name": "age", "type": "numeric"}, + {"name": "office_location", "type": "geo"}, + { + "name": "user_embedding", + "type": "vector", + "attrs": { + "dims": 3, + "distance_metric": "cosine", + "algorithm": "flat", + "datatype": "float32" + } + + } + ], +} +``` + + +```python +# construct a search index from the hash schema +hindex = SearchIndex.from_dict(hash_schema) + +# connect to local redis instance +hindex.connect("redis://localhost:6379") + +# create the index (no data yet) +hindex.create(overwrite=True) +``` + + +```python +# show the underlying storage type +hindex.storage_type +``` + + + + + + + + +#### Vectors as byte strings +One nuance when working with Hashes in Redis, is that all vectorized data must be passed as a byte string (for efficient storage, indexing, and processing). An example of that can be seen below: + + +```python +# show a single entry from the data that will be loaded +data[0] +``` + + + + + {'user': 'john', + 'age': 18, + 'job': 'engineer', + 'credit_score': 'high', + 'office_location': '-122.4194,37.7749', + 'user_embedding': b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'} + + + + +```python +# load hash data +keys = hindex.load(data) +``` + + +```python +!rvl stats -i user-hash +``` + + + Statistics: + ╭─────────────────────────────┬─────────────╮ + │ Stat Key │ Value │ + ├─────────────────────────────┼─────────────┤ + │ num_docs │ 7 │ + │ num_terms │ 6 │ + │ max_doc_id │ 7 │ + │ num_records │ 44 │ + │ percent_indexed │ 1 │ + │ hash_indexing_failures │ 0 │ + │ number_of_uses │ 1 │ + │ bytes_per_record_avg │ 3.40909 │ + │ doc_table_size_mb │ 0.000767708 │ + │ inverted_sz_mb │ 0.000143051 │ + │ key_table_size_mb │ 0.000248909 │ + │ offset_bits_per_record_avg │ 8 │ + │ offset_vectors_sz_mb │ 8.58307e-06 │ + │ offsets_per_term_avg │ 0.204545 │ + │ records_per_doc_avg │ 6.28571 │ + │ sortable_values_size_mb │ 0 │ + │ total_indexing_time │ 1.053 │ + │ total_inverted_index_blocks │ 18 │ + │ vector_index_sz_mb │ 0.0202332 │ + ╰─────────────────────────────┴─────────────╯ + + +#### Performing Queries +Once our index is created and data is loaded into the right format, we can run queries against the index with RedisVL: + + +```python +from redisvl.query import VectorQuery +from redisvl.query.filter import Tag, Text, Num + +t = (Tag("credit_score") == "high") & (Text("job") % "enginee*") & (Num("age") > 17) + +v = VectorQuery([0.1, 0.1, 0.5], + "user_embedding", + return_fields=["user", "credit_score", "age", "job", "office_location"], + filter_expression=t) + + +results = hindex.query(v) +result_print(results) + +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
+ + + +```python +# clean up +hindex.delete() + +``` + +### Working with JSON +Redis also supports native **JSON** objects. These can be multi-level (nested) objects, with full JSONPath support for updating/retrieving sub elements: + +```python +{ + "name": "bike", + "metadata": { + "model": "Deimos", + "brand": "Ergonom", + "type": "Enduro bikes", + "price": 4972, + } +} +``` + +JSON is best suited for use cases with the following characteristics: +- Ease of use and data model flexibility are top concerns +- Application data is already native JSON +- Replacing another document storage/db solution + +#### Full JSON Path support +Because Redis enables full JSON path support, when creating an index schema, elements need to be indexed and selected by their path with the desired `name` AND `path` that points to where the data is located within the objects. + +By default, RedisVL will assume the path as `$.{name}` if not provided in JSON fields schema. + + +```python +# define the json index schema +json_schema = { + "index": { + "name": "user-json", + "prefix": "user-json-docs", + "storage_type": "json", # JSON storage type + }, + "fields": [ + {"name": "user", "type": "tag"}, + {"name": "credit_score", "type": "tag"}, + {"name": "job", "type": "text"}, + {"name": "age", "type": "numeric"}, + {"name": "office_location", "type": "geo"}, + { + "name": "user_embedding", + "type": "vector", + "attrs": { + "dims": 3, + "distance_metric": "cosine", + "algorithm": "flat", + "datatype": "float32" + } + + } + ], +} +``` + + +```python +# construct a search index from the json schema +jindex = SearchIndex.from_dict(json_schema) + +# connect to local redis instance +jindex.connect("redis://localhost:6379") + +# create the index (no data yet) +jindex.create(overwrite=True) +``` + + +```python +# note the multiple indices in the same database +!rvl index listall +``` + + 11:54:18 [RedisVL] INFO Indices: + 11:54:18 [RedisVL] INFO 1. user-json + + +#### Vectors as float arrays +Vectorized data stored in JSON must be stored as a pure array (python list) of floats. We will modify our sample data to account for this below: + + +```python +import numpy as np + +json_data = data.copy() + +for d in json_data: + d['user_embedding'] = buffer_to_array(d['user_embedding'], dtype='float32') +``` + + +```python +# inspect a single JSON record +json_data[0] +``` + + + + + {'user': 'john', + 'age': 18, + 'job': 'engineer', + 'credit_score': 'high', + 'office_location': '-122.4194,37.7749', + 'user_embedding': [0.10000000149011612, 0.10000000149011612, 0.5]} + + + + +```python +keys = jindex.load(json_data) +``` + + +```python +# we can now run the exact same query as above +result_print(jindex.query(v)) +``` + + +
vector_distanceusercredit_scoreagejoboffice_location
0johnhigh18engineer-122.4194,37.7749
0.109129190445tylerhigh100engineer-122.0839,37.3861
+ + +## Cleanup + + +```python +jindex.delete() +``` diff --git a/content/integrate/redisvl/user_guide/06_rerankers.md b/content/integrate/redisvl/user_guide/06_rerankers.md new file mode 100644 index 0000000000..be54805d9a --- /dev/null +++ b/content/integrate/redisvl/user_guide/06_rerankers.md @@ -0,0 +1,223 @@ +--- +linkTitle: Rerankers +title: Rerankers +type: integration +--- + + +In this notebook, we will show how to use RedisVL to rerank search results +(documents or chunks or records) based on the input query. Today RedisVL +supports reranking through: + +- A re-ranker that uses pre-trained [Cross-Encoders](https://sbert.net/examples/applications/cross-encoder/README.html) which can use models from [Hugging Face cross encoder models](https://huggingface.co/cross-encoder) or Hugging Face models that implement a cross encoder function ([example: BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base)). +- The [Cohere /rerank API](https://docs.cohere.com/docs/rerank-2). +- The [VoyageAI /rerank API](https://docs.voyageai.com/docs/reranker). + +Before running this notebook, be sure to: +1. Have installed ``redisvl`` and have that environment active for this notebook. +2. Have a running Redis Stack instance with RediSearch > 2.4 active. + +For example, you can run Redis Stack locally with Docker: + +```bash +docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest +``` + +This will run Redis on port 6379 and RedisInsight at http://localhost:8001. + + +```python +# import necessary modules +import os +``` + +## Simple Reranking + +Reranking provides a relevance boost to search results generated by +traditional (lexical) or semantic search strategies. + +As a simple demonstration, take the passages and user query below: + + +```python +query = "What is the capital of the United States?" +docs = [ + "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274.", + "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan.", + "Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas.", + "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America.", + "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment." +] +``` + +The goal of reranking is to provide a more fine-grained quality improvement to +initial search results. With RedisVL, this would likely be results coming back +from a search operation like full text or vector. + +### Using the Cross-Encoder Reranker + +To use the cross-encoder reranker we initialize an instance of `HFCrossEncoderReranker` passing a suitable model (if no model is provided, the `cross-encoder/ms-marco-MiniLM-L-6-v2` model is used): + + +```python +from redisvl.utils.rerank import HFCrossEncoderReranker + +cross_encoder_reranker = HFCrossEncoderReranker("BAAI/bge-reranker-base") +``` + +### Rerank documents with HFCrossEncoderReranker + +With the obtained reranker instance we can rerank and truncate the list of +documents based on relevance to the initial query. + + +```python +results, scores = cross_encoder_reranker.rank(query=query, docs=docs) +``` + + +```python +for result, score in zip(results, scores): + print(score, " -- ", result) +``` + + 0.07461125403642654 -- {'content': 'Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America.'} + 0.05220315232872963 -- {'content': 'Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas.'} + 0.3802368640899658 -- {'content': 'Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274.'} + + +### Using the Cohere Reranker + +To initialize the Cohere reranker you'll need to install the cohere library and provide the right Cohere API Key. + + +```python +#!pip install cohere +``` + + +```python +import getpass + +# setup the API Key +api_key = os.environ.get("COHERE_API_KEY") or getpass.getpass("Enter your Cohere API key: ") +``` + + +```python +from redisvl.utils.rerank import CohereReranker + +cohere_reranker = CohereReranker(limit=3, api_config={"api_key": api_key}) +``` + +### Rerank documents with CohereReranker + +Below we will use the `CohereReranker` to rerank and truncate the list of +documents above based on relevance to the initial query. + + +```python +results, scores = cohere_reranker.rank(query=query, docs=docs) +``` + + +```python +for result, score in zip(results, scores): + print(score, " -- ", result) +``` + + 0.9990564 -- Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America. + 0.7516481 -- Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment. + 0.08882029 -- The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan. + + +### Working with semi-structured documents + +Often times the initial result set includes other metadata and components that could be used to steer the reranking relevancy. To accomplish this, we can set the `rank_by` argument and provide documents with those additional fields. + + +```python +docs = [ + { + "source": "wiki", + "passage": "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274." + }, + { + "source": "encyclopedia", + "passage": "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan." + }, + { + "source": "textbook", + "passage": "Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas." + }, + { + "source": "textbook", + "passage": "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America." + }, + { + "source": "wiki", + "passage": "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment." + } +] +``` + + +```python +results, scores = cohere_reranker.rank(query=query, docs=docs, rank_by=["passage", "source"]) +``` + + +```python +for result, score in zip(results, scores): + print(score, " -- ", result) +``` + + 0.9988121 -- {'source': 'textbook', 'passage': 'Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America.'} + 0.5974905 -- {'source': 'wiki', 'passage': 'Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment.'} + 0.059101548 -- {'source': 'encyclopedia', 'passage': 'The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan.'} + + +### Using the VoyageAI Reranker + +To initialize the VoyageAI reranker you'll need to install the voyaeai library and provide the right VoyageAI API Key. + + +```python +#!pip install voyageai +``` + + +```python +import getpass + +# setup the API Key +api_key = os.environ.get("VOYAGE_API_KEY") or getpass.getpass("Enter your VoyageAI API key: ") +``` + + +```python +from redisvl.utils.rerank import VoyageAIReranker + +reranker = VoyageAIReranker(model="rerank-lite-1", limit=3, api_config={"api_key": api_key})# Please check the available models at https://docs.voyageai.com/docs/reranker +``` + +### Rerank documents with VoyageAIReranker + +Below we will use the `VoyageAIReranker` to rerank and also truncate the list of +documents above based on relevance to the initial query. + + +```python +results, scores = reranker.rank(query=query, docs=docs) +``` + + +```python +for result, score in zip(results, scores): + print(score, " -- ", result) +``` + + 0.796875 -- Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America. + 0.578125 -- Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas. + 0.5625 -- Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274. + diff --git a/content/integrate/redisvl/user_guide/07_session_manager.md b/content/integrate/redisvl/user_guide/07_session_manager.md new file mode 100644 index 0000000000..05b1b292e1 --- /dev/null +++ b/content/integrate/redisvl/user_guide/07_session_manager.md @@ -0,0 +1,198 @@ +--- +linkTitle: LLM Session Memory +title: LLM Session Memory +type: integration +--- + + +Large Language Models are inherently stateless and have no knowledge of previous interactions with a user, or even of previous parts of the current conversation. While this may not be noticable when asking simple questions, it becomes a hinderance when engaging in long running conversations that rely on conversational context. + +The solution to this problem is to append the previous conversation history to each subsequent call to the LLM. + +This notebook will show how to use Redis to structure and store and retrieve this conversational session memory. + + +```python +from redisvl.extensions.session_manager import StandardSessionManager +chat_session = StandardSessionManager(name='student tutor') +``` + + 12:24:11 redisvl.index.index INFO Index already exists, not overwriting. + + +To align with common LLM APIs, Redis stores messages with `role` and `content` fields. +The supported roles are "system", "user" and "llm". + +You can store messages one at a time or all at once. + + +```python +chat_session.add_message({"role":"system", "content":"You are a helpful geography tutor, giving simple and short answers to questions about Europen countries."}) +chat_session.add_messages([ + {"role":"user", "content":"What is the capital of France?"}, + {"role":"llm", "content":"The capital is Paris."}, + {"role":"user", "content":"And what is the capital of Spain?"}, + {"role":"llm", "content":"The capital is Madrid."}, + {"role":"user", "content":"What is the population of Great Britain?"}, + {"role":"llm", "content":"As of 2023 the population of Great Britain is approximately 67 million people."},] + ) +``` + +At any point we can retrieve the recent history of the conversation. It will be ordered by entry time. + + +```python +context = chat_session.get_recent() +for message in context: + print(message) +``` + + {'role': 'llm', 'content': 'The capital is Paris.'} + {'role': 'user', 'content': 'And what is the capital of Spain?'} + {'role': 'llm', 'content': 'The capital is Madrid.'} + {'role': 'user', 'content': 'What is the population of Great Britain?'} + {'role': 'llm', 'content': 'As of 2023 the population of Great Britain is approximately 67 million people.'} + + +In many LLM flows the conversation progresses in a series of prompt and response pairs. session managers offer a convienience function `store()` to add these simply. + + +```python +prompt = "what is the size of England compared to Portugal?" +response = "England is larger in land area than Portal by about 15000 square miles." +chat_session.store(prompt, response) + +context = chat_session.get_recent(top_k=6) +for message in context: + print(message) +``` + + {'role': 'user', 'content': 'And what is the capital of Spain?'} + {'role': 'llm', 'content': 'The capital is Madrid.'} + {'role': 'user', 'content': 'What is the population of Great Britain?'} + {'role': 'llm', 'content': 'As of 2023 the population of Great Britain is approximately 67 million people.'} + {'role': 'user', 'content': 'what is the size of England compared to Portugal?'} + {'role': 'llm', 'content': 'England is larger in land area than Portal by about 15000 square miles.'} + + +## Managing multiple users and conversations + +For applications that need to handle multiple conversations concurrently, Redis supports tagging messages to keep conversations separated. + + +```python +chat_session.add_message({"role":"system", "content":"You are a helpful algebra tutor, giving simple answers to math problems."}, session_tag='student two') +chat_session.add_messages([ + {"role":"user", "content":"What is the value of x in the equation 2x + 3 = 7?"}, + {"role":"llm", "content":"The value of x is 2."}, + {"role":"user", "content":"What is the value of y in the equation 3y - 5 = 7?"}, + {"role":"llm", "content":"The value of y is 4."}], + session_tag='student two' + ) + +for math_message in chat_session.get_recent(session_tag='student two'): + print(math_message) +``` + + {'role': 'system', 'content': 'You are a helpful algebra tutor, giving simple answers to math problems.'} + {'role': 'user', 'content': 'What is the value of x in the equation 2x + 3 = 7?'} + {'role': 'llm', 'content': 'The value of x is 2.'} + {'role': 'user', 'content': 'What is the value of y in the equation 3y - 5 = 7?'} + {'role': 'llm', 'content': 'The value of y is 4.'} + + +## Semantic conversation memory +For longer conversations our list of messages keeps growing. Since LLMs are stateless we have to continue to pass this conversation history on each subsequent call to ensure the LLM has the correct context. + +A typical flow looks like this: +``` +while True: + prompt = input('enter your next question') + context = chat_session.get_recent() + response = LLM_api_call(prompt=prompt, context=context) + chat_session.store(prompt, response) +``` + +This works, but as context keeps growing so too does our LLM token count, which increases latency and cost. + +Conversation histories can be truncated, but that can lead to losing relevant information that appeared early on. + +A better solution is to pass only the relevant conversational context on each subsequent call. + +For this, RedisVL has the `SemanticSessionManager`, which uses vector similarity search to return only semantically relevant sections of the conversation. + + +```python +from redisvl.extensions.session_manager import SemanticSessionManager +semantic_session = SemanticSessionManager(name='tutor') + +semantic_session.add_messages(chat_session.get_recent(top_k=8)) +``` + + 12:24:15 redisvl.index.index INFO Index already exists, not overwriting. + + + +```python +prompt = "what have I learned about the size of England?" +semantic_session.set_distance_threshold(0.35) +context = semantic_session.get_relevant(prompt) +for message in context: + print(message) +``` + + {'role': 'user', 'content': 'what is the size of England compared to Portugal?'} + {'role': 'llm', 'content': 'England is larger in land area than Portal by about 15000 square miles.'} + + +You can adjust the degree of semantic similarity needed to be included in your context. + +Setting a distance threshold close to 0.0 will require an exact semantic match, while a distance threshold of 1.0 will include everthing. + + +```python +semantic_session.set_distance_threshold(0.7) + +larger_context = semantic_session.get_relevant(prompt) +for message in larger_context: + print(message) +``` + + {'role': 'user', 'content': 'what is the size of England compared to Portugal?'} + {'role': 'llm', 'content': 'England is larger in land area than Portal by about 15000 square miles.'} + {'role': 'user', 'content': 'What is the population of Great Britain?'} + {'role': 'llm', 'content': 'As of 2023 the population of Great Britain is approximately 67 million people.'} + + +## Conversation control + +LLMs can hallucinate on occasion and when this happens it can be useful to prune incorrect information from conversational histories so this incorrect information doesn't continue to be passed as context. + + +```python +semantic_session.store( + prompt="what is the smallest country in Europe?", + response="Monaco is the smallest country in Europe at 0.78 square miles." # Incorrect. Vatican City is the smallest country in Europe + ) + +# get the key of the incorrect message +context = semantic_session.get_recent(top_k=1, raw=True) +bad_key = context[0]['entry_id'] +semantic_session.drop(bad_key) + +corrected_context = semantic_session.get_recent() +for message in corrected_context: + print(message) +``` + + {'role': 'user', 'content': 'What is the population of Great Britain?'} + {'role': 'llm', 'content': 'As of 2023 the population of Great Britain is approximately 67 million people.'} + {'role': 'user', 'content': 'what is the size of England compared to Portugal?'} + {'role': 'llm', 'content': 'England is larger in land area than Portal by about 15000 square miles.'} + {'role': 'user', 'content': 'what is the smallest country in Europe?'} + + + +```python +chat_session.clear() +``` diff --git a/content/integrate/redisvl/user_guide/08_semantic_router.md b/content/integrate/redisvl/user_guide/08_semantic_router.md new file mode 100644 index 0000000000..b144455506 --- /dev/null +++ b/content/integrate/redisvl/user_guide/08_semantic_router.md @@ -0,0 +1,315 @@ +--- +linkTitle: Semantic Routing +title: Semantic Routing +type: integration +--- + + +RedisVL provides a `SemanticRouter` interface to utilize Redis' built-in search & aggregation in order to perform +KNN-style classification over a set of `Route` references to determine the best match. + +This notebook will go over how to use Redis as a Semantic Router for your applications + +## Define the Routes + +Below we define 3 different routes. One for `technology`, one for `sports`, and +another for `entertainment`. Now for this example, the goal here is +surely topic "classification". But you can create routes and references for +almost anything. + +Each route has a set of references that cover the "semantic surface area" of the +route. The incoming query from a user needs to be semantically similar to one or +more of the references in order to "match" on the route. + +Additionally, each route has a `distance_threshold` which determines the maximum distance between the query and the reference for the query to be routed to the route. This value is unique to each route. + + +```python +from redisvl.extensions.router import Route + + +# Define routes for the semantic router +technology = Route( + name="technology", + references=[ + "what are the latest advancements in AI?", + "tell me about the newest gadgets", + "what's trending in tech?" + ], + metadata={"category": "tech", "priority": 1}, + distance_threshold=1.0 +) + +sports = Route( + name="sports", + references=[ + "who won the game last night?", + "tell me about the upcoming sports events", + "what's the latest in the world of sports?", + "sports", + "basketball and football" + ], + metadata={"category": "sports", "priority": 2}, + distance_threshold=0.5 +) + +entertainment = Route( + name="entertainment", + references=[ + "what are the top movies right now?", + "who won the best actor award?", + "what's new in the entertainment industry?" + ], + metadata={"category": "entertainment", "priority": 3}, + distance_threshold=0.7 +) + +``` + +## Initialize the SemanticRouter + +``SemanticRouter`` will automatically create an index within Redis upon initialization for the route references. By default, it uses the `HFTextVectorizer` to +generate embeddings for each route reference. + + +```python +import os +from redisvl.extensions.router import SemanticRouter +from redisvl.utils.vectorize import HFTextVectorizer + +os.environ["TOKENIZERS_PARALLELISM"] = "false" + +# Initialize the SemanticRouter +router = SemanticRouter( + name="topic-router", + vectorizer=HFTextVectorizer(), + routes=[technology, sports, entertainment], + redis_url="redis://localhost:6379", + overwrite=True # Blow away any other routing index with this name +) +``` + + /Users/robert.shelton/.pyenv/versions/3.11.9/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. + warnings.warn( + /Users/robert.shelton/.pyenv/versions/3.11.9/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. + warnings.warn( + + + 14:07:31 redisvl.index.index INFO Index already exists, overwriting. + + + +```python +router.vectorizer +``` + + + + + HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2', dims=768) + + + + +```python +# look at the index specification created for the semantic router +!rvl index info -i topic-router +``` + + + + Index Information: + ╭──────────────┬────────────────┬──────────────────┬─────────────────┬────────────╮ + │ Index Name │ Storage Type │ Prefixes │ Index Options │ Indexing │ + ├──────────────┼────────────────┼──────────────────┼─────────────────┼────────────┤ + │ topic-router │ HASH │ ['topic-router'] │ [] │ 0 │ + ╰──────────────┴────────────────┴──────────────────┴─────────────────┴────────────╯ + Index Fields: + ╭────────────┬─────────────┬────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬─────────────────┬────────────────╮ + │ Name │ Attribute │ Type │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ + ├────────────┼─────────────┼────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼─────────────────┼────────────────┤ + │ route_name │ route_name │ TAG │ SEPARATOR │ , │ │ │ │ │ │ │ + │ reference │ reference │ TEXT │ WEIGHT │ 1 │ │ │ │ │ │ │ + │ vector │ vector │ VECTOR │ algorithm │ FLAT │ data_type │ FLOAT32 │ dim │ 768 │ distance_metric │ COSINE │ + ╰────────────┴─────────────┴────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴─────────────────┴────────────────╯ + + +## Simple routing + + +```python +# Query the router with a statement +route_match = router("Can you tell me about the latest in artificial intelligence?") +route_match +``` + + + + + RouteMatch(name='technology', distance=0.119614303112) + + + + +```python +# Query the router with a statement and return a miss +route_match = router("are aliens real?") +route_match +``` + + + + + RouteMatch(name=None, distance=None) + + + + +```python +# Toggle the runtime distance threshold +route_match = router("Which basketball team will win the NBA finals?") +route_match +``` + + + + + RouteMatch(name=None, distance=None) + + + +We can also route a statement to many routes and order them by distance: + + +```python +# Perform multi-class classification with route_many() -- toggle the max_k and the distance_threshold +route_matches = router.route_many("Lebron James", max_k=3) +route_matches +``` + + + + + [] + + + + +```python +# Toggle the aggregation method -- note the different distances in the result +from redisvl.extensions.router.schema import DistanceAggregationMethod + +route_matches = router.route_many("Lebron James", aggregation_method=DistanceAggregationMethod.min, max_k=3) +route_matches +``` + + + + + [] + + + +Note the different route match distances. This is because we used the `min` aggregation method instead of the default `avg` approach. + +## Update the routing config + + +```python +from redisvl.extensions.router import RoutingConfig + +router.update_routing_config( + RoutingConfig(aggregation_method=DistanceAggregationMethod.min, max_k=3) +) +``` + + +```python +route_matches = router.route_many("Lebron James") +route_matches +``` + + + + + [] + + + +## Router serialization + + +```python +router.to_dict() +``` + + + + + {'name': 'topic-router', + 'routes': [{'name': 'technology', + 'references': ['what are the latest advancements in AI?', + 'tell me about the newest gadgets', + "what's trending in tech?"], + 'metadata': {'category': 'tech', 'priority': '1'}, + 'distance_threshold': 1.0}, + {'name': 'sports', + 'references': ['who won the game last night?', + 'tell me about the upcoming sports events', + "what's the latest in the world of sports?", + 'sports', + 'basketball and football'], + 'metadata': {'category': 'sports', 'priority': '2'}, + 'distance_threshold': 0.5}, + {'name': 'entertainment', + 'references': ['what are the top movies right now?', + 'who won the best actor award?', + "what's new in the entertainment industry?"], + 'metadata': {'category': 'entertainment', 'priority': '3'}, + 'distance_threshold': 0.7}], + 'vectorizer': {'type': 'hf', + 'model': 'sentence-transformers/all-mpnet-base-v2'}, + 'routing_config': {'distance_threshold': 0.5, + 'max_k': 3, + 'aggregation_method': 'min'}} + + + + +```python +router2 = SemanticRouter.from_dict(router.to_dict(), redis_url="redis://localhost:6379") + +assert router2.to_dict() == router.to_dict() +``` + + 14:07:34 redisvl.index.index INFO Index already exists, not overwriting. + + + +```python +router.to_yaml("router.yaml", overwrite=True) +``` + + +```python +router3 = SemanticRouter.from_yaml("router.yaml", redis_url="redis://localhost:6379") + +assert router3.to_dict() == router2.to_dict() == router.to_dict() +``` + + 14:07:34 redisvl.index.index INFO Index already exists, not overwriting. + + +## Clean up the router + + +```python +# Use clear to flush all routes from the index +router.clear() +``` + + +```python +# Use delete to clear the index and remove it completely +router.delete() +``` diff --git a/content/integrate/redisvl/user_guide/_index.md b/content/integrate/redisvl/user_guide/_index.md new file mode 100644 index 0000000000..4bcd38f865 --- /dev/null +++ b/content/integrate/redisvl/user_guide/_index.md @@ -0,0 +1,56 @@ +--- +linkTitle: User Guides +title: User Guides +type: integration +--- + + +User guides provide helpful resources for using RedisVL and its different components. + + + +* [Getting Started with RedisVL](01_getting_started/) + * [Define an `IndexSchema`](01_getting_started/#define-an-indexschema) + * [Sample Dataset Preparation](01_getting_started/#sample-dataset-preparation) + * [Create a `SearchIndex`](01_getting_started/#create-a-searchindex) + * [Inspect with the `rvl` CLI](01_getting_started/#inspect-with-the-rvl-cli) + * [Load Data to `SearchIndex`](01_getting_started/#load-data-to-searchindex) + * [Creating `VectorQuery` Objects](01_getting_started/#creating-vectorquery-objects) + * [Using an Asynchronous Redis Client](01_getting_started/#using-an-asynchronous-redis-client) + * [Updating a schema](01_getting_started/#updating-a-schema) + * [Check Index Stats](01_getting_started/#check-index-stats) + * [Cleanup](01_getting_started/#cleanup) +* [Querying with RedisVL](02_hybrid_queries/) + * [Hybrid Queries](02_hybrid_queries/#hybrid-queries) + * [Combining Filters](02_hybrid_queries/#combining-filters) + * [Non-vector Queries](02_hybrid_queries/#non-vector-queries) + * [Count Queries](02_hybrid_queries/#count-queries) + * [Range Queries](02_hybrid_queries/#range-queries) + * [Advanced Query Modifiers](02_hybrid_queries/#advanced-query-modifiers) +* [Semantic Caching for LLMs](03_llmcache/) + * [Initializing `SemanticCache`](03_llmcache/#initializing-semanticcache) + * [Basic Cache Usage](03_llmcache/#basic-cache-usage) + * [Customize the Distance Threshhold](03_llmcache/#customize-the-distance-threshhold) + * [Utilize TTL](03_llmcache/#utilize-ttl) + * [Simple Performance Testing](03_llmcache/#simple-performance-testing) + * [Cache Access Controls, Tags & Filters](03_llmcache/#cache-access-controls-tags-filters) +* [Vectorizers](04_vectorizers/) + * [Creating Text Embeddings](04_vectorizers/#creating-text-embeddings) + * [Search with Provider Embeddings](04_vectorizers/#search-with-provider-embeddings) + * [Selecting your float data type](04_vectorizers/#selecting-your-float-data-type) +* [Hash vs JSON Storage](05_hash_vs_json/) + * [Hash or JSON – how to choose?](05_hash_vs_json/#hash-or-json-how-to-choose) + * [Cleanup](05_hash_vs_json/#cleanup) +* [Rerankers](06_rerankers/) + * [Simple Reranking](06_rerankers/#simple-reranking) +* [LLM Session Memory](07_session_manager/) + * [Managing multiple users and conversations](07_session_manager/#managing-multiple-users-and-conversations) + * [Semantic conversation memory](07_session_manager/#semantic-conversation-memory) + * [Conversation control](07_session_manager/#conversation-control) +* [Semantic Routing](08_semantic_router/) + * [Define the Routes](08_semantic_router/#define-the-routes) + * [Initialize the SemanticRouter](08_semantic_router/#initialize-the-semanticrouter) + * [Simple routing](08_semantic_router/#simple-routing) + * [Update the routing config](08_semantic_router/#update-the-routing-config) + * [Router serialization](08_semantic_router/#router-serialization) + * [Clean up the router](08_semantic_router/#clean-up-the-router) From b33287c785c97fd0646e27bd0e786cf20e397988 Mon Sep 17 00:00:00 2001 From: paoloredis Date: Thu, 20 Mar 2025 16:28:55 +0100 Subject: [PATCH 02/13] Fix relrefs --- content/integrate/redisvl/api/cache.md | 12 +++---- content/integrate/redisvl/api/filter.md | 34 ++++++++++---------- content/integrate/redisvl/api/query.md | 24 +++++++------- content/integrate/redisvl/api/router.md | 22 ++++++------- content/integrate/redisvl/api/schema.md | 4 +-- content/integrate/redisvl/api/searchindex.md | 12 +++---- 6 files changed, 54 insertions(+), 54 deletions(-) diff --git a/content/integrate/redisvl/api/cache.md b/content/integrate/redisvl/api/cache.md index cbfc7169b0..7e8cc42937 100644 --- a/content/integrate/redisvl/api/cache.md +++ b/content/integrate/redisvl/api/cache.md @@ -61,7 +61,7 @@ LLM responses. * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to include in each returned result. If None, defaults to all available fields in the cached entry. - * **filter_expression** (*Optional* *[*[*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]*) – Optional filter expression + * **filter_expression** (*Optional* *[*[*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]*) – Optional filter expression that can be used to filter cache results. Defaults to None and the full cache will be searched. * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic @@ -167,7 +167,7 @@ LLM responses. * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to include in each returned result. If None, defaults to all available fields in the cached entry. - * **filter_expression** (*Optional* *[*[*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]*) – Optional filter expression + * **filter_expression** (*Optional* *[*[*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]*) – Optional filter expression that can be used to filter cache results. Defaults to None and the full cache will be searched. * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic @@ -288,14 +288,14 @@ cache.update(key, metadata={"hit_count": 1, "model_name": "Llama-2-7b"}) ) ``` -#### *property* aindex *: [AsyncSearchIndex](searchindex.md#redisvl.index.AsyncSearchIndex) | None* +#### *property* aindex *: [AsyncSearchIndex]({{< relref "searchindex/#asyncsearchindex" >}}) | None* The underlying AsyncSearchIndex for the cache. * **Returns:** The async search index. * **Return type:** - [AsyncSearchIndex](searchindex.md#redisvl.index.AsyncSearchIndex) + [AsyncSearchIndex]({{< relref "searchindex/#asyncsearchindex" >}}) #### *property* distance_threshold *: float* @@ -306,14 +306,14 @@ The semantic distance threshold for the cache. * **Return type:** float -#### *property* index *: [SearchIndex](searchindex.md#redisvl.index.SearchIndex)* +#### *property* index *: [SearchIndex]({{< relref "searchindex/#searchindex" >}})* The underlying SearchIndex for the cache. * **Returns:** The search index. * **Return type:** - [SearchIndex](searchindex.md#redisvl.index.SearchIndex) + [SearchIndex]({{< relref "searchindex/#searchindex" >}}) #### *property* ttl *: int | None* diff --git a/content/integrate/redisvl/api/filter.md b/content/integrate/redisvl/api/filter.md index b61699e6f1..65d1d014cd 100644 --- a/content/integrate/redisvl/api/filter.md +++ b/content/integrate/redisvl/api/filter.md @@ -51,8 +51,8 @@ built by combining filter statements using the & and | operators. * **Parameters:** * **\_filter** (*str* *|* *None*) * **operator** (*FilterOperator* *|* *None*) - * **left** ([*FilterExpression*](#redisvl.query.filter.FilterExpression) *|* *None*) - * **right** ([*FilterExpression*](#redisvl.query.filter.FilterExpression) *|* *None*) + * **left** ([*FilterExpression*](#filterexpression) *|* *None*) + * **right** ([*FilterExpression*](#filterexpression) *|* *None*) ## Tag @@ -70,7 +70,7 @@ Create a Tag equality filter expression. * **Parameters:** **other** (*Union* *[**List* *[**str* *]* *,* *str* *]*) – The tag(s) to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Tag @@ -85,7 +85,7 @@ Create a Tag inequality filter expression. * **Parameters:** **other** (*Union* *[**List* *[**str* *]* *,* *str* *]*) – The tag(s) to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Tag @@ -116,7 +116,7 @@ filters that enforce an exact match on the supplied term(s). * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Text @@ -134,7 +134,7 @@ term(s). * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Text @@ -154,7 +154,7 @@ equality filter expression. * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Text @@ -185,7 +185,7 @@ Create a Numeric equality filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Num @@ -199,7 +199,7 @@ Create a Numeric greater than or equal to filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Num @@ -214,7 +214,7 @@ Create a Numeric greater than filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Num @@ -229,7 +229,7 @@ Create a Numeric less than or equal to filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Num @@ -244,7 +244,7 @@ Create a Numeric less than filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Num @@ -259,7 +259,7 @@ Create a Numeric inequality filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Num @@ -289,9 +289,9 @@ Redis index. Create a geographic filter within a specified GeoRadius. * **Parameters:** - **other** ([*GeoRadius*](#redisvl.query.filter.GeoRadius)) – The geographic spec to filter on. + **other** ([*GeoRadius*](#georadius)) – The geographic spec to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Geo, GeoRadius @@ -304,9 +304,9 @@ f = Geo("location") == GeoRadius(-122.4194, 37.7749, 1, unit="m") Create a geographic filter outside of a specified GeoRadius. * **Parameters:** - **other** ([*GeoRadius*](#redisvl.query.filter.GeoRadius)) – The geographic spec to filter on. + **other** ([*GeoRadius*](#georadius)) – The geographic spec to filter on. * **Return type:** - [*FilterExpression*](#redisvl.query.filter.FilterExpression) + [*FilterExpression*](#filterexpression) ```python from redisvl.query.filter import Geo, GeoRadius diff --git a/content/integrate/redisvl/api/query.md b/content/integrate/redisvl/api/query.md index b866c890dd..1a2b86af1c 100644 --- a/content/integrate/redisvl/api/query.md +++ b/content/integrate/redisvl/api/query.md @@ -24,7 +24,7 @@ expression. against in the database. * **return_fields** (*List* *[**str* *]*) – The declared fields to return with search results. - * **filter_expression** (*Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *,* *optional*) – A filter to apply + * **filter_expression** (*Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *,* *optional*) – A filter to apply along with the vector search. Defaults to None. * **dtype** (*str* *,* *optional*) – The dtype of the vector. Defaults to “float32”. @@ -165,7 +165,7 @@ Default is TFIDF. Set the filter expression for the query. * **Parameters:** - **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]* *,* *optional*) – The filter + **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter expression or query string to use on the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. @@ -224,7 +224,7 @@ Ask the engine to return document search scores. * **Return type:** *Query* -#### *property* filter *: str | [FilterExpression](filter.md#redisvl.query.filter.FilterExpression)* +#### *property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})* The filter expression for the query. @@ -256,7 +256,7 @@ distance threshold. against in the database. * **return_fields** (*List* *[**str* *]*) – The declared fields to return with search results. - * **filter_expression** (*Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *,* *optional*) – A filter to apply + * **filter_expression** (*Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *,* *optional*) – A filter to apply along with the range query. Defaults to None. * **dtype** (*str* *,* *optional*) – The dtype of the vector. Defaults to “float32”. @@ -407,7 +407,7 @@ Set the distance threshold for the query. Set the filter expression for the query. * **Parameters:** - **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]* *,* *optional*) – The filter + **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter expression or query string to use on the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. @@ -475,7 +475,7 @@ Return the distance threshold for the query. * **Return type:** float -#### *property* filter *: str | [FilterExpression](filter.md#redisvl.query.filter.FilterExpression)* +#### *property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})* The filter expression for the query. @@ -501,7 +501,7 @@ Bases: `BaseQuery` A query for running a filtered search with a filter expression. * **Parameters:** - * **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]*) – The optional filter + * **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]*) – The optional filter expression to query with. Defaults to ‘\*’. * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to return. * **num_results** (*Optional* *[**int* *]* *,* *optional*) – The number of results to return. Defaults to 10. @@ -632,7 +632,7 @@ Default is TFIDF. Set the filter expression for the query. * **Parameters:** - **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]* *,* *optional*) – The filter + **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter expression or query string to use on the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. @@ -691,7 +691,7 @@ Ask the engine to return document search scores. * **Return type:** *Query* -#### *property* filter *: str | [FilterExpression](filter.md#redisvl.query.filter.FilterExpression)* +#### *property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})* The filter expression for the query. @@ -712,7 +712,7 @@ Bases: `BaseQuery` A query for a simple count operation provided some filter expression. * **Parameters:** - * **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]*) – The filter expression to query with. Defaults to None. + * **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]*) – The filter expression to query with. Defaults to None. * **params** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The parameters for the query. Defaults to None. * **dialect** (*int*) * **Raises:** @@ -848,7 +848,7 @@ Default is TFIDF. Set the filter expression for the query. * **Parameters:** - **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*](filter.md#redisvl.query.filter.FilterExpression) *]* *]* *,* *optional*) – The filter + **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter expression or query string to use on the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. @@ -907,7 +907,7 @@ Ask the engine to return document search scores. * **Return type:** *Query* -#### *property* filter *: str | [FilterExpression](filter.md#redisvl.query.filter.FilterExpression)* +#### *property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})* The filter expression for the query. diff --git a/content/integrate/redisvl/api/router.md b/content/integrate/redisvl/api/router.md index 6ee03e5204..c329c8d1f0 100644 --- a/content/integrate/redisvl/api/router.md +++ b/content/integrate/redisvl/api/router.md @@ -17,9 +17,9 @@ Initialize the SemanticRouter. * **Parameters:** * **name** (*str*) – The name of the semantic router. - * **routes** (*List* *[*[*Route*](#redisvl.extensions.router.Route) *]*) – List of Route objects. + * **routes** (*List* *[*[*Route*](#route) *]*) – List of Route objects. * **vectorizer** (*BaseVectorizer* *,* *optional*) – The vectorizer used to embed route references. Defaults to default HFTextVectorizer. - * **routing_config** ([*RoutingConfig*](#redisvl.extensions.router.RoutingConfig) *,* *optional*) – Configuration for routing behavior. Defaults to the default RoutingConfig. + * **routing_config** ([*RoutingConfig*](#routingconfig) *,* *optional*) – Configuration for routing behavior. Defaults to the default RoutingConfig. * **redis_client** (*Optional* *[**Redis* *]* *,* *optional*) – Redis client for connection. Defaults to None. * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. * **overwrite** (*bool* *,* *optional*) – Whether to overwrite existing index. Defaults to False. @@ -49,7 +49,7 @@ Create a SemanticRouter from a dictionary. * **Returns:** The semantic router instance. * **Return type:** - [SemanticRouter](#redisvl.extensions.router.SemanticRouter) + [SemanticRouter](#semanticrouter) * **Raises:** **ValueError** – If required data is missing or invalid. @@ -72,7 +72,7 @@ Create a SemanticRouter from a YAML file. * **Returns:** The semantic router instance. * **Return type:** - [SemanticRouter](#redisvl.extensions.router.SemanticRouter) + [SemanticRouter](#semanticrouter) * **Raises:** * **ValueError** – If the file path is invalid. * **FileNotFoundError** – If the file does not exist. @@ -91,7 +91,7 @@ Get a route by its name. * **Returns:** The selected Route object or None if not found. * **Return type:** - Optional[[Route](#redisvl.extensions.router.Route)] + Optional[[Route](#route)] #### model_post_init(context, /) @@ -123,11 +123,11 @@ Query the semantic router with a given statement or vector for multiple matches. * **vector** (*Optional* *[**List* *[**float* *]* *]*) – The input vector to be queried. * **max_k** (*Optional* *[**int* *]*) – The maximum number of top matches to return. * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic distance. - * **aggregation_method** (*Optional* *[*[*DistanceAggregationMethod*](#redisvl.extensions.router.schema.DistanceAggregationMethod) *]*) – The aggregation method used for vector distances. + * **aggregation_method** (*Optional* *[*[*DistanceAggregationMethod*](#distanceaggregationmethod) *]*) – The aggregation method used for vector distances. * **Returns:** The matching routes and their details. * **Return type:** - List[[RouteMatch](#redisvl.extensions.router.schema.RouteMatch)] + List[[RouteMatch](#routematch)] #### to_dict() @@ -171,7 +171,7 @@ router.to_yaml("router.yaml") Update the routing configuration. * **Parameters:** - **routing_config** ([*RoutingConfig*](#redisvl.extensions.router.RoutingConfig)) – The new routing configuration. + **routing_config** ([*RoutingConfig*](#routingconfig)) – The new routing configuration. #### model_config *: ClassVar[ConfigDict]* *= {'arbitrary_types_allowed': True}* @@ -199,11 +199,11 @@ Get the distance thresholds for each route. * **Return type:** Dict[str, float] -#### routes *: List[[Route](#redisvl.extensions.router.Route)]* +#### routes *: List[[Route](#route)]* List of Route objects. -#### routing_config *: [RoutingConfig](#redisvl.extensions.router.RoutingConfig)* +#### routing_config *: [RoutingConfig](#routingconfig)* Configuration for routing behavior. @@ -226,7 +226,7 @@ self is explicitly positional-only to allow self as a field name. * **Parameters:** * **max_k** (*Annotated* *[**int* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=False* *,* *default=1* *,* *metadata=* *[**Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *]* *)* *]*) - * **aggregation_method** ([*DistanceAggregationMethod*](#redisvl.extensions.router.schema.DistanceAggregationMethod)) + * **aggregation_method** ([*DistanceAggregationMethod*](#distanceaggregationmethod)) #### max_k *: Annotated[int, FieldInfo(annotation=NoneType, required=False, default=1, metadata=[Strict(strict=True), Gt(gt=0)])]* diff --git a/content/integrate/redisvl/api/schema.md b/content/integrate/redisvl/api/schema.md index dd52f67269..1191d5f7ac 100644 --- a/content/integrate/redisvl/api/schema.md +++ b/content/integrate/redisvl/api/schema.md @@ -173,7 +173,7 @@ Create an IndexSchema from a dictionary. * **Returns:** The index schema. * **Return type:** - [IndexSchema](#redisvl.schema.IndexSchema) + [IndexSchema](#indexschema) ```python from redisvl.schema import IndexSchema @@ -210,7 +210,7 @@ Create an IndexSchema from a YAML file. * **Returns:** The index schema. * **Return type:** - [IndexSchema](#redisvl.schema.IndexSchema) + [IndexSchema](#indexschema) ```python from redisvl.schema import IndexSchema diff --git a/content/integrate/redisvl/api/searchindex.md b/content/integrate/redisvl/api/searchindex.md index edac140c40..975159cfee 100644 --- a/content/integrate/redisvl/api/searchindex.md +++ b/content/integrate/redisvl/api/searchindex.md @@ -44,7 +44,7 @@ Initialize the RedisVL search index with a schema, Redis client kwargs. * **Parameters:** - * **schema** ([*IndexSchema*](schema.md#redisvl.schema.IndexSchema)) – Index schema object. + * **schema** ([*IndexSchema*]({{< relref "schema/#indexschema" >}})) – Index schema object. * **redis_client** (*Optional* *[**redis.Redis* *]*) – An instantiated redis client. * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to @@ -183,7 +183,7 @@ Create a SearchIndex from a dictionary. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [SearchIndex](#redisvl.index.SearchIndex) + [SearchIndex](#searchindex) ```python from redisvl.index import SearchIndex @@ -220,7 +220,7 @@ Create a SearchIndex from a YAML schema file. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [SearchIndex](#redisvl.index.SearchIndex) + [SearchIndex](#searchindex) ```python from redisvl.index import SearchIndex @@ -467,7 +467,7 @@ await index.delete(drop=True) Initialize the RedisVL async search index with a schema. * **Parameters:** - * **schema** ([*IndexSchema*](schema.md#redisvl.schema.IndexSchema)) – Index schema object. + * **schema** ([*IndexSchema*]({{< relref "schema/#indexschema" >}})) – Index schema object. * **connection_args** (*Dict* *[**str* *,* *Any* *]* *,* *optional*) – Redis client connection args. @@ -600,7 +600,7 @@ Create a SearchIndex from a dictionary. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [SearchIndex](#redisvl.index.SearchIndex) + [SearchIndex](#searchindex) ```python from redisvl.index import SearchIndex @@ -637,7 +637,7 @@ Create a SearchIndex from a YAML schema file. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [SearchIndex](#redisvl.index.SearchIndex) + [SearchIndex](#searchindex) ```python from redisvl.index import SearchIndex From b026e02cad11328dfba2108777124bc76af062cd Mon Sep 17 00:00:00 2001 From: paoloredis Date: Fri, 21 Mar 2025 10:40:21 +0100 Subject: [PATCH 03/13] Format class and function signatures --- content/integrate/redisvl/api/cache.md | 34 ++-- content/integrate/redisvl/api/filter.md | 50 ++--- content/integrate/redisvl/api/query.md | 192 +++++++++--------- content/integrate/redisvl/api/reranker.md | 30 +-- content/integrate/redisvl/api/router.md | 72 +++---- content/integrate/redisvl/api/schema.md | 28 +-- content/integrate/redisvl/api/searchindex.md | 108 +++++----- .../integrate/redisvl/api/session_manager.md | 38 ++-- content/integrate/redisvl/api/vectorizer.md | 92 ++++----- 9 files changed, 322 insertions(+), 322 deletions(-) diff --git a/content/integrate/redisvl/api/cache.md b/content/integrate/redisvl/api/cache.md index 7e8cc42937..04f2eae7dc 100644 --- a/content/integrate/redisvl/api/cache.md +++ b/content/integrate/redisvl/api/cache.md @@ -9,7 +9,7 @@ type: integration -### *class* SemanticCache(name='llmcache', distance_threshold=0.1, ttl=None, vectorizer=None, filterable_fields=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, \*\*kwargs) +### `*class* SemanticCache(name='llmcache', distance_threshold=0.1, ttl=None, vectorizer=None, filterable_fields=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, **kwargs)` Bases: `BaseLLMCache` @@ -41,7 +41,7 @@ Semantic Cache for Large Language Models. * **ValueError** – If the threshold is not between 0 and 1. * **ValueError** – If existing schema does not match new schema and overwrite is False. -#### *async* acheck(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None) +#### `*async* acheck(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None)` Async check the semantic cache for results similar to the specified prompt or vector. @@ -82,7 +82,7 @@ response = await cache.acheck( ) ``` -#### *async* adrop(ids=None, keys=None) +#### `*async* adrop(ids=None, keys=None)` Async expire specific entries from the cache by id or specific Redis key. @@ -93,7 +93,7 @@ Redis key. * **Return type:** None -#### *async* astore(prompt, response, vector=None, metadata=None, filters=None, ttl=None) +#### `*async* astore(prompt, response, vector=None, metadata=None, filters=None, ttl=None)` Async stores the specified key-value pair in the cache along with metadata. @@ -126,7 +126,7 @@ key = await cache.astore( ) ``` -#### *async* aupdate(key, \*\*kwargs) +#### `*async* aupdate(key, **kwargs)` Async update specific fields within an existing cache entry. If no fields are passed, then only the document TTL is refreshed. @@ -147,7 +147,7 @@ await cache.aupdate( ) ``` -#### check(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None) +#### `check(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None)` Checks the semantic cache for results similar to the specified prompt or vector. @@ -188,14 +188,14 @@ response = cache.check( ) ``` -#### clear() +#### `clear()` Clear the cache of all keys while preserving the index. * **Return type:** None -#### delete() +#### `delete()` Clear the semantic cache of all keys and remove the underlying search index. @@ -203,7 +203,7 @@ index. * **Return type:** None -#### drop(ids=None, keys=None) +#### `drop(ids=None, keys=None)` Manually expire specific entries from the cache by id or specific Redis key. @@ -214,7 +214,7 @@ Redis key. * **Return type:** None -#### set_threshold(distance_threshold) +#### `set_threshold(distance_threshold)` Sets the semantic distance threshold for the cache. @@ -226,7 +226,7 @@ Sets the semantic distance threshold for the cache. * **Return type:** None -#### set_ttl(ttl=None) +#### `set_ttl(ttl=None)` Set the default TTL, in seconds, for entries in the cache. @@ -236,7 +236,7 @@ Set the default TTL, in seconds, for entries in the cache. * **Raises:** **ValueError** – If the time-to-live value is not an integer. -#### store(prompt, response, vector=None, metadata=None, filters=None, ttl=None) +#### `store(prompt, response, vector=None, metadata=None, filters=None, ttl=None)` Stores the specified key-value pair in the cache along with metadata. @@ -269,7 +269,7 @@ key = cache.store( ) ``` -#### update(key, \*\*kwargs) +#### `update(key, **kwargs)` Update specific fields within an existing cache entry. If no fields are passed, then only the document TTL is refreshed. @@ -288,7 +288,7 @@ cache.update(key, metadata={"hit_count": 1, "model_name": "Llama-2-7b"}) ) ``` -#### *property* aindex *: [AsyncSearchIndex]({{< relref "searchindex/#asyncsearchindex" >}}) | None* +#### `*property* aindex *: [AsyncSearchIndex]({{< relref "searchindex/#asyncsearchindex" >}}) | None*` The underlying AsyncSearchIndex for the cache. @@ -297,7 +297,7 @@ The underlying AsyncSearchIndex for the cache. * **Return type:** [AsyncSearchIndex]({{< relref "searchindex/#asyncsearchindex" >}}) -#### *property* distance_threshold *: float* +#### `*property* distance_threshold *: float*` The semantic distance threshold for the cache. @@ -306,7 +306,7 @@ The semantic distance threshold for the cache. * **Return type:** float -#### *property* index *: [SearchIndex]({{< relref "searchindex/#searchindex" >}})* +#### `*property* index *: [SearchIndex]({{< relref "searchindex/#searchindex" >}})*` The underlying SearchIndex for the cache. @@ -315,6 +315,6 @@ The underlying SearchIndex for the cache. * **Return type:** [SearchIndex]({{< relref "searchindex/#searchindex" >}}) -#### *property* ttl *: int | None* +#### `*property* ttl *: int | None*` The default TTL, in seconds, for entries in the cache. diff --git a/content/integrate/redisvl/api/filter.md b/content/integrate/redisvl/api/filter.md index 65d1d014cd..22fb494021 100644 --- a/content/integrate/redisvl/api/filter.md +++ b/content/integrate/redisvl/api/filter.md @@ -9,7 +9,7 @@ type: integration ## FilterExpression -### *class* FilterExpression(\_filter=None, operator=None, left=None, right=None) +### `*class* FilterExpression(_filter=None, operator=None, left=None, right=None)` A FilterExpression is a logical combination of filters in RedisVL. @@ -44,7 +44,7 @@ v = VectorQuery( ) ``` -#### NOTE +#### `NOTE` Filter expressions are typically not called directly. Instead they are built by combining filter statements using the & and | operators. @@ -56,14 +56,14 @@ built by combining filter statements using the & and | operators. ## Tag -### *class* Tag(field) +### `*class* Tag(field)` A Tag filter can be applied to Tag fields * **Parameters:** **field** (*str*) -#### \_\_eq_\_(other) +#### `__eq__(other)` Create a Tag equality filter expression. @@ -78,7 +78,7 @@ from redisvl.query.filter import Tag f = Tag("brand") == "nike" ``` -#### \_\_ne_\_(other) +#### `__ne__(other)` Create a Tag inequality filter expression. @@ -92,7 +92,7 @@ from redisvl.query.filter import Tag f = Tag("brand") != "nike" ``` -#### \_\_str_\_() +#### `__str__()` Return the Redis Query string for the Tag filter @@ -101,14 +101,14 @@ Return the Redis Query string for the Tag filter ## Text -### *class* Text(field) +### `*class* Text(field)` A Text is a FilterField representing a text field in a Redis index. * **Parameters:** **field** (*str*) -#### \_\_eq_\_(other) +#### `__eq__(other)` Create a Text equality filter expression. These expressions yield filters that enforce an exact match on the supplied term(s). @@ -124,7 +124,7 @@ from redisvl.query.filter import Text f = Text("job") == "engineer" ``` -#### \_\_mod_\_(other) +#### `__mod__(other)` Create a Text “LIKE” filter expression. A flexible expression that yields filters that can use a variety of additional operators like @@ -145,7 +145,7 @@ f = Text("job") % "engineer|doctor" # contains either term in field f = Text("job") % "engineer doctor" # contains both terms in field ``` -#### \_\_ne_\_(other) +#### `__ne__(other)` Create a Text inequality filter expression. These expressions yield negated filters on exact matches on the supplied term(s). Opposite of an @@ -162,7 +162,7 @@ from redisvl.query.filter import Text f = Text("job") != "engineer" ``` -#### \_\_str_\_() +#### `__str__()` Return the Redis Query string for the Text filter @@ -171,14 +171,14 @@ Return the Redis Query string for the Text filter ## Num -### *class* Num(field) +### `*class* Num(field)` A Num is a FilterField representing a numeric field in a Redis index. * **Parameters:** **field** (*str*) -#### \_\_eq_\_(other) +#### `__eq__(other)` Create a Numeric equality filter expression. @@ -192,7 +192,7 @@ from redisvl.query.filter import Num f = Num("zipcode") == 90210 ``` -#### \_\_ge_\_(other) +#### `__ge__(other)` Create a Numeric greater than or equal to filter expression. @@ -207,7 +207,7 @@ from redisvl.query.filter import Num f = Num("age") >= 18 ``` -#### \_\_gt_\_(other) +#### `__gt__(other)` Create a Numeric greater than filter expression. @@ -222,7 +222,7 @@ from redisvl.query.filter import Num f = Num("age") > 18 ``` -#### \_\_le_\_(other) +#### `__le__(other)` Create a Numeric less than or equal to filter expression. @@ -237,7 +237,7 @@ from redisvl.query.filter import Num f = Num("age") <= 18 ``` -#### \_\_lt_\_(other) +#### `__lt__(other)` Create a Numeric less than filter expression. @@ -252,7 +252,7 @@ from redisvl.query.filter import Num f = Num("age") < 18 ``` -#### \_\_ne_\_(other) +#### `__ne__(other)` Create a Numeric inequality filter expression. @@ -267,7 +267,7 @@ from redisvl.query.filter import Num f = Num("zipcode") != 90210 ``` -#### \_\_str_\_() +#### `__str__()` Return the Redis Query string for the Numeric filter @@ -276,7 +276,7 @@ Return the Redis Query string for the Numeric filter ## Geo -### *class* Geo(field) +### `*class* Geo(field)` A Geo is a FilterField representing a geographic (lat/lon) field in a Redis index. @@ -284,7 +284,7 @@ Redis index. * **Parameters:** **field** (*str*) -#### \_\_eq_\_(other) +#### `__eq__(other)` Create a geographic filter within a specified GeoRadius. @@ -299,7 +299,7 @@ from redisvl.query.filter import Geo, GeoRadius f = Geo("location") == GeoRadius(-122.4194, 37.7749, 1, unit="m") ``` -#### \_\_ne_\_(other) +#### `__ne__(other)` Create a geographic filter outside of a specified GeoRadius. @@ -314,7 +314,7 @@ from redisvl.query.filter import Geo, GeoRadius f = Geo("location") != GeoRadius(-122.4194, 37.7749, 1, unit="m") ``` -#### \_\_str_\_() +#### `__str__()` Return the Redis Query string for the Geo filter @@ -323,7 +323,7 @@ Return the Redis Query string for the Geo filter ## GeoRadius -### *class* GeoRadius(longitude, latitude, radius=1, unit='km') +### `*class* GeoRadius(longitude, latitude, radius=1, unit='km')` A GeoRadius is a GeoSpec representing a geographic radius. @@ -337,7 +337,7 @@ Create a GeoRadius specification (GeoSpec) * **Raises:** **ValueError** – If the unit is not one of “m”, “km”, “mi”, or “ft”. -#### \_\_init_\_(longitude, latitude, radius=1, unit='km') +#### `__init__(longitude, latitude, radius=1, unit='km')` Create a GeoRadius specification (GeoSpec) diff --git a/content/integrate/redisvl/api/query.md b/content/integrate/redisvl/api/query.md index 1a2b86af1c..6b7123bd8e 100644 --- a/content/integrate/redisvl/api/query.md +++ b/content/integrate/redisvl/api/query.md @@ -11,7 +11,7 @@ queries for different use cases. Each query class wraps the `redis-py` Query mod ## VectorQuery -### *class* VectorQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False) +### `*class* VectorQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False)` Bases: `BaseVectorQuery`, `BaseQuery` @@ -42,10 +42,10 @@ expression. * **Raises:** **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression -#### NOTE +#### `NOTE` Learn more about vector queries in Redis: [https://redis.io/docs/interact/search-and-query/search/vectors/#knn-search](https://redis.io/docs/interact/search-and-query/search/vectors/#knn-search) -#### dialect(dialect) +#### `dialect(dialect)` Add a dialect field to the query. @@ -56,7 +56,7 @@ Add a dialect field to the query. * **Return type:** *Query* -#### expander(expander) +#### `expander(expander)` Add a expander field to the query. @@ -67,7 +67,7 @@ Add a expander field to the query. * **Return type:** *Query* -#### in_order() +#### `in_order()` Match only documents where the query terms appear in the same order in the document. @@ -76,7 +76,7 @@ i.e. for the query “hello world”, we do not match “world hello” * **Return type:** *Query* -#### language(language) +#### `language(language)` Analyze the query as being in the specified language. @@ -85,7 +85,7 @@ Analyze the query as being in the specified language. * **Return type:** *Query* -#### limit_fields(\*fields) +#### `limit_fields(*fields)` Limit the search to specific TEXT fields only. @@ -98,7 +98,7 @@ from the defined schema. * **Return type:** *Query* -#### limit_ids(\*ids) +#### `limit_ids(*ids)` Limit the results to a specific set of pre-known document ids of any length. @@ -106,14 +106,14 @@ ids of any length. * **Return type:** *Query* -#### no_content() +#### `no_content()` Set the query to only return ids and not the document content. * **Return type:** *Query* -#### no_stopwords() +#### `no_stopwords()` Prevent the query from being filtered for stopwords. Only useful in very big queries that you are certain contain @@ -122,7 +122,7 @@ no stopwords. * **Return type:** *Query* -#### paging(offset, num) +#### `paging(offset, num)` Set the paging for the query (defaults to 0..10). @@ -135,21 +135,21 @@ Set the paging for the query (defaults to 0..10). * **Return type:** *Query* -#### query_string() +#### `query_string()` Return the query string of this query only. * **Return type:** str -#### return_fields(\*fields) +#### `return_fields(*fields)` Add fields to return fields. * **Return type:** *Query* -#### scorer(scorer) +#### `scorer(scorer)` Use a different scoring function to evaluate document relevance. Default is TFIDF. @@ -160,7 +160,7 @@ Default is TFIDF. * **Return type:** *Query* -#### set_filter(filter_expression=None) +#### `set_filter(filter_expression=None)` Set the filter expression for the query. @@ -170,7 +170,7 @@ Set the filter expression for the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. -#### slop(slop) +#### `slop(slop)` Allow a maximum of N intervening non matched terms between phrase terms (0 means exact phrase). @@ -180,7 +180,7 @@ phrase terms (0 means exact phrase). * **Return type:** *Query* -#### sort_by(field, asc=True) +#### `sort_by(field, asc=True)` Add a sortby field to the query. @@ -193,7 +193,7 @@ Add a sortby field to the query. * **Return type:** *Query* -#### timeout(timeout) +#### `timeout(timeout)` overrides the timeout parameter of the module @@ -202,7 +202,7 @@ overrides the timeout parameter of the module * **Return type:** *Query* -#### verbatim() +#### `verbatim()` Set the query to be verbatim, i.e. use no query expansion or stemming. @@ -210,25 +210,25 @@ or stemming. * **Return type:** *Query* -#### with_payloads() +#### `with_payloads()` Ask the engine to return document payloads. * **Return type:** *Query* -#### with_scores() +#### `with_scores()` Ask the engine to return document search scores. * **Return type:** *Query* -#### *property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})* +#### `*property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})*` The filter expression for the query. -#### *property* params *: Dict[str, Any]* +#### `*property* params *: Dict[str, Any]*` Return the parameters for the query. @@ -237,13 +237,13 @@ Return the parameters for the query. * **Return type:** Dict[str, Any] -#### *property* query *: BaseQuery* +#### `*property* query *: BaseQuery*` Return self as the query object. ## VectorRangeQuery -### *class* VectorRangeQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', distance_threshold=0.2, num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False) +### `*class* VectorRangeQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', distance_threshold=0.2, num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False)` Bases: `BaseVectorQuery`, `BaseQuery` @@ -277,10 +277,10 @@ distance threshold. * **Raises:** **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression -#### NOTE +#### `NOTE` Learn more about vector range queries: [https://redis.io/docs/interact/search-and-query/search/vectors/#range-query](https://redis.io/docs/interact/search-and-query/search/vectors/#range-query) -#### dialect(dialect) +#### `dialect(dialect)` Add a dialect field to the query. @@ -291,7 +291,7 @@ Add a dialect field to the query. * **Return type:** *Query* -#### expander(expander) +#### `expander(expander)` Add a expander field to the query. @@ -302,7 +302,7 @@ Add a expander field to the query. * **Return type:** *Query* -#### in_order() +#### `in_order()` Match only documents where the query terms appear in the same order in the document. @@ -311,7 +311,7 @@ i.e. for the query “hello world”, we do not match “world hello” * **Return type:** *Query* -#### language(language) +#### `language(language)` Analyze the query as being in the specified language. @@ -320,7 +320,7 @@ Analyze the query as being in the specified language. * **Return type:** *Query* -#### limit_fields(\*fields) +#### `limit_fields(*fields)` Limit the search to specific TEXT fields only. @@ -333,7 +333,7 @@ from the defined schema. * **Return type:** *Query* -#### limit_ids(\*ids) +#### `limit_ids(*ids)` Limit the results to a specific set of pre-known document ids of any length. @@ -341,14 +341,14 @@ ids of any length. * **Return type:** *Query* -#### no_content() +#### `no_content()` Set the query to only return ids and not the document content. * **Return type:** *Query* -#### no_stopwords() +#### `no_stopwords()` Prevent the query from being filtered for stopwords. Only useful in very big queries that you are certain contain @@ -357,7 +357,7 @@ no stopwords. * **Return type:** *Query* -#### paging(offset, num) +#### `paging(offset, num)` Set the paging for the query (defaults to 0..10). @@ -370,21 +370,21 @@ Set the paging for the query (defaults to 0..10). * **Return type:** *Query* -#### query_string() +#### `query_string()` Return the query string of this query only. * **Return type:** str -#### return_fields(\*fields) +#### `return_fields(*fields)` Add fields to return fields. * **Return type:** *Query* -#### scorer(scorer) +#### `scorer(scorer)` Use a different scoring function to evaluate document relevance. Default is TFIDF. @@ -395,14 +395,14 @@ Default is TFIDF. * **Return type:** *Query* -#### set_distance_threshold(distance_threshold) +#### `set_distance_threshold(distance_threshold)` Set the distance threshold for the query. * **Parameters:** **distance_threshold** (*float*) – vector distance -#### set_filter(filter_expression=None) +#### `set_filter(filter_expression=None)` Set the filter expression for the query. @@ -412,7 +412,7 @@ Set the filter expression for the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. -#### slop(slop) +#### `slop(slop)` Allow a maximum of N intervening non matched terms between phrase terms (0 means exact phrase). @@ -422,7 +422,7 @@ phrase terms (0 means exact phrase). * **Return type:** *Query* -#### sort_by(field, asc=True) +#### `sort_by(field, asc=True)` Add a sortby field to the query. @@ -435,7 +435,7 @@ Add a sortby field to the query. * **Return type:** *Query* -#### timeout(timeout) +#### `timeout(timeout)` overrides the timeout parameter of the module @@ -444,7 +444,7 @@ overrides the timeout parameter of the module * **Return type:** *Query* -#### verbatim() +#### `verbatim()` Set the query to be verbatim, i.e. use no query expansion or stemming. @@ -452,21 +452,21 @@ or stemming. * **Return type:** *Query* -#### with_payloads() +#### `with_payloads()` Ask the engine to return document payloads. * **Return type:** *Query* -#### with_scores() +#### `with_scores()` Ask the engine to return document search scores. * **Return type:** *Query* -#### *property* distance_threshold *: float* +#### `*property* distance_threshold *: float*` Return the distance threshold for the query. @@ -475,11 +475,11 @@ Return the distance threshold for the query. * **Return type:** float -#### *property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})* +#### `*property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})*` The filter expression for the query. -#### *property* params *: Dict[str, Any]* +#### `*property* params *: Dict[str, Any]*` Return the parameters for the query. @@ -488,13 +488,13 @@ Return the parameters for the query. * **Return type:** Dict[str, Any] -#### *property* query *: BaseQuery* +#### `*property* query *: BaseQuery*` Return self as the query object. ## FilterQuery -### *class* FilterQuery(filter_expression=None, return_fields=None, num_results=10, dialect=2, sort_by=None, in_order=False, params=None) +### `*class* FilterQuery(filter_expression=None, return_fields=None, num_results=10, dialect=2, sort_by=None, in_order=False, params=None)` Bases: `BaseQuery` @@ -512,7 +512,7 @@ A query for running a filtered search with a filter expression. * **Raises:** **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression -#### dialect(dialect) +#### `dialect(dialect)` Add a dialect field to the query. @@ -523,7 +523,7 @@ Add a dialect field to the query. * **Return type:** *Query* -#### expander(expander) +#### `expander(expander)` Add a expander field to the query. @@ -534,7 +534,7 @@ Add a expander field to the query. * **Return type:** *Query* -#### in_order() +#### `in_order()` Match only documents where the query terms appear in the same order in the document. @@ -543,7 +543,7 @@ i.e. for the query “hello world”, we do not match “world hello” * **Return type:** *Query* -#### language(language) +#### `language(language)` Analyze the query as being in the specified language. @@ -552,7 +552,7 @@ Analyze the query as being in the specified language. * **Return type:** *Query* -#### limit_fields(\*fields) +#### `limit_fields(*fields)` Limit the search to specific TEXT fields only. @@ -565,7 +565,7 @@ from the defined schema. * **Return type:** *Query* -#### limit_ids(\*ids) +#### `limit_ids(*ids)` Limit the results to a specific set of pre-known document ids of any length. @@ -573,14 +573,14 @@ ids of any length. * **Return type:** *Query* -#### no_content() +#### `no_content()` Set the query to only return ids and not the document content. * **Return type:** *Query* -#### no_stopwords() +#### `no_stopwords()` Prevent the query from being filtered for stopwords. Only useful in very big queries that you are certain contain @@ -589,7 +589,7 @@ no stopwords. * **Return type:** *Query* -#### paging(offset, num) +#### `paging(offset, num)` Set the paging for the query (defaults to 0..10). @@ -602,21 +602,21 @@ Set the paging for the query (defaults to 0..10). * **Return type:** *Query* -#### query_string() +#### `query_string()` Return the query string of this query only. * **Return type:** str -#### return_fields(\*fields) +#### `return_fields(*fields)` Add fields to return fields. * **Return type:** *Query* -#### scorer(scorer) +#### `scorer(scorer)` Use a different scoring function to evaluate document relevance. Default is TFIDF. @@ -627,7 +627,7 @@ Default is TFIDF. * **Return type:** *Query* -#### set_filter(filter_expression=None) +#### `set_filter(filter_expression=None)` Set the filter expression for the query. @@ -637,7 +637,7 @@ Set the filter expression for the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. -#### slop(slop) +#### `slop(slop)` Allow a maximum of N intervening non matched terms between phrase terms (0 means exact phrase). @@ -647,7 +647,7 @@ phrase terms (0 means exact phrase). * **Return type:** *Query* -#### sort_by(field, asc=True) +#### `sort_by(field, asc=True)` Add a sortby field to the query. @@ -660,7 +660,7 @@ Add a sortby field to the query. * **Return type:** *Query* -#### timeout(timeout) +#### `timeout(timeout)` overrides the timeout parameter of the module @@ -669,7 +669,7 @@ overrides the timeout parameter of the module * **Return type:** *Query* -#### verbatim() +#### `verbatim()` Set the query to be verbatim, i.e. use no query expansion or stemming. @@ -677,35 +677,35 @@ or stemming. * **Return type:** *Query* -#### with_payloads() +#### `with_payloads()` Ask the engine to return document payloads. * **Return type:** *Query* -#### with_scores() +#### `with_scores()` Ask the engine to return document search scores. * **Return type:** *Query* -#### *property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})* +#### `*property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})*` The filter expression for the query. -#### *property* params *: Dict[str, Any]* +#### `*property* params *: Dict[str, Any]*` Return the query parameters. -#### *property* query *: BaseQuery* +#### `*property* query *: BaseQuery*` Return self as the query object. ## CountQuery -### *class* CountQuery(filter_expression=None, dialect=2, params=None) +### `*class* CountQuery(filter_expression=None, dialect=2, params=None)` Bases: `BaseQuery` @@ -728,7 +728,7 @@ query = CountQuery(filter_expression=t) count = index.query(query) ``` -#### dialect(dialect) +#### `dialect(dialect)` Add a dialect field to the query. @@ -739,7 +739,7 @@ Add a dialect field to the query. * **Return type:** *Query* -#### expander(expander) +#### `expander(expander)` Add a expander field to the query. @@ -750,7 +750,7 @@ Add a expander field to the query. * **Return type:** *Query* -#### in_order() +#### `in_order()` Match only documents where the query terms appear in the same order in the document. @@ -759,7 +759,7 @@ i.e. for the query “hello world”, we do not match “world hello” * **Return type:** *Query* -#### language(language) +#### `language(language)` Analyze the query as being in the specified language. @@ -768,7 +768,7 @@ Analyze the query as being in the specified language. * **Return type:** *Query* -#### limit_fields(\*fields) +#### `limit_fields(*fields)` Limit the search to specific TEXT fields only. @@ -781,7 +781,7 @@ from the defined schema. * **Return type:** *Query* -#### limit_ids(\*ids) +#### `limit_ids(*ids)` Limit the results to a specific set of pre-known document ids of any length. @@ -789,14 +789,14 @@ ids of any length. * **Return type:** *Query* -#### no_content() +#### `no_content()` Set the query to only return ids and not the document content. * **Return type:** *Query* -#### no_stopwords() +#### `no_stopwords()` Prevent the query from being filtered for stopwords. Only useful in very big queries that you are certain contain @@ -805,7 +805,7 @@ no stopwords. * **Return type:** *Query* -#### paging(offset, num) +#### `paging(offset, num)` Set the paging for the query (defaults to 0..10). @@ -818,21 +818,21 @@ Set the paging for the query (defaults to 0..10). * **Return type:** *Query* -#### query_string() +#### `query_string()` Return the query string of this query only. * **Return type:** str -#### return_fields(\*fields) +#### `return_fields(*fields)` Add fields to return fields. * **Return type:** *Query* -#### scorer(scorer) +#### `scorer(scorer)` Use a different scoring function to evaluate document relevance. Default is TFIDF. @@ -843,7 +843,7 @@ Default is TFIDF. * **Return type:** *Query* -#### set_filter(filter_expression=None) +#### `set_filter(filter_expression=None)` Set the filter expression for the query. @@ -853,7 +853,7 @@ Set the filter expression for the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. -#### slop(slop) +#### `slop(slop)` Allow a maximum of N intervening non matched terms between phrase terms (0 means exact phrase). @@ -863,7 +863,7 @@ phrase terms (0 means exact phrase). * **Return type:** *Query* -#### sort_by(field, asc=True) +#### `sort_by(field, asc=True)` Add a sortby field to the query. @@ -876,7 +876,7 @@ Add a sortby field to the query. * **Return type:** *Query* -#### timeout(timeout) +#### `timeout(timeout)` overrides the timeout parameter of the module @@ -885,7 +885,7 @@ overrides the timeout parameter of the module * **Return type:** *Query* -#### verbatim() +#### `verbatim()` Set the query to be verbatim, i.e. use no query expansion or stemming. @@ -893,28 +893,28 @@ or stemming. * **Return type:** *Query* -#### with_payloads() +#### `with_payloads()` Ask the engine to return document payloads. * **Return type:** *Query* -#### with_scores() +#### `with_scores()` Ask the engine to return document search scores. * **Return type:** *Query* -#### *property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})* +#### `*property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})*` The filter expression for the query. -#### *property* params *: Dict[str, Any]* +#### `*property* params *: Dict[str, Any]*` Return the query parameters. -#### *property* query *: BaseQuery* +#### `*property* query *: BaseQuery*` Return self as the query object. diff --git a/content/integrate/redisvl/api/reranker.md b/content/integrate/redisvl/api/reranker.md index 235434877a..4d15bda662 100644 --- a/content/integrate/redisvl/api/reranker.md +++ b/content/integrate/redisvl/api/reranker.md @@ -9,7 +9,7 @@ type: integration -### *class* CohereReranker(model='rerank-english-v3.0', rank_by=None, limit=5, return_score=True, api_config=None) +### `*class* CohereReranker(model='rerank-english-v3.0', rank_by=None, limit=5, return_score=True, api_config=None)` Bases: `BaseReranker` @@ -59,7 +59,7 @@ and API configuration. * **ImportError** – If the cohere library is not installed. * **ValueError** – If the API key is not provided. -#### *async* arank(query, docs, \*\*kwargs) +#### `*async* arank(query, docs, **kwargs)` Rerank documents based on the provided query using the Cohere rerank API. @@ -76,7 +76,7 @@ query’s context. * **Return type:** Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -88,7 +88,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### rank(query, docs, \*\*kwargs) +#### `rank(query, docs, **kwargs)` Rerank documents based on the provided query using the Cohere rerank API. @@ -105,7 +105,7 @@ query’s context. * **Return type:** Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -113,7 +113,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* HFCrossEncoderReranker(model='cross-encoder/ms-marco-MiniLM-L-6-v2', limit=3, return_score=True, \*, rank_by=None) +### `*class* HFCrossEncoderReranker(model='cross-encoder/ms-marco-MiniLM-L-6-v2', limit=3, return_score=True, *, rank_by=None)` Bases: `BaseReranker` @@ -149,7 +149,7 @@ Initialize the HFCrossEncoderReranker with a specified model and ranking criteri * **return_score** (*bool*) – Whether to return scores alongside the reranked results. * **rank_by** (*List* *[**str* *]* *|* *None*) -#### *async* arank(query, docs, \*\*kwargs) +#### `*async* arank(query, docs, **kwargs)` Asynchronously rerank documents based on the provided query using the loaded cross-encoder model. @@ -165,7 +165,7 @@ in a manner that is potentially more relevant to the query’s context. * **Return type:** Union[Tuple[List[Dict[str, Any]], List[float]], List[Dict[str, Any]]] -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -177,7 +177,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### rank(query, docs, \*\*kwargs) +#### `rank(query, docs, **kwargs)` Rerank documents based on the provided query using the loaded cross-encoder model. @@ -193,7 +193,7 @@ in a manner that is potentially more relevant to the query’s context. * **Return type:** Union[Tuple[List[Dict[str, Any]], List[float]], List[Dict[str, Any]]] -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -201,7 +201,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* VoyageAIReranker(model, rank_by=None, limit=5, return_score=True, api_config=None) +### `*class* VoyageAIReranker(model, rank_by=None, limit=5, return_score=True, api_config=None)` Bases: `BaseReranker` @@ -250,7 +250,7 @@ and API configuration. * **ImportError** – If the voyageai library is not installed. * **ValueError** – If the API key is not provided. -#### *async* arank(query, docs, \*\*kwargs) +#### `*async* arank(query, docs, **kwargs)` Rerank documents based on the provided query using the VoyageAI rerank API. @@ -267,7 +267,7 @@ query’s context. * **Return type:** Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -279,7 +279,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### rank(query, docs, \*\*kwargs) +#### `rank(query, docs, **kwargs)` Rerank documents based on the provided query using the VoyageAI rerank API. @@ -296,6 +296,6 @@ query’s context. * **Return type:** Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. diff --git a/content/integrate/redisvl/api/router.md b/content/integrate/redisvl/api/router.md index c329c8d1f0..b94617b820 100644 --- a/content/integrate/redisvl/api/router.md +++ b/content/integrate/redisvl/api/router.md @@ -9,7 +9,7 @@ type: integration ## Semantic Router -### *class* SemanticRouter(name, routes, vectorizer=None, routing_config=None, redis_client=None, redis_url='redis://localhost:6379', overwrite=False, connection_kwargs={}) +### `*class* SemanticRouter(name, routes, vectorizer=None, routing_config=None, redis_client=None, redis_url='redis://localhost:6379', overwrite=False, connection_kwargs={})` Semantic Router for managing and querying route vectors. @@ -26,21 +26,21 @@ Initialize the SemanticRouter. * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments for the redis client. Defaults to empty {}. -#### clear() +#### `clear()` Flush all routes from the semantic router index. * **Return type:** None -#### delete() +#### `delete()` Delete the semantic router index. * **Return type:** None -#### *classmethod* from_dict(data, \*\*kwargs) +#### `*classmethod* from_dict(data, **kwargs)` Create a SemanticRouter from a dictionary. @@ -63,7 +63,7 @@ router_data = { router = SemanticRouter.from_dict(router_data) ``` -#### *classmethod* from_yaml(file_path, \*\*kwargs) +#### `*classmethod* from_yaml(file_path, **kwargs)` Create a SemanticRouter from a YAML file. @@ -82,7 +82,7 @@ from redisvl.extensions.router import SemanticRouter router = SemanticRouter.from_yaml("router.yaml", redis_url="redis://localhost:6379") ``` -#### get(route_name) +#### `get(route_name)` Get a route by its name. @@ -93,7 +93,7 @@ Get a route by its name. * **Return type:** Optional[[Route](#route)] -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -105,7 +105,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### remove_route(route_name) +#### `remove_route(route_name)` Remove a route and all references from the semantic router. @@ -114,7 +114,7 @@ Remove a route and all references from the semantic router. * **Return type:** None -#### route_many(statement=None, vector=None, max_k=None, distance_threshold=None, aggregation_method=None) +#### `route_many(statement=None, vector=None, max_k=None, distance_threshold=None, aggregation_method=None)` Query the semantic router with a given statement or vector for multiple matches. @@ -129,7 +129,7 @@ Query the semantic router with a given statement or vector for multiple matches. * **Return type:** List[[RouteMatch](#routematch)] -#### to_dict() +#### `to_dict()` Convert the SemanticRouter instance to a dictionary. @@ -144,7 +144,7 @@ router = SemanticRouter(name="example_router", routes=[], redis_url="redis://loc router_dict = router.to_dict() ``` -#### to_yaml(file_path, overwrite=True) +#### `to_yaml(file_path, overwrite=True)` Write the semantic router to a YAML file. @@ -166,22 +166,22 @@ router = SemanticRouter( router.to_yaml("router.yaml") ``` -#### update_routing_config(routing_config) +#### `update_routing_config(routing_config)` Update the routing configuration. * **Parameters:** **routing_config** ([*RoutingConfig*](#routingconfig)) – The new routing configuration. -#### model_config *: ClassVar[ConfigDict]* *= {'arbitrary_types_allowed': True}* +#### `model_config *: ClassVar[ConfigDict]* *= {'arbitrary_types_allowed': True}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. -#### name *: str* +#### `name *: str*` The name of the semantic router. -#### *property* route_names *: List[str]* +#### `*property* route_names *: List[str]*` Get the list of route names. @@ -190,7 +190,7 @@ Get the list of route names. * **Return type:** List[str] -#### *property* route_thresholds *: Dict[str, float | None]* +#### `*property* route_thresholds *: Dict[str, float | None]*` Get the distance thresholds for each route. @@ -199,21 +199,21 @@ Get the distance thresholds for each route. * **Return type:** Dict[str, float] -#### routes *: List[[Route](#route)]* +#### `routes *: List[[Route](#route)]*` List of Route objects. -#### routing_config *: [RoutingConfig](#routingconfig)* +#### `routing_config *: [RoutingConfig](#routingconfig)*` Configuration for routing behavior. -#### vectorizer *: BaseVectorizer* +#### `vectorizer *: BaseVectorizer*` The vectorizer used to embed route references. ## Routing Config -### *class* RoutingConfig(\*, max_k=1, aggregation_method=DistanceAggregationMethod.avg) +### `*class* RoutingConfig(*, max_k=1, aggregation_method=DistanceAggregationMethod.avg)` Configuration for routing behavior. @@ -228,17 +228,17 @@ self is explicitly positional-only to allow self as a field name. * **max_k** (*Annotated* *[**int* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=False* *,* *default=1* *,* *metadata=* *[**Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *]* *)* *]*) * **aggregation_method** ([*DistanceAggregationMethod*](#distanceaggregationmethod)) -#### max_k *: Annotated[int, FieldInfo(annotation=NoneType, required=False, default=1, metadata=[Strict(strict=True), Gt(gt=0)])]* +#### `max_k *: Annotated[int, FieldInfo(annotation=NoneType, required=False, default=1, metadata=[Strict(strict=True), Gt(gt=0)])]*` Aggregation method to use to classify queries. -#### model_config *: ClassVar[ConfigDict]* *= {'extra': 'ignore'}* +#### `model_config *: ClassVar[ConfigDict]* *= {'extra': 'ignore'}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. ## Route -### *class* Route(\*, name, references, metadata={}, distance_threshold=0.5) +### `*class* Route(*, name, references, metadata={}, distance_threshold=0.5)` Model representing a routing path with associated metadata and thresholds. @@ -255,29 +255,29 @@ self is explicitly positional-only to allow self as a field name. * **metadata** (*Dict* *[**str* *,* *Any* *]*) * **distance_threshold** (*Annotated* *[**float* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=True* *,* *metadata=* *[**Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *,* *Le* *(**le=1* *)* *]* *)* *]*) -#### distance_threshold *: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Strict(strict=True), Gt(gt=0), Le(le=1)])]* +#### `distance_threshold *: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Strict(strict=True), Gt(gt=0), Le(le=1)])]*` Distance threshold for matching the route. -#### metadata *: Dict[str, Any]* +#### `metadata *: Dict[str, Any]*` Metadata associated with the route. -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. -#### name *: str* +#### `name *: str*` The name of the route. -#### references *: List[str]* +#### `references *: List[str]*` List of reference phrases for the route. ## Route Match -### *class* RouteMatch(\*, name=None, distance=None) +### `*class* RouteMatch(*, name=None, distance=None)` Model representing a matched route with distance information. @@ -292,32 +292,32 @@ self is explicitly positional-only to allow self as a field name. * **name** (*str* *|* *None*) * **distance** (*float* *|* *None*) -#### distance *: float | None* +#### `distance *: float | None*` The vector distance between the statement and the matched route. -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. -#### name *: str | None* +#### `name *: str | None*` The matched route name. ## Distance Aggregation Method -### *class* DistanceAggregationMethod(value, names=, \*values, module=None, qualname=None, type=None, start=1, boundary=None) +### `*class* DistanceAggregationMethod(value, names=, *values, module=None, qualname=None, type=None, start=1, boundary=None)` Enumeration for distance aggregation methods. -#### avg *= 'avg'* +#### `avg *= 'avg'*` Compute the average of the vector distances. -#### min *= 'min'* +#### `min *= 'min'*` Compute the minimum of the vector distances. -#### sum *= 'sum'* +#### `sum *= 'sum'*` Compute the sum of the vector distances. diff --git a/content/integrate/redisvl/api/schema.md b/content/integrate/redisvl/api/schema.md index 1191d5f7ac..eb3bfef6d3 100644 --- a/content/integrate/redisvl/api/schema.md +++ b/content/integrate/redisvl/api/schema.md @@ -18,7 +18,7 @@ field configurations using the following three components: -### *class* IndexSchema(\*, index, fields={}, version='0.1.0') +### `*class* IndexSchema(*, index, fields={}, version='0.1.0')` A schema definition for a search index in Redis, used in RedisVL for configuring index settings and organizing vector and metadata fields. @@ -89,7 +89,7 @@ schema = IndexSchema.from_dict({ }) ``` -#### NOTE +#### `NOTE` The fields attribute in the schema must contain unique field names to ensure correct and unambiguous field references. @@ -105,7 +105,7 @@ self is explicitly positional-only to allow self as a field name. * **fields** (*Dict* *[**str* *,* *BaseField* *]*) * **version** (*Literal* *[* *'0.1.0'* *]*) -#### add_field(field_inputs) +#### `add_field(field_inputs)` Adds a single field to the index schema based on the specified field type and attributes. @@ -135,7 +135,7 @@ schema.add_field({ }) ``` -#### add_fields(fields) +#### `add_fields(fields)` Extends the schema with additional fields. @@ -164,7 +164,7 @@ schema.add_fields([ ]) ``` -#### *classmethod* from_dict(data) +#### `*classmethod* from_dict(data)` Create an IndexSchema from a dictionary. @@ -201,7 +201,7 @@ schema = IndexSchema.from_dict({ }) ``` -#### *classmethod* from_yaml(file_path) +#### `*classmethod* from_yaml(file_path)` Create an IndexSchema from a YAML file. @@ -217,7 +217,7 @@ from redisvl.schema import IndexSchema schema = IndexSchema.from_yaml("schema.yaml") ``` -#### remove_field(field_name) +#### `remove_field(field_name)` Removes a field from the schema based on the specified name. @@ -227,7 +227,7 @@ existing fields. * **Parameters:** **field_name** (*str*) – The name of the field to be removed. -#### to_dict() +#### `to_dict()` Serialize the index schema model to a dictionary, handling Enums and other special cases properly. @@ -237,7 +237,7 @@ and other special cases properly. * **Return type:** Dict[str, Any] -#### to_yaml(file_path, overwrite=True) +#### `to_yaml(file_path, overwrite=True)` Write the index schema to a YAML file. @@ -249,7 +249,7 @@ Write the index schema to a YAML file. * **Return type:** None -#### *property* field_names *: List[str]* +#### `*property* field_names *: List[str]*` A list of field names associated with the index schema. @@ -258,19 +258,19 @@ A list of field names associated with the index schema. * **Return type:** List[str] -#### fields *: Dict[str, BaseField]* +#### `fields *: Dict[str, BaseField]*` Fields associated with the search index and their properties -#### index *: IndexInfo* +#### `index *: IndexInfo*` Details of the basic index configurations. -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. -#### version *: Literal['0.1.0']* +#### `version *: Literal['0.1.0']*` Version of the underlying index schema. diff --git a/content/integrate/redisvl/api/searchindex.md b/content/integrate/redisvl/api/searchindex.md index 975159cfee..748f485852 100644 --- a/content/integrate/redisvl/api/searchindex.md +++ b/content/integrate/redisvl/api/searchindex.md @@ -14,7 +14,7 @@ type: integration ## SearchIndex -### *class* SearchIndex(schema, redis_client=None, redis_url=None, connection_args={}, \*\*kwargs) +### `*class* SearchIndex(schema, redis_client=None, redis_url=None, connection_args={}, **kwargs)` A search index class for interacting with Redis as a vector database. @@ -52,7 +52,7 @@ kwargs. * **connection_args** (*Dict* *[**str* *,* *Any* *]* *,* *optional*) – Redis client connection args. -#### aggregate(\*args, \*\*kwargs) +#### `aggregate(*args, **kwargs)` Perform an aggregation operation against the index. @@ -65,7 +65,7 @@ to the redis-py ft().aggregate() method. * **Return type:** Result -#### clear() +#### `clear()` Clear all keys in Redis associated with the index, leaving the index available and in-place for future insertions or updates. @@ -75,7 +75,7 @@ available and in-place for future insertions or updates. * **Return type:** int -#### connect(redis_url=None, \*\*kwargs) +#### `connect(redis_url=None, **kwargs)` Connect to a Redis instance using the provided redis_url, falling back to the REDIS_URL environment variable (if available). @@ -97,7 +97,7 @@ extra options specific to the Redis connection. index.connect(redis_url="redis://localhost:6379") ``` -#### create(overwrite=False, drop=False) +#### `create(overwrite=False, drop=False)` Create an index in Redis with the current schema and properties. @@ -123,7 +123,7 @@ index.create(overwrite=True) index.create(overwrite=True, drop=True) ``` -#### delete(drop=True) +#### `delete(drop=True)` Delete the search index while optionally dropping all keys associated with the index. @@ -134,11 +134,11 @@ with the index. * **Raises:** **redis.exceptions.ResponseError** – If the index does not exist. -#### disconnect() +#### `disconnect()` Disconnect from the Redis database. -#### drop_keys(keys) +#### `drop_keys(keys)` Remove a specific entry or entries from the index by it’s key ID. @@ -149,7 +149,7 @@ Remove a specific entry or entries from the index by it’s key ID. * **Return type:** int -#### exists() +#### `exists()` Check if the index exists in Redis. @@ -158,7 +158,7 @@ Check if the index exists in Redis. * **Return type:** bool -#### fetch(id) +#### `fetch(id)` Fetch an object from Redis by id. @@ -174,7 +174,7 @@ or derived from some domain-specific metadata combination * **Return type:** Dict[str, Any] -#### *classmethod* from_dict(schema_dict, \*\*kwargs) +#### `*classmethod* from_dict(schema_dict, **kwargs)` Create a SearchIndex from a dictionary. @@ -200,7 +200,7 @@ index = SearchIndex.from_dict({ }) ``` -#### *classmethod* from_existing(name, redis_client=None, redis_url=None, \*\*kwargs) +#### `*classmethod* from_existing(name, redis_client=None, redis_url=None, **kwargs)` Initialize from an existing search index in Redis by index name. @@ -211,7 +211,7 @@ Initialize from an existing search index in Redis by index name. * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to connect to. -#### *classmethod* from_yaml(schema_path, \*\*kwargs) +#### `*classmethod* from_yaml(schema_path, **kwargs)` Create a SearchIndex from a YAML schema file. @@ -228,7 +228,7 @@ from redisvl.index import SearchIndex index = SearchIndex.from_yaml("schemas/schema.yaml") ``` -#### info(name=None) +#### `info(name=None)` Get information about the index. @@ -240,7 +240,7 @@ Get information about the index. * **Return type:** dict -#### key(id) +#### `key(id)` Construct a redis key as a combination of an index key prefix (optional) and specified id. @@ -257,7 +257,7 @@ id or chunk id). * **Return type:** str -#### listall() +#### `listall()` List all search indices in Redis database. @@ -266,7 +266,7 @@ List all search indices in Redis database. * **Return type:** List[str] -#### load(data, id_field=None, keys=None, ttl=None, preprocess=None, batch_size=None) +#### `load(data, id_field=None, keys=None, ttl=None, preprocess=None, batch_size=None)` Load objects to the Redis database. Returns the list of keys loaded to Redis. @@ -316,7 +316,7 @@ def add_field(d): keys = index.load(data, preprocess=add_field) ``` -#### paginate(query, page_size=30) +#### `paginate(query, page_size=30)` Execute a given query against the index and return results in paginated batches. @@ -344,12 +344,12 @@ for result_batch in index.paginate(query, page_size=10): pass ``` -#### NOTE +#### `NOTE` The page_size parameter controls the number of items each result batch contains. Adjust this value based on performance considerations and the expected volume of search results. -#### query(query) +#### `query(query)` Execute a query on the index. @@ -375,7 +375,7 @@ query = VectorQuery( results = index.query(query) ``` -#### search(\*args, \*\*kwargs) +#### `search(*args, **kwargs)` Perform a search against the index. @@ -388,7 +388,7 @@ to the redis-py ft().search() method. * **Return type:** Result -#### set_client(redis_client, \*\*kwargs) +#### `set_client(redis_client, **kwargs)` Manually set the Redis client to use with the search index. @@ -411,25 +411,25 @@ index = SearchIndex.from_yaml("schemas/schema.yaml") index.set_client(client) ``` -#### *property* client *: Redis | None* +#### `*property* client *: Redis | None*` The underlying redis-py client object. -#### *property* key_separator *: str* +#### `*property* key_separator *: str*` The optional separator between a defined prefix and key value in forming a Redis key. -#### *property* name *: str* +#### `*property* name *: str*` The name of the Redis search index. -#### *property* prefix *: str* +#### `*property* prefix *: str*` The optional key prefix that comes before a unique key value in forming a Redis key. -#### *property* storage_type *: StorageType* +#### `*property* storage_type *: StorageType*` The underlying storage type for the search index; either hash or json. @@ -438,7 +438,7 @@ hash or json. ## AsyncSearchIndex -### *class* AsyncSearchIndex(schema, \*\*kwargs) +### `*class* AsyncSearchIndex(schema, **kwargs)` A search index class for interacting with Redis as a vector database in async-mode. @@ -471,7 +471,7 @@ Initialize the RedisVL async search index with a schema. * **connection_args** (*Dict* *[**str* *,* *Any* *]* *,* *optional*) – Redis client connection args. -#### *async* aggregate(\*args, \*\*kwargs) +#### `*async* aggregate(*args, **kwargs)` Perform an aggregation operation against the index. @@ -484,7 +484,7 @@ to the redis-py ft().aggregate() method. * **Return type:** Result -#### *async* clear() +#### `*async* clear()` Clear all keys in Redis associated with the index, leaving the index available and in-place for future insertions or updates. @@ -494,7 +494,7 @@ available and in-place for future insertions or updates. * **Return type:** int -#### *async* connect(redis_url=None, \*\*kwargs) +#### `*async* connect(redis_url=None, **kwargs)` Connect to a Redis instance using the provided redis_url, falling back to the REDIS_URL environment variable (if available). @@ -516,7 +516,7 @@ extra options specific to the Redis connection. index.connect(redis_url="redis://localhost:6379") ``` -#### *async* create(overwrite=False, drop=False) +#### `*async* create(overwrite=False, drop=False)` Asynchronously create an index in Redis with the current schema : and properties. @@ -543,7 +543,7 @@ await index.create(overwrite=True) await index.create(overwrite=True, drop=True) ``` -#### *async* delete(drop=True) +#### `*async* delete(drop=True)` Delete the search index. @@ -553,11 +553,11 @@ Delete the search index. * **Raises:** **redis.exceptions.ResponseError** – If the index does not exist. -#### disconnect() +#### `disconnect()` Disconnect and cleanup the underlying async redis connection. -#### *async* drop_keys(keys) +#### `*async* drop_keys(keys)` Remove a specific entry or entries from the index by it’s key ID. @@ -568,7 +568,7 @@ Remove a specific entry or entries from the index by it’s key ID. * **Return type:** int -#### *async* exists() +#### `*async* exists()` Check if the index exists in Redis. @@ -577,7 +577,7 @@ Check if the index exists in Redis. * **Return type:** bool -#### *async* fetch(id) +#### `*async* fetch(id)` Asynchronously etch an object from Redis by id. The id is typically either a unique identifier, or derived from some domain-specific @@ -591,7 +591,7 @@ metadata combination (like a document id or chunk id). * **Return type:** Dict[str, Any] -#### *classmethod* from_dict(schema_dict, \*\*kwargs) +#### `*classmethod* from_dict(schema_dict, **kwargs)` Create a SearchIndex from a dictionary. @@ -617,7 +617,7 @@ index = SearchIndex.from_dict({ }) ``` -#### *async classmethod* from_existing(name, redis_client=None, redis_url=None, \*\*kwargs) +#### `*async classmethod* from_existing(name, redis_client=None, redis_url=None, **kwargs)` Initialize from an existing search index in Redis by index name. @@ -628,7 +628,7 @@ Initialize from an existing search index in Redis by index name. * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to connect to. -#### *classmethod* from_yaml(schema_path, \*\*kwargs) +#### `*classmethod* from_yaml(schema_path, **kwargs)` Create a SearchIndex from a YAML schema file. @@ -645,7 +645,7 @@ from redisvl.index import SearchIndex index = SearchIndex.from_yaml("schemas/schema.yaml") ``` -#### *async* info(name=None) +#### `*async* info(name=None)` Get information about the index. @@ -657,7 +657,7 @@ Get information about the index. * **Return type:** dict -#### key(id) +#### `key(id)` Construct a redis key as a combination of an index key prefix (optional) and specified id. @@ -674,7 +674,7 @@ id or chunk id). * **Return type:** str -#### *async* listall() +#### `*async* listall()` List all search indices in Redis database. @@ -683,7 +683,7 @@ List all search indices in Redis database. * **Return type:** List[str] -#### *async* load(data, id_field=None, keys=None, ttl=None, preprocess=None, concurrency=None) +#### `*async* load(data, id_field=None, keys=None, ttl=None, preprocess=None, concurrency=None)` Asynchronously load objects to Redis with concurrency control. Returns the list of keys loaded to Redis. @@ -733,7 +733,7 @@ async def add_field(d): keys = await index.load(data, preprocess=add_field) ``` -#### *async* paginate(query, page_size=30) +#### `*async* paginate(query, page_size=30)` Execute a given query against the index and return results in paginated batches. @@ -761,12 +761,12 @@ async for result_batch in index.paginate(query, page_size=10): pass ``` -#### NOTE +#### `NOTE` The page_size parameter controls the number of items each result batch contains. Adjust this value based on performance considerations and the expected volume of search results. -#### *async* query(query) +#### `*async* query(query)` Asynchronously execute a query on the index. @@ -792,7 +792,7 @@ query = VectorQuery( results = await index.query(query) ``` -#### *async* search(\*args, \*\*kwargs) +#### `*async* search(*args, **kwargs)` Perform a search on this index. @@ -805,7 +805,7 @@ to the redis-py ft.search() method. * **Return type:** Result -#### *async* set_client(redis_client) +#### `*async* set_client(redis_client)` Manually set the Redis client to use with the search index. @@ -829,25 +829,25 @@ index = AsyncSearchIndex.from_yaml("schemas/schema.yaml") await index.set_client(client) ``` -#### *property* client *: Redis | None* +#### `*property* client *: Redis | None*` The underlying redis-py client object. -#### *property* key_separator *: str* +#### `*property* key_separator *: str*` The optional separator between a defined prefix and key value in forming a Redis key. -#### *property* name *: str* +#### `*property* name *: str*` The name of the Redis search index. -#### *property* prefix *: str* +#### `*property* prefix *: str*` The optional key prefix that comes before a unique key value in forming a Redis key. -#### *property* storage_type *: StorageType* +#### `*property* storage_type *: StorageType*` The underlying storage type for the search index; either hash or json. diff --git a/content/integrate/redisvl/api/session_manager.md b/content/integrate/redisvl/api/session_manager.md index 7b668e8701..37cd7051f5 100644 --- a/content/integrate/redisvl/api/session_manager.md +++ b/content/integrate/redisvl/api/session_manager.md @@ -9,7 +9,7 @@ type: integration -### *class* SemanticSessionManager(name, session_tag=None, prefix=None, vectorizer=None, distance_threshold=0.3, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, \*\*kwargs) +### `*class* SemanticSessionManager(name, session_tag=None, prefix=None, vectorizer=None, distance_threshold=0.3, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, **kwargs)` Bases: `BaseSessionManager` @@ -40,7 +40,7 @@ responses. The proposed schema will support a single vector embedding constructed from either the prompt or response in a single string. -#### add_message(message, session_tag=None) +#### `add_message(message, session_tag=None)` Insert a single prompt or response into the session memory. A timestamp is associated with it so that it can be later sorted @@ -53,7 +53,7 @@ in sequential ordering after retrieval. * **Return type:** None -#### add_messages(messages, session_tag=None) +#### `add_messages(messages, session_tag=None)` Insert a list of prompts and responses into the session memory. A timestamp is associated with each so that they can be later sorted @@ -66,21 +66,21 @@ in sequential ordering after retrieval. * **Return type:** None -#### clear() +#### `clear()` Clears the chat session history. * **Return type:** None -#### delete() +#### `delete()` Clear all conversation keys and remove the search index. * **Return type:** None -#### drop(id=None) +#### `drop(id=None)` Remove a specific exchange from the conversation history. @@ -90,7 +90,7 @@ Remove a specific exchange from the conversation history. * **Return type:** None -#### get_recent(top_k=5, as_text=False, raw=False, session_tag=None) +#### `get_recent(top_k=5, as_text=False, raw=False, session_tag=None)` Retreive the recent conversation history in sequential order. @@ -110,7 +110,7 @@ Retreive the recent conversation history in sequential order. * **Raises:** **ValueError** – if top_k is not an integer greater than or equal to 0. -#### get_relevant(prompt, as_text=False, top_k=5, fall_back=False, session_tag=None, raw=False, distance_threshold=None) +#### `get_relevant(prompt, as_text=False, top_k=5, fall_back=False, session_tag=None, raw=False, distance_threshold=None)` Searches the chat history for information semantically related to the specified prompt. @@ -141,7 +141,7 @@ context to the next LLM call. Raises ValueError: if top_k is not an integer greater or equal to 0. -#### store(prompt, response, session_tag=None) +#### `store(prompt, response, session_tag=None)` Insert a prompt:response pair into the session memory. A timestamp is associated with each message so that they can be later sorted @@ -155,7 +155,7 @@ in sequential ordering after retrieval. * **Return type:** None -#### *property* messages *: List[str] | List[Dict[str, str]]* +#### `*property* messages *: List[str] | List[Dict[str, str]]*` Returns the full chat history. @@ -163,7 +163,7 @@ Returns the full chat history. -### *class* StandardSessionManager(name, session_tag=None, prefix=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, \*\*kwargs) +### `*class* StandardSessionManager(name, session_tag=None, prefix=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, **kwargs)` Bases: `BaseSessionManager` @@ -189,7 +189,7 @@ responses. The proposed schema will support a single combined vector embedding constructed from the prompt & response in a single string. -#### add_message(message, session_tag=None) +#### `add_message(message, session_tag=None)` Insert a single prompt or response into the session memory. A timestamp is associated with it so that it can be later sorted @@ -202,7 +202,7 @@ in sequential ordering after retrieval. * **Return type:** None -#### add_messages(messages, session_tag=None) +#### `add_messages(messages, session_tag=None)` Insert a list of prompts and responses into the session memory. A timestamp is associated with each so that they can be later sorted @@ -215,21 +215,21 @@ in sequential ordering after retrieval. * **Return type:** None -#### clear() +#### `clear()` Clears the chat session history. * **Return type:** None -#### delete() +#### `delete()` Clear all conversation keys and remove the search index. * **Return type:** None -#### drop(id=None) +#### `drop(id=None)` Remove a specific exchange from the conversation history. @@ -239,7 +239,7 @@ Remove a specific exchange from the conversation history. * **Return type:** None -#### get_recent(top_k=5, as_text=False, raw=False, session_tag=None) +#### `get_recent(top_k=5, as_text=False, raw=False, session_tag=None)` Retrieve the recent conversation history in sequential order. @@ -259,7 +259,7 @@ Retrieve the recent conversation history in sequential order. * **Raises:** **ValueError** – if top_k is not an integer greater than or equal to 0. -#### store(prompt, response, session_tag=None) +#### `store(prompt, response, session_tag=None)` Insert a prompt:response pair into the session memory. A timestamp is associated with each exchange so that they can be later sorted @@ -273,6 +273,6 @@ in sequential ordering after retrieval. * **Return type:** None -#### *property* messages *: List[str] | List[Dict[str, str]]* +#### `*property* messages *: List[str] | List[Dict[str, str]]*` Returns the full chat history. diff --git a/content/integrate/redisvl/api/vectorizer.md b/content/integrate/redisvl/api/vectorizer.md index cf54d3a2be..4ebb444a67 100644 --- a/content/integrate/redisvl/api/vectorizer.md +++ b/content/integrate/redisvl/api/vectorizer.md @@ -9,7 +9,7 @@ type: integration -### *class* HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2', dtype='float32', \*, dims=None) +### `*class* HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2', dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -50,7 +50,7 @@ Initialize the Hugging Face text vectorizer. * **ValueError** – If there is an error setting the embedding model dimensions. * **ValueError** – If an invalid dtype is provided. -#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `embed(text, preprocess=None, as_buffer=False, **kwargs)` Embed a chunk of text using the Hugging Face sentence transformer. @@ -67,7 +67,7 @@ Embed a chunk of text using the Hugging Face sentence transformer. * **Raises:** **TypeError** – If the wrong input type is passed in for the text. -#### embed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, \*\*kwargs) +#### `embed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, **kwargs)` Asynchronously embed many chunks of texts using the Hugging Face sentence transformer. @@ -87,7 +87,7 @@ sentence transformer. * **Raises:** **TypeError** – If the wrong input type is passed in for the test. -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -99,7 +99,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -107,7 +107,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* OpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', \*, dims=None) +### `*class* OpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -156,7 +156,7 @@ Initialize the OpenAI vectorizer. * **ValueError** – If the OpenAI API key is not provided. * **ValueError** – If an invalid dtype is provided. -#### aembed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `aembed(text, preprocess=None, as_buffer=False, **kwargs)` Asynchronously embed a chunk of text using the OpenAI API. @@ -173,7 +173,7 @@ Asynchronously embed a chunk of text using the OpenAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the text. -#### aembed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, \*\*kwargs) +#### `aembed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, **kwargs)` Asynchronously embed many chunks of texts using the OpenAI API. @@ -192,7 +192,7 @@ Asynchronously embed many chunks of texts using the OpenAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the text. -#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `embed(text, preprocess=None, as_buffer=False, **kwargs)` Embed a chunk of text using the OpenAI API. @@ -209,7 +209,7 @@ Embed a chunk of text using the OpenAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the text. -#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) +#### `embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, **kwargs)` Embed many chunks of texts using the OpenAI API. @@ -228,7 +228,7 @@ Embed many chunks of texts using the OpenAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the text. -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -240,7 +240,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -248,7 +248,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* AzureOpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', \*, dims=None) +### `*class* AzureOpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -304,7 +304,7 @@ Initialize the AzureOpenAI vectorizer. * **ValueError** – If the AzureOpenAI API key, version, or endpoint are not provided. * **ValueError** – If an invalid dtype is provided. -#### aembed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `aembed(text, preprocess=None, as_buffer=False, **kwargs)` Asynchronously embed a chunk of text using the OpenAI API. @@ -321,7 +321,7 @@ Asynchronously embed a chunk of text using the OpenAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the test. -#### aembed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, \*\*kwargs) +#### `aembed_many(texts, preprocess=None, batch_size=1000, as_buffer=False, **kwargs)` Asynchronously embed many chunks of texts using the AzureOpenAI API. @@ -340,7 +340,7 @@ Asynchronously embed many chunks of texts using the AzureOpenAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the test. -#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `embed(text, preprocess=None, as_buffer=False, **kwargs)` Embed a chunk of text using the AzureOpenAI API. @@ -357,7 +357,7 @@ Embed a chunk of text using the AzureOpenAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the test. -#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) +#### `embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, **kwargs)` Embed many chunks of texts using the AzureOpenAI API. @@ -376,7 +376,7 @@ Embed many chunks of texts using the AzureOpenAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the test. -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -388,7 +388,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -396,7 +396,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* VertexAITextVectorizer(model='textembedding-gecko', api_config=None, dtype='float32', \*, dims=None) +### `*class* VertexAITextVectorizer(model='textembedding-gecko', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -446,7 +446,7 @@ Initialize the VertexAI vectorizer. * **ValueError** – If the API key is not provided. * **ValueError** – If an invalid dtype is provided. -#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `embed(text, preprocess=None, as_buffer=False, **kwargs)` Embed a chunk of text using the VertexAI API. @@ -463,7 +463,7 @@ Embed a chunk of text using the VertexAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the test. -#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) +#### `embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, **kwargs)` Embed many chunks of texts using the VertexAI API. @@ -482,7 +482,7 @@ Embed many chunks of texts using the VertexAI API. * **Raises:** **TypeError** – If the wrong input type is passed in for the test. -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -494,7 +494,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -502,7 +502,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* CohereTextVectorizer(model='embed-english-v3.0', api_config=None, dtype='float32', \*, dims=None) +### `*class* CohereTextVectorizer(model='embed-english-v3.0', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -553,7 +553,7 @@ Visit [https://cohere.ai/embed](https://cohere.ai/embed) to learn about embeddin * **ValueError** – If the API key is not provided. * **ValueError** – If an invalid dtype is provided. -#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `embed(text, preprocess=None, as_buffer=False, **kwargs)` Embed a chunk of text using the Cohere Embeddings API. @@ -588,7 +588,7 @@ task downstream, you should set input_type= “classification” or * **Raises:** **TypeError** – In an invalid input_type is provided. -#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) +#### `embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, **kwargs)` Embed many chunks of text using the Cohere Embeddings API. @@ -625,7 +625,7 @@ task downstream, you should set input_type= “classification” or * **Raises:** **TypeError** – In an invalid input_type is provided. -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -637,7 +637,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -645,7 +645,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* BedrockTextVectorizer(model='amazon.titan-embed-text-v2:0', api_config=None, dtype='float32', \*, dims=None) +### `*class* BedrockTextVectorizer(model='amazon.titan-embed-text-v2:0', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -697,7 +697,7 @@ Initialize the AWS Bedrock Vectorizer. * **ImportError** – If boto3 is not installed. * **ValueError** – If an invalid dtype is provided. -#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `embed(text, preprocess=None, as_buffer=False, **kwargs)` Embed a chunk of text using Amazon Bedrock. @@ -712,7 +712,7 @@ Embed a chunk of text using Amazon Bedrock. * **Raises:** **TypeError** – If text is not a string. -#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) +#### `embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, **kwargs)` Embed multiple texts using Amazon Bedrock. @@ -728,7 +728,7 @@ Embed multiple texts using Amazon Bedrock. * **Raises:** **TypeError** – If texts is not a list of strings. -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -740,7 +740,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -748,7 +748,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* CustomTextVectorizer(embed, embed_many=None, aembed=None, aembed_many=None, dtype='float32') +### `*class* CustomTextVectorizer(embed, embed_many=None, aembed=None, aembed_many=None, dtype='float32')` Bases: `BaseVectorizer` @@ -788,7 +788,7 @@ Initialize the Custom vectorizer. * **Raises:** **ValueError** – if embedding validation fails. -#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `embed(text, preprocess=None, as_buffer=False, **kwargs)` Generate an embedding for a single piece of text using your sync embed function. @@ -803,7 +803,7 @@ Generate an embedding for a single piece of text using your sync embed function. * **Raises:** **TypeError** – If the input is not a string. -#### embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, \*\*kwargs) +#### `embed_many(texts, preprocess=None, batch_size=10, as_buffer=False, **kwargs)` Generate embeddings for multiple pieces of text in batches using your sync embed_many function. @@ -820,7 +820,7 @@ Generate embeddings for multiple pieces of text in batches using your sync embed * **TypeError** – If the input is not a list of strings. * **NotImplementedError** – If no embed_many function was provided. -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -832,7 +832,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -840,7 +840,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### *class* VoyageAITextVectorizer(model='voyage-large-2', api_config=None, dtype='float32', \*, dims=None) +### `*class* VoyageAITextVectorizer(model='voyage-large-2', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -890,7 +890,7 @@ Visit [https://docs.voyageai.com/docs/embeddings](https://docs.voyageai.com/docs * **ImportError** – If the voyageai library is not installed. * **ValueError** – If the API key is not provided. -#### aembed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `aembed(text, preprocess=None, as_buffer=False, **kwargs)` Embed a chunk of text using the VoyageAI Embeddings API. @@ -921,7 +921,7 @@ querying the database, you should set the input_type=”query”. * **Raises:** **TypeError** – In an invalid input_type is provided. -#### aembed_many(texts, preprocess=None, batch_size=None, as_buffer=False, \*\*kwargs) +#### `aembed_many(texts, preprocess=None, batch_size=None, as_buffer=False, **kwargs)` Embed many chunks of text using the VoyageAI Embeddings API. @@ -954,7 +954,7 @@ querying the database, you should set the input_type=”query”. * **Raises:** **TypeError** – In an invalid input_type is provided. -#### embed(text, preprocess=None, as_buffer=False, \*\*kwargs) +#### `embed(text, preprocess=None, as_buffer=False, **kwargs)` Embed a chunk of text using the VoyageAI Embeddings API. @@ -985,7 +985,7 @@ querying the database, you should set the input_type=”query”. * **Raises:** **TypeError** – If an invalid input_type is provided. -#### embed_many(texts, preprocess=None, batch_size=None, as_buffer=False, \*\*kwargs) +#### `embed_many(texts, preprocess=None, batch_size=None, as_buffer=False, **kwargs)` Embed many chunks of text using the VoyageAI Embeddings API. @@ -1018,7 +1018,7 @@ querying the database, you should set the input_type=”query”. * **Raises:** **TypeError** – If an invalid input_type is provided. -#### model_post_init(context, /) +#### `model_post_init(context, /)` This function is meant to behave like a BaseModel method to initialise private attributes. @@ -1030,6 +1030,6 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### model_config *: ClassVar[ConfigDict]* *= {}* +#### `model_config *: ClassVar[ConfigDict]* *= {}*` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. From 13e0275204ad350c7cd3ee529850c2c9ea0dae2c Mon Sep 17 00:00:00 2001 From: paoloredis Date: Fri, 21 Mar 2025 10:51:16 +0100 Subject: [PATCH 04/13] Remove documentation page pointing to external site --- content/integrate/redisvl/docs.md | 9 --------- 1 file changed, 9 deletions(-) delete mode 100644 content/integrate/redisvl/docs.md diff --git a/content/integrate/redisvl/docs.md b/content/integrate/redisvl/docs.md deleted file mode 100644 index 0d362fcffe..0000000000 --- a/content/integrate/redisvl/docs.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -description: RedisVL documentation -linkTitle: Documentation -title: Documentation -type: integration -weight: 3 ---- - -Full documentation for RedisVL, including a [getting started guide](https://docs.redisvl.com/en/stable/user_guide/01_getting_started.html), is available [here](https://docs.redisvl.com/en/stable/). From 14800cd14dc37add0b6a9314f9ea04b3fa81ade2 Mon Sep 17 00:00:00 2001 From: paoloredis Date: Fri, 21 Mar 2025 16:46:29 +0100 Subject: [PATCH 05/13] Fix formatting --- content/integrate/redisvl/api/cache.md | 70 +++++----- content/integrate/redisvl/api/filter.md | 50 +++---- content/integrate/redisvl/api/query.md | 80 +++++------ content/integrate/redisvl/api/reranker.md | 40 +++--- content/integrate/redisvl/api/router.md | 94 ++++++------- content/integrate/redisvl/api/schema.md | 28 ++-- content/integrate/redisvl/api/searchindex.md | 126 +++++++++--------- .../integrate/redisvl/api/session_manager.md | 58 ++++---- content/integrate/redisvl/api/vectorizer.md | 116 ++++++++-------- 9 files changed, 331 insertions(+), 331 deletions(-) diff --git a/content/integrate/redisvl/api/cache.md b/content/integrate/redisvl/api/cache.md index 04f2eae7dc..788e266731 100644 --- a/content/integrate/redisvl/api/cache.md +++ b/content/integrate/redisvl/api/cache.md @@ -9,7 +9,7 @@ type: integration -### `*class* SemanticCache(name='llmcache', distance_threshold=0.1, ttl=None, vectorizer=None, filterable_fields=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, **kwargs)` +### `class SemanticCache(name='llmcache', distance_threshold=0.1, ttl=None, vectorizer=None, filterable_fields=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, **kwargs)` Bases: `BaseLLMCache` @@ -22,16 +22,16 @@ Semantic Cache for Large Language Models. Defaults to “llmcache”. * **distance_threshold** (*float* *,* *optional*) – Semantic threshold for the cache. Defaults to 0.1. - * **ttl** (*Optional* *[**int* *]* *,* *optional*) – The time-to-live for records cached + * **ttl** (*Optional* *[* *int* *]* *,* *optional*) – The time-to-live for records cached in Redis. Defaults to None. - * **vectorizer** (*Optional* *[**BaseVectorizer* *]* *,* *optional*) – The vectorizer for the cache. + * **vectorizer** (*Optional* *[* *BaseVectorizer* *]* *,* *optional*) – The vectorizer for the cache. Defaults to HFTextVectorizer. - * **filterable_fields** (*Optional* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *]*) – An optional list of RedisVL fields + * **filterable_fields** (*Optional* *[* *List* *[* *Dict* *[* *str* *,* *Any* *]* *]* *]*) – An optional list of RedisVL fields that can be used to customize cache retrieval with filters. - * **redis_client** (*Optional* *[**Redis* *]* *,* *optional*) – A redis client connection instance. + * **redis_client** (*Optional* *[* *Redis* *]* *,* *optional*) – A redis client connection instance. Defaults to None. * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. - * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments + * **connection_kwargs** (*Dict* *[* *str* *,* *Any* *]*) – The connection arguments for the redis client. Defaults to empty {}. * **overwrite** (*bool*) – Whether or not to force overwrite the schema for the semantic cache index. Defaults to false. @@ -41,7 +41,7 @@ Semantic Cache for Large Language Models. * **ValueError** – If the threshold is not between 0 and 1. * **ValueError** – If existing schema does not match new schema and overwrite is False. -#### `*async* acheck(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None)` +#### `async acheck(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None)` Async check the semantic cache for results similar to the specified prompt or vector. @@ -52,19 +52,19 @@ input. It checks for semantically similar prompts and fetches the cached LLM responses. * **Parameters:** - * **prompt** (*Optional* *[**str* *]* *,* *optional*) – The text prompt to search for in + * **prompt** (*Optional* *[* *str* *]* *,* *optional*) – The text prompt to search for in the cache. - * **vector** (*Optional* *[**List* *[**float* *]* *]* *,* *optional*) – The vector representation + * **vector** (*Optional* *[* *List* *[* *float* *]* *]* *,* *optional*) – The vector representation of the prompt to search for in the cache. * **num_results** (*int* *,* *optional*) – The number of cached results to return. Defaults to 1. - * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to include + * **return_fields** (*Optional* *[* *List* *[* *str* *]* *]* *,* *optional*) – The fields to include in each returned result. If None, defaults to all available fields in the cached entry. * **filter_expression** (*Optional* *[*[*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]*) – Optional filter expression that can be used to filter cache results. Defaults to None and the full cache will be searched. - * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic + * **distance_threshold** (*Optional* *[* *float* *]*) – The threshold for semantic vector distance. * **Returns:** A list of dicts containing the requested @@ -82,32 +82,32 @@ response = await cache.acheck( ) ``` -#### `*async* adrop(ids=None, keys=None)` +#### `async adrop(ids=None, keys=None)` Async expire specific entries from the cache by id or specific Redis key. * **Parameters:** - * **ids** (*Optional* *[**str* *]*) – The document ID or IDs to remove from the cache. - * **keys** (*Optional* *[**str* *]*) – The Redis keys to remove from the cache. + * **ids** (*Optional* *[* *str* *]*) – The document ID or IDs to remove from the cache. + * **keys** (*Optional* *[* *str* *]*) – The Redis keys to remove from the cache. * **Return type:** None -#### `*async* astore(prompt, response, vector=None, metadata=None, filters=None, ttl=None)` +#### `async astore(prompt, response, vector=None, metadata=None, filters=None, ttl=None)` Async stores the specified key-value pair in the cache along with metadata. * **Parameters:** * **prompt** (*str*) – The user prompt to cache. * **response** (*str*) – The LLM response to cache. - * **vector** (*Optional* *[**List* *[**float* *]* *]* *,* *optional*) – The prompt vector to + * **vector** (*Optional* *[* *List* *[* *float* *]* *]* *,* *optional*) – The prompt vector to cache. Defaults to None, and the prompt vector is generated on demand. - * **metadata** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The optional metadata to cache + * **metadata** (*Optional* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *optional*) – The optional metadata to cache alongside the prompt and response. Defaults to None. - * **filters** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]*) – The optional tag to assign to the cache entry. + * **filters** (*Optional* *[* *Dict* *[* *str* *,* *Any* *]* *]*) – The optional tag to assign to the cache entry. Defaults to None. - * **ttl** (*Optional* *[**int* *]*) – The optional TTL override to use on this individual cache + * **ttl** (*Optional* *[* *int* *]*) – The optional TTL override to use on this individual cache entry. Defaults to the global TTL setting. * **Returns:** The Redis key for the entries added to the semantic cache. @@ -126,7 +126,7 @@ key = await cache.astore( ) ``` -#### `*async* aupdate(key, **kwargs)` +#### `async aupdate(key, **kwargs)` Async update specific fields within an existing cache entry. If no fields are passed, then only the document TTL is refreshed. @@ -158,19 +158,19 @@ input. It checks for semantically similar prompts and fetches the cached LLM responses. * **Parameters:** - * **prompt** (*Optional* *[**str* *]* *,* *optional*) – The text prompt to search for in + * **prompt** (*Optional* *[* *str* *]* *,* *optional*) – The text prompt to search for in the cache. - * **vector** (*Optional* *[**List* *[**float* *]* *]* *,* *optional*) – The vector representation + * **vector** (*Optional* *[* *List* *[* *float* *]* *]* *,* *optional*) – The vector representation of the prompt to search for in the cache. * **num_results** (*int* *,* *optional*) – The number of cached results to return. Defaults to 1. - * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to include + * **return_fields** (*Optional* *[* *List* *[* *str* *]* *]* *,* *optional*) – The fields to include in each returned result. If None, defaults to all available fields in the cached entry. * **filter_expression** (*Optional* *[*[*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]*) – Optional filter expression that can be used to filter cache results. Defaults to None and the full cache will be searched. - * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic + * **distance_threshold** (*Optional* *[* *float* *]*) – The threshold for semantic vector distance. * **Returns:** A list of dicts containing the requested @@ -209,8 +209,8 @@ Manually expire specific entries from the cache by id or specific Redis key. * **Parameters:** - * **ids** (*Optional* *[**str* *]*) – The document ID or IDs to remove from the cache. - * **keys** (*Optional* *[**str* *]*) – The Redis keys to remove from the cache. + * **ids** (*Optional* *[* *str* *]*) – The document ID or IDs to remove from the cache. + * **keys** (*Optional* *[* *str* *]*) – The Redis keys to remove from the cache. * **Return type:** None @@ -231,7 +231,7 @@ Sets the semantic distance threshold for the cache. Set the default TTL, in seconds, for entries in the cache. * **Parameters:** - **ttl** (*Optional* *[**int* *]* *,* *optional*) – The optional time-to-live expiration + **ttl** (*Optional* *[* *int* *]* *,* *optional*) – The optional time-to-live expiration for the cache, in seconds. * **Raises:** **ValueError** – If the time-to-live value is not an integer. @@ -243,14 +243,14 @@ Stores the specified key-value pair in the cache along with metadata. * **Parameters:** * **prompt** (*str*) – The user prompt to cache. * **response** (*str*) – The LLM response to cache. - * **vector** (*Optional* *[**List* *[**float* *]* *]* *,* *optional*) – The prompt vector to + * **vector** (*Optional* *[* *List* *[* *float* *]* *]* *,* *optional*) – The prompt vector to cache. Defaults to None, and the prompt vector is generated on demand. - * **metadata** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The optional metadata to cache + * **metadata** (*Optional* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *optional*) – The optional metadata to cache alongside the prompt and response. Defaults to None. - * **filters** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]*) – The optional tag to assign to the cache entry. + * **filters** (*Optional* *[* *Dict* *[* *str* *,* *Any* *]* *]*) – The optional tag to assign to the cache entry. Defaults to None. - * **ttl** (*Optional* *[**int* *]*) – The optional TTL override to use on this individual cache + * **ttl** (*Optional* *[* *int* *]*) – The optional TTL override to use on this individual cache entry. Defaults to the global TTL setting. * **Returns:** The Redis key for the entries added to the semantic cache. @@ -288,7 +288,7 @@ cache.update(key, metadata={"hit_count": 1, "model_name": "Llama-2-7b"}) ) ``` -#### `*property* aindex *: [AsyncSearchIndex]({{< relref "searchindex/#asyncsearchindex" >}}) | None*` +#### `property aindex: `[`AsyncSearchIndex`]({{< relref "searchindex/#asyncsearchindex" >}})` | None` The underlying AsyncSearchIndex for the cache. @@ -297,7 +297,7 @@ The underlying AsyncSearchIndex for the cache. * **Return type:** [AsyncSearchIndex]({{< relref "searchindex/#asyncsearchindex" >}}) -#### `*property* distance_threshold *: float*` +#### `property distance_threshold: float` The semantic distance threshold for the cache. @@ -306,7 +306,7 @@ The semantic distance threshold for the cache. * **Return type:** float -#### `*property* index *: [SearchIndex]({{< relref "searchindex/#searchindex" >}})*` +#### `property index: `[`SearchIndex`]({{< relref "searchindex/#searchindex" >}})` ` The underlying SearchIndex for the cache. @@ -315,6 +315,6 @@ The underlying SearchIndex for the cache. * **Return type:** [SearchIndex]({{< relref "searchindex/#searchindex" >}}) -#### `*property* ttl *: int | None*` +#### `property ttl: int | None` The default TTL, in seconds, for entries in the cache. diff --git a/content/integrate/redisvl/api/filter.md b/content/integrate/redisvl/api/filter.md index 22fb494021..c4e6f80885 100644 --- a/content/integrate/redisvl/api/filter.md +++ b/content/integrate/redisvl/api/filter.md @@ -9,7 +9,7 @@ type: integration ## FilterExpression -### `*class* FilterExpression(_filter=None, operator=None, left=None, right=None)` +### `class FilterExpression(_filter=None, operator=None, left=None, right=None)` A FilterExpression is a logical combination of filters in RedisVL. @@ -51,12 +51,12 @@ built by combining filter statements using the & and | operators. * **Parameters:** * **\_filter** (*str* *|* *None*) * **operator** (*FilterOperator* *|* *None*) - * **left** ([*FilterExpression*](#filterexpression) *|* *None*) - * **right** ([*FilterExpression*](#filterexpression) *|* *None*) + * **left** ([filterexpression](#filterexpression) *|* *None*) + * **right** ([filterexpression](#filterexpression) *|* *None*) ## Tag -### `*class* Tag(field)` +### `class Tag(field)` A Tag filter can be applied to Tag fields @@ -68,9 +68,9 @@ A Tag filter can be applied to Tag fields Create a Tag equality filter expression. * **Parameters:** - **other** (*Union* *[**List* *[**str* *]* *,* *str* *]*) – The tag(s) to filter on. + **other** (*Union* *[* *List* *[* *str* *]* *,* *str* *]*) – The tag(s) to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Tag @@ -83,9 +83,9 @@ f = Tag("brand") == "nike" Create a Tag inequality filter expression. * **Parameters:** - **other** (*Union* *[**List* *[**str* *]* *,* *str* *]*) – The tag(s) to filter on. + **other** (*Union* *[* *List* *[* *str* *]* *,* *str* *]*) – The tag(s) to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Tag @@ -101,7 +101,7 @@ Return the Redis Query string for the Tag filter ## Text -### `*class* Text(field)` +### `class Text(field)` A Text is a FilterField representing a text field in a Redis index. @@ -116,7 +116,7 @@ filters that enforce an exact match on the supplied term(s). * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Text @@ -134,7 +134,7 @@ term(s). * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Text @@ -154,7 +154,7 @@ equality filter expression. * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Text @@ -171,7 +171,7 @@ Return the Redis Query string for the Text filter ## Num -### `*class* Num(field)` +### `class Num(field)` A Num is a FilterField representing a numeric field in a Redis index. @@ -185,7 +185,7 @@ Create a Numeric equality filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -199,7 +199,7 @@ Create a Numeric greater than or equal to filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -214,7 +214,7 @@ Create a Numeric greater than filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -229,7 +229,7 @@ Create a Numeric less than or equal to filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -244,7 +244,7 @@ Create a Numeric less than filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -259,7 +259,7 @@ Create a Numeric inequality filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -276,7 +276,7 @@ Return the Redis Query string for the Numeric filter ## Geo -### `*class* Geo(field)` +### `class Geo(field)` A Geo is a FilterField representing a geographic (lat/lon) field in a Redis index. @@ -289,9 +289,9 @@ Redis index. Create a geographic filter within a specified GeoRadius. * **Parameters:** - **other** ([*GeoRadius*](#georadius)) – The geographic spec to filter on. + **other** ([georadius](#georadius)) – The geographic spec to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Geo, GeoRadius @@ -304,9 +304,9 @@ f = Geo("location") == GeoRadius(-122.4194, 37.7749, 1, unit="m") Create a geographic filter outside of a specified GeoRadius. * **Parameters:** - **other** ([*GeoRadius*](#georadius)) – The geographic spec to filter on. + **other** ([georadius](#georadius)) – The geographic spec to filter on. * **Return type:** - [*FilterExpression*](#filterexpression) + [filterexpression](#filterexpression) ```python from redisvl.query.filter import Geo, GeoRadius @@ -323,7 +323,7 @@ Return the Redis Query string for the Geo filter ## GeoRadius -### `*class* GeoRadius(longitude, latitude, radius=1, unit='km')` +### `class GeoRadius(longitude, latitude, radius=1, unit='km')` A GeoRadius is a GeoSpec representing a geographic radius. diff --git a/content/integrate/redisvl/api/query.md b/content/integrate/redisvl/api/query.md index 6b7123bd8e..b419568990 100644 --- a/content/integrate/redisvl/api/query.md +++ b/content/integrate/redisvl/api/query.md @@ -11,7 +11,7 @@ queries for different use cases. Each query class wraps the `redis-py` Query mod ## VectorQuery -### `*class* VectorQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False)` +### `class VectorQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False)` Bases: `BaseVectorQuery`, `BaseQuery` @@ -19,12 +19,12 @@ A query for running a vector search along with an optional filter expression. * **Parameters:** - * **vector** (*List* *[**float* *]*) – The vector to perform the vector search with. + * **vector** (*List* *[* *float* *]*) – The vector to perform the vector search with. * **vector_field_name** (*str*) – The name of the vector field to search against in the database. - * **return_fields** (*List* *[**str* *]*) – The declared fields to return with search + * **return_fields** (*List* *[* *str* *]*) – The declared fields to return with search results. - * **filter_expression** (*Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *,* *optional*) – A filter to apply + * **filter_expression** (*Union* *[* *str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *,* *optional*) – A filter to apply along with the vector search. Defaults to None. * **dtype** (*str* *,* *optional*) – The dtype of the vector. Defaults to “float32”. @@ -34,7 +34,7 @@ expression. distance. Defaults to True. * **dialect** (*int* *,* *optional*) – The RediSearch query dialect. Defaults to 2. - * **sort_by** (*Optional* *[**str* *]*) – The field to order the results by. Defaults + * **sort_by** (*Optional* *[* *str* *]*) – The field to order the results by. Defaults to None. Results will be ordered by vector distance. * **in_order** (*bool*) – Requires the terms in the field to have the same order as the terms in the query filter, regardless of @@ -94,7 +94,7 @@ Limit the search to specific TEXT fields only. from the defined schema. * **Parameters:** - **fields** (*List* *[**str* *]*) + **fields** (*List* *[* *str* *]*) * **Return type:** *Query* @@ -165,7 +165,7 @@ Default is TFIDF. Set the filter expression for the query. * **Parameters:** - **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter + **filter_expression** (*Optional* *[* *Union* *[* *str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter expression or query string to use on the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. @@ -224,11 +224,11 @@ Ask the engine to return document search scores. * **Return type:** *Query* -#### `*property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})*` +#### `property filter: str | `[`FilterExpression`]({{< relref "filter/#filterexpression" >}})` ` The filter expression for the query. -#### `*property* params *: Dict[str, Any]*` +#### `property params: Dict[str, Any]` Return the parameters for the query. @@ -237,13 +237,13 @@ Return the parameters for the query. * **Return type:** Dict[str, Any] -#### `*property* query *: BaseQuery*` +#### `property query: BaseQuery` Return self as the query object. ## VectorRangeQuery -### `*class* VectorRangeQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', distance_threshold=0.2, num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False)` +### `class VectorRangeQuery(vector, vector_field_name, return_fields=None, filter_expression=None, dtype='float32', distance_threshold=0.2, num_results=10, return_score=True, dialect=2, sort_by=None, in_order=False)` Bases: `BaseVectorQuery`, `BaseQuery` @@ -251,12 +251,12 @@ A query for running a filtered vector search based on semantic distance threshold. * **Parameters:** - * **vector** (*List* *[**float* *]*) – The vector to perform the range query with. + * **vector** (*List* *[* *float* *]*) – The vector to perform the range query with. * **vector_field_name** (*str*) – The name of the vector field to search against in the database. - * **return_fields** (*List* *[**str* *]*) – The declared fields to return with search + * **return_fields** (*List* *[* *str* *]*) – The declared fields to return with search results. - * **filter_expression** (*Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *,* *optional*) – A filter to apply + * **filter_expression** (*Union* *[* *str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *,* *optional*) – A filter to apply along with the range query. Defaults to None. * **dtype** (*str* *,* *optional*) – The dtype of the vector. Defaults to “float32”. @@ -269,7 +269,7 @@ distance threshold. distance. Defaults to True. * **dialect** (*int* *,* *optional*) – The RediSearch query dialect. Defaults to 2. - * **sort_by** (*Optional* *[**str* *]*) – The field to order the results by. Defaults + * **sort_by** (*Optional* *[* *str* *]*) – The field to order the results by. Defaults to None. Results will be ordered by vector distance. * **in_order** (*bool*) – Requires the terms in the field to have the same order as the terms in the query filter, regardless of @@ -329,7 +329,7 @@ Limit the search to specific TEXT fields only. from the defined schema. * **Parameters:** - **fields** (*List* *[**str* *]*) + **fields** (*List* *[* *str* *]*) * **Return type:** *Query* @@ -407,7 +407,7 @@ Set the distance threshold for the query. Set the filter expression for the query. * **Parameters:** - **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter + **filter_expression** (*Optional* *[* *Union* *[* *str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter expression or query string to use on the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. @@ -466,7 +466,7 @@ Ask the engine to return document search scores. * **Return type:** *Query* -#### `*property* distance_threshold *: float*` +#### `property distance_threshold: float` Return the distance threshold for the query. @@ -475,11 +475,11 @@ Return the distance threshold for the query. * **Return type:** float -#### `*property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})*` +#### `property filter: str | `[`FilterExpression`]({{< relref "filter/#filterexpression" >}})` ` The filter expression for the query. -#### `*property* params *: Dict[str, Any]*` +#### `property params: Dict[str, Any]` Return the parameters for the query. @@ -488,27 +488,27 @@ Return the parameters for the query. * **Return type:** Dict[str, Any] -#### `*property* query *: BaseQuery*` +#### `property query: BaseQuery` Return self as the query object. ## FilterQuery -### `*class* FilterQuery(filter_expression=None, return_fields=None, num_results=10, dialect=2, sort_by=None, in_order=False, params=None)` +### `class FilterQuery(filter_expression=None, return_fields=None, num_results=10, dialect=2, sort_by=None, in_order=False, params=None)` Bases: `BaseQuery` A query for running a filtered search with a filter expression. * **Parameters:** - * **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]*) – The optional filter + * **filter_expression** (*Optional* *[* *Union* *[* *str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]*) – The optional filter expression to query with. Defaults to ‘\*’. - * **return_fields** (*Optional* *[**List* *[**str* *]* *]* *,* *optional*) – The fields to return. - * **num_results** (*Optional* *[**int* *]* *,* *optional*) – The number of results to return. Defaults to 10. + * **return_fields** (*Optional* *[* *List* *[* *str* *]* *]* *,* *optional*) – The fields to return. + * **num_results** (*Optional* *[* *int* *]* *,* *optional*) – The number of results to return. Defaults to 10. * **dialect** (*int* *,* *optional*) – The query dialect. Defaults to 2. - * **sort_by** (*Optional* *[**str* *]* *,* *optional*) – The field to order the results by. Defaults to None. + * **sort_by** (*Optional* *[* *str* *]* *,* *optional*) – The field to order the results by. Defaults to None. * **in_order** (*bool* *,* *optional*) – Requires the terms in the field to have the same order as the terms in the query filter. Defaults to False. - * **params** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The parameters for the query. Defaults to None. + * **params** (*Optional* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *optional*) – The parameters for the query. Defaults to None. * **Raises:** **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression @@ -561,7 +561,7 @@ Limit the search to specific TEXT fields only. from the defined schema. * **Parameters:** - **fields** (*List* *[**str* *]*) + **fields** (*List* *[* *str* *]*) * **Return type:** *Query* @@ -632,7 +632,7 @@ Default is TFIDF. Set the filter expression for the query. * **Parameters:** - **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter + **filter_expression** (*Optional* *[* *Union* *[* *str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter expression or query string to use on the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. @@ -691,29 +691,29 @@ Ask the engine to return document search scores. * **Return type:** *Query* -#### `*property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})*` +#### `property filter: str | `[`FilterExpression`]({{< relref "filter/#filterexpression" >}})` ` The filter expression for the query. -#### `*property* params *: Dict[str, Any]*` +#### `property params: Dict[str, Any]` Return the query parameters. -#### `*property* query *: BaseQuery*` +#### `property query: BaseQuery` Return self as the query object. ## CountQuery -### `*class* CountQuery(filter_expression=None, dialect=2, params=None)` +### `class CountQuery(filter_expression=None, dialect=2, params=None)` Bases: `BaseQuery` A query for a simple count operation provided some filter expression. * **Parameters:** - * **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]*) – The filter expression to query with. Defaults to None. - * **params** (*Optional* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *optional*) – The parameters for the query. Defaults to None. + * **filter_expression** (*Optional* *[* *Union* *[* *str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]*) – The filter expression to query with. Defaults to None. + * **params** (*Optional* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *optional*) – The parameters for the query. Defaults to None. * **dialect** (*int*) * **Raises:** **TypeError** – If filter_expression is not of type redisvl.query.FilterExpression @@ -777,7 +777,7 @@ Limit the search to specific TEXT fields only. from the defined schema. * **Parameters:** - **fields** (*List* *[**str* *]*) + **fields** (*List* *[* *str* *]*) * **Return type:** *Query* @@ -848,7 +848,7 @@ Default is TFIDF. Set the filter expression for the query. * **Parameters:** - **filter_expression** (*Optional* *[**Union* *[**str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter + **filter_expression** (*Optional* *[* *Union* *[* *str* *,* [*FilterExpression*]({{< relref "filter/#filterexpression" >}}) *]* *]* *,* *optional*) – The filter expression or query string to use on the query. * **Raises:** **TypeError** – If filter_expression is not a valid FilterExpression or string. @@ -907,14 +907,14 @@ Ask the engine to return document search scores. * **Return type:** *Query* -#### `*property* filter *: str | [FilterExpression]({{< relref "filter/#filterexpression" >}})*` +#### `property filter: str | `[`FilterExpression`]({{< relref "filter/#filterexpression" >}})` ` The filter expression for the query. -#### `*property* params *: Dict[str, Any]*` +#### `property params: Dict[str, Any]` Return the query parameters. -#### `*property* query *: BaseQuery*` +#### `property query: BaseQuery` Return self as the query object. diff --git a/content/integrate/redisvl/api/reranker.md b/content/integrate/redisvl/api/reranker.md index 4d15bda662..11a40b5327 100644 --- a/content/integrate/redisvl/api/reranker.md +++ b/content/integrate/redisvl/api/reranker.md @@ -9,7 +9,7 @@ type: integration -### `*class* CohereReranker(model='rerank-english-v3.0', rank_by=None, limit=5, return_score=True, api_config=None)` +### `class CohereReranker(model='rerank-english-v3.0', rank_by=None, limit=5, return_score=True, api_config=None)` Bases: `BaseReranker` @@ -45,7 +45,7 @@ and API configuration. * **Parameters:** * **model** (*str*) – The identifier for the Cohere model used for reranking. Defaults to ‘rerank-english-v3.0’. - * **rank_by** (*Optional* *[**List* *[**str* *]* *]*) – Optional list of keys specifying the + * **rank_by** (*Optional* *[* *List* *[* *str* *]* *]*) – Optional list of keys specifying the attributes in the documents that should be considered for ranking. None means ranking will rely on the model’s default behavior. @@ -53,13 +53,13 @@ and API configuration. reranking. Must be a positive integer. * **return_score** (*bool*) – Whether to return scores alongside the reranked results. - * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the API key. + * **api_config** (*Optional* *[* *Dict* *]* *,* *optional*) – Dictionary containing the API key. Defaults to None. * **Raises:** * **ImportError** – If the cohere library is not installed. * **ValueError** – If the API key is not provided. -#### `*async* arank(query, docs, **kwargs)` +#### `async arank(query, docs, **kwargs)` Rerank documents based on the provided query using the Cohere rerank API. @@ -69,7 +69,7 @@ query’s context. * **Parameters:** * **query** (*str*) – The user’s search query. - * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents + * **docs** (*Union* *[* *List* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *List* *[* *str* *]* *]*) – The list of documents to be ranked, either as dictionaries or strings. * **Returns:** The reranked list of documents and optionally associated scores. @@ -98,14 +98,14 @@ query’s context. * **Parameters:** * **query** (*str*) – The user’s search query. - * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents + * **docs** (*Union* *[* *List* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *List* *[* *str* *]* *]*) – The list of documents to be ranked, either as dictionaries or strings. * **Returns:** The reranked list of documents and optionally associated scores. * **Return type:** Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -113,7 +113,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* HFCrossEncoderReranker(model='cross-encoder/ms-marco-MiniLM-L-6-v2', limit=3, return_score=True, *, rank_by=None)` +### `class HFCrossEncoderReranker(model='cross-encoder/ms-marco-MiniLM-L-6-v2', limit=3, return_score=True, *, rank_by=None)` Bases: `BaseReranker` @@ -147,9 +147,9 @@ Initialize the HFCrossEncoderReranker with a specified model and ranking criteri Defaults to ‘cross-encoder/ms-marco-MiniLM-L-6-v2’. * **limit** (*int*) – The maximum number of results to return after reranking. Must be a positive integer. * **return_score** (*bool*) – Whether to return scores alongside the reranked results. - * **rank_by** (*List* *[**str* *]* *|* *None*) + * **rank_by** (*List* *[* *str* *]* *|* *None*) -#### `*async* arank(query, docs, **kwargs)` +#### `async arank(query, docs, **kwargs)` Asynchronously rerank documents based on the provided query using the loaded cross-encoder model. @@ -158,7 +158,7 @@ in a manner that is potentially more relevant to the query’s context. * **Parameters:** * **query** (*str*) – The user’s search query. - * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents to be ranked, + * **docs** (*Union* *[* *List* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *List* *[* *str* *]* *]*) – The list of documents to be ranked, either as dictionaries or strings. * **Returns:** The reranked list of documents and optionally associated scores. @@ -186,14 +186,14 @@ in a manner that is potentially more relevant to the query’s context. * **Parameters:** * **query** (*str*) – The user’s search query. - * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents to be ranked, + * **docs** (*Union* *[* *List* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *List* *[* *str* *]* *]*) – The list of documents to be ranked, either as dictionaries or strings. * **Returns:** The reranked list of documents and optionally associated scores. * **Return type:** Union[Tuple[List[Dict[str, Any]], List[float]], List[Dict[str, Any]]] -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -201,7 +201,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* VoyageAIReranker(model, rank_by=None, limit=5, return_score=True, api_config=None)` +### `class VoyageAIReranker(model, rank_by=None, limit=5, return_score=True, api_config=None)` Bases: `BaseReranker` @@ -236,7 +236,7 @@ and API configuration. * **Parameters:** * **model** (*str*) – The identifier for the VoyageAI model used for reranking. - * **rank_by** (*Optional* *[**List* *[**str* *]* *]*) – Optional list of keys specifying the + * **rank_by** (*Optional* *[* *List* *[* *str* *]* *]*) – Optional list of keys specifying the attributes in the documents that should be considered for ranking. None means ranking will rely on the model’s default behavior. @@ -244,13 +244,13 @@ and API configuration. reranking. Must be a positive integer. * **return_score** (*bool*) – Whether to return scores alongside the reranked results. - * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the API key. + * **api_config** (*Optional* *[* *Dict* *]* *,* *optional*) – Dictionary containing the API key. Defaults to None. * **Raises:** * **ImportError** – If the voyageai library is not installed. * **ValueError** – If the API key is not provided. -#### `*async* arank(query, docs, **kwargs)` +#### `async arank(query, docs, **kwargs)` Rerank documents based on the provided query using the VoyageAI rerank API. @@ -260,7 +260,7 @@ query’s context. * **Parameters:** * **query** (*str*) – The user’s search query. - * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents + * **docs** (*Union* *[* *List* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *List* *[* *str* *]* *]*) – The list of documents to be ranked, either as dictionaries or strings. * **Returns:** The reranked list of documents and optionally associated scores. @@ -289,13 +289,13 @@ query’s context. * **Parameters:** * **query** (*str*) – The user’s search query. - * **docs** (*Union* *[**List* *[**Dict* *[**str* *,* *Any* *]* *]* *,* *List* *[**str* *]* *]*) – The list of documents + * **docs** (*Union* *[* *List* *[* *Dict* *[* *str* *,* *Any* *]* *]* *,* *List* *[* *str* *]* *]*) – The list of documents to be ranked, either as dictionaries or strings. * **Returns:** The reranked list of documents and optionally associated scores. * **Return type:** Union[Tuple[Union[List[Dict[str, Any]], List[str]], float], List[Dict[str, Any]]] -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. diff --git a/content/integrate/redisvl/api/router.md b/content/integrate/redisvl/api/router.md index b94617b820..cdd46fb7ea 100644 --- a/content/integrate/redisvl/api/router.md +++ b/content/integrate/redisvl/api/router.md @@ -9,7 +9,7 @@ type: integration ## Semantic Router -### `*class* SemanticRouter(name, routes, vectorizer=None, routing_config=None, redis_client=None, redis_url='redis://localhost:6379', overwrite=False, connection_kwargs={})` +### `class SemanticRouter(name, routes, vectorizer=None, routing_config=None, redis_client=None, redis_url='redis://localhost:6379', overwrite=False, connection_kwargs={})` Semantic Router for managing and querying route vectors. @@ -17,13 +17,13 @@ Initialize the SemanticRouter. * **Parameters:** * **name** (*str*) – The name of the semantic router. - * **routes** (*List* *[*[*Route*](#route) *]*) – List of Route objects. + * **routes** (*List* *[*[route](#route) *]*) – List of Route objects. * **vectorizer** (*BaseVectorizer* *,* *optional*) – The vectorizer used to embed route references. Defaults to default HFTextVectorizer. - * **routing_config** ([*RoutingConfig*](#routingconfig) *,* *optional*) – Configuration for routing behavior. Defaults to the default RoutingConfig. - * **redis_client** (*Optional* *[**Redis* *]* *,* *optional*) – Redis client for connection. Defaults to None. + * **routing_config** ([routingconfig](#routingconfig) *,* *optional*) – Configuration for routing behavior. Defaults to the default RoutingConfig. + * **redis_client** (*Optional* *[* *Redis* *]* *,* *optional*) – Redis client for connection. Defaults to None. * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. * **overwrite** (*bool* *,* *optional*) – Whether to overwrite existing index. Defaults to False. - * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments + * **connection_kwargs** (*Dict* *[* *str* *,* *Any* *]*) – The connection arguments for the redis client. Defaults to empty {}. #### `clear()` @@ -40,16 +40,16 @@ Delete the semantic router index. * **Return type:** None -#### `*classmethod* from_dict(data, **kwargs)` +#### `classmethod from_dict(data, **kwargs)` Create a SemanticRouter from a dictionary. * **Parameters:** - **data** (*Dict* *[**str* *,* *Any* *]*) – The dictionary containing the semantic router data. + **data** (*Dict* *[* *str* *,* *Any* *]*) – The dictionary containing the semantic router data. * **Returns:** The semantic router instance. * **Return type:** - [SemanticRouter](#semanticrouter) + [semanticrouter](#semanticrouter) * **Raises:** **ValueError** – If required data is missing or invalid. @@ -63,7 +63,7 @@ router_data = { router = SemanticRouter.from_dict(router_data) ``` -#### `*classmethod* from_yaml(file_path, **kwargs)` +#### `classmethod from_yaml(file_path, **kwargs)` Create a SemanticRouter from a YAML file. @@ -72,7 +72,7 @@ Create a SemanticRouter from a YAML file. * **Returns:** The semantic router instance. * **Return type:** - [SemanticRouter](#semanticrouter) + [semanticrouter](#semanticrouter) * **Raises:** * **ValueError** – If the file path is invalid. * **FileNotFoundError** – If the file does not exist. @@ -91,7 +91,7 @@ Get a route by its name. * **Returns:** The selected Route object or None if not found. * **Return type:** - Optional[[Route](#route)] + Optional[[route](#route)] #### `model_post_init(context, /)` @@ -119,15 +119,15 @@ Remove a route and all references from the semantic router. Query the semantic router with a given statement or vector for multiple matches. * **Parameters:** - * **statement** (*Optional* *[**str* *]*) – The input statement to be queried. - * **vector** (*Optional* *[**List* *[**float* *]* *]*) – The input vector to be queried. - * **max_k** (*Optional* *[**int* *]*) – The maximum number of top matches to return. - * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic distance. - * **aggregation_method** (*Optional* *[*[*DistanceAggregationMethod*](#distanceaggregationmethod) *]*) – The aggregation method used for vector distances. + * **statement** (*Optional* *[* *str* *]*) – The input statement to be queried. + * **vector** (*Optional* *[* *List* *[* *float* *]* *]*) – The input vector to be queried. + * **max_k** (*Optional* *[* *int* *]*) – The maximum number of top matches to return. + * **distance_threshold** (*Optional* *[* *float* *]*) – The threshold for semantic distance. + * **aggregation_method** (*Optional* *[*[distanceaggregationmethod](#distanceaggregationmethod) *]*) – The aggregation method used for vector distances. * **Returns:** The matching routes and their details. * **Return type:** - List[[RouteMatch](#routematch)] + List[[routematch](#routematch)] #### `to_dict()` @@ -171,17 +171,17 @@ router.to_yaml("router.yaml") Update the routing configuration. * **Parameters:** - **routing_config** ([*RoutingConfig*](#routingconfig)) – The new routing configuration. + **routing_config** ([routingconfig](#routingconfig)) – The new routing configuration. -#### `model_config *: ClassVar[ConfigDict]* *= {'arbitrary_types_allowed': True}*` +#### `model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. -#### `name *: str*` +#### `name: str` The name of the semantic router. -#### `*property* route_names *: List[str]*` +#### `property route_names: List[str]` Get the list of route names. @@ -190,7 +190,7 @@ Get the list of route names. * **Return type:** List[str] -#### `*property* route_thresholds *: Dict[str, float | None]*` +#### `property route_thresholds: Dict[str, float | None]` Get the distance thresholds for each route. @@ -199,21 +199,21 @@ Get the distance thresholds for each route. * **Return type:** Dict[str, float] -#### `routes *: List[[Route](#route)]*` +#### `routes: `List[[route](#route)] List of Route objects. -#### `routing_config *: [RoutingConfig](#routingconfig)*` +#### `routing_config: `[routingconfig](#routingconfig) Configuration for routing behavior. -#### `vectorizer *: BaseVectorizer*` +#### `vectorizer: BaseVectorizer` The vectorizer used to embed route references. ## Routing Config -### `*class* RoutingConfig(*, max_k=1, aggregation_method=DistanceAggregationMethod.avg)` +### `class RoutingConfig(*, max_k=1, aggregation_method=DistanceAggregationMethod.avg)` Configuration for routing behavior. @@ -225,20 +225,20 @@ validated to form a valid model. self is explicitly positional-only to allow self as a field name. * **Parameters:** - * **max_k** (*Annotated* *[**int* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=False* *,* *default=1* *,* *metadata=* *[**Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *]* *)* *]*) - * **aggregation_method** ([*DistanceAggregationMethod*](#distanceaggregationmethod)) + * **max_k** (*Annotated* *[* *int* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=False* *,* *default=1* *,* *metadata=* *[* *Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *]* *)* *]*) + * **aggregation_method** ([distanceaggregationmethod](#distanceaggregationmethod)) -#### `max_k *: Annotated[int, FieldInfo(annotation=NoneType, required=False, default=1, metadata=[Strict(strict=True), Gt(gt=0)])]*` +#### `max_k: Annotated[int, FieldInfo(annotation=NoneType, required=False, default=1, metadata=[Strict(strict=True), Gt(gt=0)])]` Aggregation method to use to classify queries. -#### `model_config *: ClassVar[ConfigDict]* *= {'extra': 'ignore'}*` +#### `model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. ## Route -### `*class* Route(*, name, references, metadata={}, distance_threshold=0.5)` +### `class Route(*, name, references, metadata={}, distance_threshold=0.5)` Model representing a routing path with associated metadata and thresholds. @@ -251,33 +251,33 @@ self is explicitly positional-only to allow self as a field name. * **Parameters:** * **name** (*str*) - * **references** (*List* *[**str* *]*) - * **metadata** (*Dict* *[**str* *,* *Any* *]*) - * **distance_threshold** (*Annotated* *[**float* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=True* *,* *metadata=* *[**Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *,* *Le* *(**le=1* *)* *]* *)* *]*) + * **references** (*List* *[* *str* *]*) + * **metadata** (*Dict* *[* *str* *,* *Any* *]*) + * **distance_threshold** (*Annotated* *[* *float* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=True* *,* *metadata=* *[* *Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *,* *Le* *(**le=1* *)* *]* *)* *]*) -#### `distance_threshold *: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Strict(strict=True), Gt(gt=0), Le(le=1)])]*` +#### `distance_threshold: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Strict(strict=True), Gt(gt=0), Le(le=1)])]` Distance threshold for matching the route. -#### `metadata *: Dict[str, Any]*` +#### `metadata: Dict[str, Any]` Metadata associated with the route. -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. -#### `name *: str*` +#### `name: str` The name of the route. -#### `references *: List[str]*` +#### `references: List[str]` List of reference phrases for the route. ## Route Match -### `*class* RouteMatch(*, name=None, distance=None)` +### `class RouteMatch(*, name=None, distance=None)` Model representing a matched route with distance information. @@ -292,32 +292,32 @@ self is explicitly positional-only to allow self as a field name. * **name** (*str* *|* *None*) * **distance** (*float* *|* *None*) -#### `distance *: float | None*` +#### `distance: float | None` The vector distance between the statement and the matched route. -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. -#### `name *: str | None*` +#### `name: str | None` The matched route name. ## Distance Aggregation Method -### `*class* DistanceAggregationMethod(value, names=, *values, module=None, qualname=None, type=None, start=1, boundary=None)` +### `class DistanceAggregationMethod(value, names=, *values, module=None, qualname=None, type=None, start=1, boundary=None)` Enumeration for distance aggregation methods. -#### `avg *= 'avg'*` +#### `avg = 'avg'` Compute the average of the vector distances. -#### `min *= 'min'*` +#### `min = 'min'` Compute the minimum of the vector distances. -#### `sum *= 'sum'*` +#### `sum = 'sum'` Compute the sum of the vector distances. diff --git a/content/integrate/redisvl/api/schema.md b/content/integrate/redisvl/api/schema.md index eb3bfef6d3..3060e2170e 100644 --- a/content/integrate/redisvl/api/schema.md +++ b/content/integrate/redisvl/api/schema.md @@ -18,7 +18,7 @@ field configurations using the following three components: -### `*class* IndexSchema(*, index, fields={}, version='0.1.0')` +### `class IndexSchema(*, index, fields={}, version='0.1.0')` A schema definition for a search index in Redis, used in RedisVL for configuring index settings and organizing vector and metadata fields. @@ -102,7 +102,7 @@ self is explicitly positional-only to allow self as a field name. * **Parameters:** * **index** (*IndexInfo*) - * **fields** (*Dict* *[**str* *,* *BaseField* *]*) + * **fields** (*Dict* *[* *str* *,* *BaseField* *]*) * **version** (*Literal* *[* *'0.1.0'* *]*) #### `add_field(field_inputs)` @@ -114,7 +114,7 @@ This method allows for the addition of individual fields to the schema, providing flexibility in defining the structure of the index. * **Parameters:** - **field_inputs** (*Dict* *[**str* *,* *Any* *]*) – A field to add. + **field_inputs** (*Dict* *[* *str* *,* *Any* *]*) – A field to add. * **Raises:** **ValueError** – If the field name or type are not provided or if the name already exists within the schema. @@ -143,7 +143,7 @@ This method allows dynamically adding new fields to the index schema. It processes a list of field definitions. * **Parameters:** - **fields** (*List* *[**Dict* *[**str* *,* *Any* *]* *]*) – A list of fields to add. + **fields** (*List* *[* *Dict* *[* *str* *,* *Any* *]* *]*) – A list of fields to add. * **Raises:** **ValueError** – If a field with the same name already exists in the schema. @@ -164,16 +164,16 @@ schema.add_fields([ ]) ``` -#### `*classmethod* from_dict(data)` +#### `classmethod from_dict(data)` Create an IndexSchema from a dictionary. * **Parameters:** - **data** (*Dict* *[**str* *,* *Any* *]*) – The index schema data. + **data** (*Dict* *[* *str* *,* *Any* *]*) – The index schema data. * **Returns:** The index schema. * **Return type:** - [IndexSchema](#indexschema) + [indexschema](#indexschema) ```python from redisvl.schema import IndexSchema @@ -201,7 +201,7 @@ schema = IndexSchema.from_dict({ }) ``` -#### `*classmethod* from_yaml(file_path)` +#### `classmethod from_yaml(file_path)` Create an IndexSchema from a YAML file. @@ -210,7 +210,7 @@ Create an IndexSchema from a YAML file. * **Returns:** The index schema. * **Return type:** - [IndexSchema](#indexschema) + [indexschema](#indexschema) ```python from redisvl.schema import IndexSchema @@ -249,7 +249,7 @@ Write the index schema to a YAML file. * **Return type:** None -#### `*property* field_names *: List[str]*` +#### `property field_names: List[str]` A list of field names associated with the index schema. @@ -258,19 +258,19 @@ A list of field names associated with the index schema. * **Return type:** List[str] -#### `fields *: Dict[str, BaseField]*` +#### `fields: Dict[str, BaseField]` Fields associated with the search index and their properties -#### `index *: IndexInfo*` +#### `index: IndexInfo` Details of the basic index configurations. -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. -#### `version *: Literal['0.1.0']*` +#### `version: Literal['0.1.0']` Version of the underlying index schema. diff --git a/content/integrate/redisvl/api/searchindex.md b/content/integrate/redisvl/api/searchindex.md index 748f485852..cb5a803731 100644 --- a/content/integrate/redisvl/api/searchindex.md +++ b/content/integrate/redisvl/api/searchindex.md @@ -14,7 +14,7 @@ type: integration ## SearchIndex -### `*class* SearchIndex(schema, redis_client=None, redis_url=None, connection_args={}, **kwargs)` +### `class SearchIndex(schema, redis_client=None, redis_url=None, connection_args={}, **kwargs)` A search index class for interacting with Redis as a vector database. @@ -45,11 +45,11 @@ kwargs. * **Parameters:** * **schema** ([*IndexSchema*]({{< relref "schema/#indexschema" >}})) – Index schema object. - * **redis_client** (*Optional* *[**redis.Redis* *]*) – An + * **redis_client** (*Optional* *[* *redis.Redis* *]*) – An instantiated redis client. - * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to + * **redis_url** (*Optional* *[* *str* *]*) – The URL of the Redis server to connect to. - * **connection_args** (*Dict* *[**str* *,* *Any* *]* *,* *optional*) – Redis client connection + * **connection_args** (*Dict* *[* *str* *,* *Any* *]* *,* *optional*) – Redis client connection args. #### `aggregate(*args, **kwargs)` @@ -84,7 +84,7 @@ Note: Additional keyword arguments (\*\*kwargs) can be used to provide extra options specific to the Redis connection. * **Parameters:** - **redis_url** (*Optional* *[**str* *]* *,* *optional*) – The URL of the Redis server to + **redis_url** (*Optional* *[* *str* *]* *,* *optional*) – The URL of the Redis server to connect to. If not provided, the method defaults to using the REDIS_URL environment variable. * **Raises:** @@ -143,7 +143,7 @@ Disconnect from the Redis database. Remove a specific entry or entries from the index by it’s key ID. * **Parameters:** - **keys** (*Union* *[**str* *,* *List* *[**str* *]* *]*) – The document ID or IDs to remove from the index. + **keys** (*Union* *[* *str* *,* *List* *[* *str* *]* *]*) – The document ID or IDs to remove from the index. * **Returns:** Count of records deleted from Redis. * **Return type:** @@ -174,16 +174,16 @@ or derived from some domain-specific metadata combination * **Return type:** Dict[str, Any] -#### `*classmethod* from_dict(schema_dict, **kwargs)` +#### `classmethod from_dict(schema_dict, **kwargs)` Create a SearchIndex from a dictionary. * **Parameters:** - **schema_dict** (*Dict* *[**str* *,* *Any* *]*) – A dictionary containing the schema. + **schema_dict** (*Dict* *[* *str* *,* *Any* *]*) – A dictionary containing the schema. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [SearchIndex](#searchindex) + [searchindex](#searchindex) ```python from redisvl.index import SearchIndex @@ -200,18 +200,18 @@ index = SearchIndex.from_dict({ }) ``` -#### `*classmethod* from_existing(name, redis_client=None, redis_url=None, **kwargs)` +#### `classmethod from_existing(name, redis_client=None, redis_url=None, **kwargs)` Initialize from an existing search index in Redis by index name. * **Parameters:** * **name** (*str*) – Name of the search index in Redis. - * **redis_client** (*Optional* *[**redis.Redis* *]*) – An + * **redis_client** (*Optional* *[* *redis.Redis* *]*) – An instantiated redis client. - * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to + * **redis_url** (*Optional* *[* *str* *]*) – The URL of the Redis server to connect to. -#### `*classmethod* from_yaml(schema_path, **kwargs)` +#### `classmethod from_yaml(schema_path, **kwargs)` Create a SearchIndex from a YAML schema file. @@ -220,7 +220,7 @@ Create a SearchIndex from a YAML schema file. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [SearchIndex](#searchindex) + [searchindex](#searchindex) ```python from redisvl.index import SearchIndex @@ -276,17 +276,17 @@ optional preprocessing steps, and setting optional expiration (TTL policies) on keys. * **Parameters:** - * **data** (*Iterable* *[**Any* *]*) – An iterable of objects to store. - * **id_field** (*Optional* *[**str* *]* *,* *optional*) – Specified field used as the id + * **data** (*Iterable* *[* *Any* *]*) – An iterable of objects to store. + * **id_field** (*Optional* *[* *str* *]* *,* *optional*) – Specified field used as the id portion of the redis key (after the prefix) for each object. Defaults to None. - * **keys** (*Optional* *[**Iterable* *[**str* *]* *]* *,* *optional*) – Optional iterable of keys. + * **keys** (*Optional* *[* *Iterable* *[* *str* *]* *]* *,* *optional*) – Optional iterable of keys. Must match the length of objects if provided. Defaults to None. - * **ttl** (*Optional* *[**int* *]* *,* *optional*) – Time-to-live in seconds for each key. + * **ttl** (*Optional* *[* *int* *]* *,* *optional*) – Time-to-live in seconds for each key. Defaults to None. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – A function to preprocess + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – A function to preprocess objects before storage. Defaults to None. - * **batch_size** (*Optional* *[**int* *]* *,* *optional*) – Number of objects to write in + * **batch_size** (*Optional* *[* *int* *]* *,* *optional*) – Number of objects to write in a single Redis pipeline execution. Defaults to class’s default batch size. * **Returns:** @@ -411,25 +411,25 @@ index = SearchIndex.from_yaml("schemas/schema.yaml") index.set_client(client) ``` -#### `*property* client *: Redis | None*` +#### `property client: Redis | None` The underlying redis-py client object. -#### `*property* key_separator *: str*` +#### `property key_separator: str` The optional separator between a defined prefix and key value in forming a Redis key. -#### `*property* name *: str*` +#### `property name: str` The name of the Redis search index. -#### `*property* prefix *: str*` +#### `property prefix: str` The optional key prefix that comes before a unique key value in forming a Redis key. -#### `*property* storage_type *: StorageType*` +#### `property storage_type: StorageType` The underlying storage type for the search index; either hash or json. @@ -438,7 +438,7 @@ hash or json. ## AsyncSearchIndex -### `*class* AsyncSearchIndex(schema, **kwargs)` +### `class AsyncSearchIndex(schema, **kwargs)` A search index class for interacting with Redis as a vector database in async-mode. @@ -468,10 +468,10 @@ Initialize the RedisVL async search index with a schema. * **Parameters:** * **schema** ([*IndexSchema*]({{< relref "schema/#indexschema" >}})) – Index schema object. - * **connection_args** (*Dict* *[**str* *,* *Any* *]* *,* *optional*) – Redis client connection + * **connection_args** (*Dict* *[* *str* *,* *Any* *]* *,* *optional*) – Redis client connection args. -#### `*async* aggregate(*args, **kwargs)` +#### `async aggregate(*args, **kwargs)` Perform an aggregation operation against the index. @@ -484,7 +484,7 @@ to the redis-py ft().aggregate() method. * **Return type:** Result -#### `*async* clear()` +#### `async clear()` Clear all keys in Redis associated with the index, leaving the index available and in-place for future insertions or updates. @@ -494,7 +494,7 @@ available and in-place for future insertions or updates. * **Return type:** int -#### `*async* connect(redis_url=None, **kwargs)` +#### `async connect(redis_url=None, **kwargs)` Connect to a Redis instance using the provided redis_url, falling back to the REDIS_URL environment variable (if available). @@ -503,7 +503,7 @@ Note: Additional keyword arguments (\*\*kwargs) can be used to provide extra options specific to the Redis connection. * **Parameters:** - **redis_url** (*Optional* *[**str* *]* *,* *optional*) – The URL of the Redis server to + **redis_url** (*Optional* *[* *str* *]* *,* *optional*) – The URL of the Redis server to connect to. If not provided, the method defaults to using the REDIS_URL environment variable. * **Raises:** @@ -516,7 +516,7 @@ extra options specific to the Redis connection. index.connect(redis_url="redis://localhost:6379") ``` -#### `*async* create(overwrite=False, drop=False)` +#### `async create(overwrite=False, drop=False)` Asynchronously create an index in Redis with the current schema : and properties. @@ -543,7 +543,7 @@ await index.create(overwrite=True) await index.create(overwrite=True, drop=True) ``` -#### `*async* delete(drop=True)` +#### `async delete(drop=True)` Delete the search index. @@ -557,18 +557,18 @@ Delete the search index. Disconnect and cleanup the underlying async redis connection. -#### `*async* drop_keys(keys)` +#### `async drop_keys(keys)` Remove a specific entry or entries from the index by it’s key ID. * **Parameters:** - **keys** (*Union* *[**str* *,* *List* *[**str* *]* *]*) – The document ID or IDs to remove from the index. + **keys** (*Union* *[* *str* *,* *List* *[* *str* *]* *]*) – The document ID or IDs to remove from the index. * **Returns:** Count of records deleted from Redis. * **Return type:** int -#### `*async* exists()` +#### `async exists()` Check if the index exists in Redis. @@ -577,7 +577,7 @@ Check if the index exists in Redis. * **Return type:** bool -#### `*async* fetch(id)` +#### `async fetch(id)` Asynchronously etch an object from Redis by id. The id is typically either a unique identifier, or derived from some domain-specific @@ -591,16 +591,16 @@ metadata combination (like a document id or chunk id). * **Return type:** Dict[str, Any] -#### `*classmethod* from_dict(schema_dict, **kwargs)` +#### `classmethod from_dict(schema_dict, **kwargs)` Create a SearchIndex from a dictionary. * **Parameters:** - **schema_dict** (*Dict* *[**str* *,* *Any* *]*) – A dictionary containing the schema. + **schema_dict** (*Dict* *[* *str* *,* *Any* *]*) – A dictionary containing the schema. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [SearchIndex](#searchindex) + [searchindex](#searchindex) ```python from redisvl.index import SearchIndex @@ -617,18 +617,18 @@ index = SearchIndex.from_dict({ }) ``` -#### `*async classmethod* from_existing(name, redis_client=None, redis_url=None, **kwargs)` +#### `async classmethod* from_existing(name, redis_client=None, redis_url=None, **kwargs)` Initialize from an existing search index in Redis by index name. * **Parameters:** * **name** (*str*) – Name of the search index in Redis. - * **redis_client** (*Optional* *[**redis.Redis* *]*) – An + * **redis_client** (*Optional* *[* *redis.Redis* *]*) – An instantiated redis client. - * **redis_url** (*Optional* *[**str* *]*) – The URL of the Redis server to + * **redis_url** (*Optional* *[* *str* *]*) – The URL of the Redis server to connect to. -#### `*classmethod* from_yaml(schema_path, **kwargs)` +#### `classmethod from_yaml(schema_path, **kwargs)` Create a SearchIndex from a YAML schema file. @@ -637,7 +637,7 @@ Create a SearchIndex from a YAML schema file. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [SearchIndex](#searchindex) + [searchindex](#searchindex) ```python from redisvl.index import SearchIndex @@ -645,7 +645,7 @@ from redisvl.index import SearchIndex index = SearchIndex.from_yaml("schemas/schema.yaml") ``` -#### `*async* info(name=None)` +#### `async info(name=None)` Get information about the index. @@ -674,7 +674,7 @@ id or chunk id). * **Return type:** str -#### `*async* listall()` +#### `async listall()` List all search indices in Redis database. @@ -683,7 +683,7 @@ List all search indices in Redis database. * **Return type:** List[str] -#### `*async* load(data, id_field=None, keys=None, ttl=None, preprocess=None, concurrency=None)` +#### `async load(data, id_field=None, keys=None, ttl=None, preprocess=None, concurrency=None)` Asynchronously load objects to Redis with concurrency control. Returns the list of keys loaded to Redis. @@ -693,17 +693,17 @@ optional preprocessing steps, and setting optional expiration (TTL policies) on keys. * **Parameters:** - * **data** (*Iterable* *[**Any* *]*) – An iterable of objects to store. - * **id_field** (*Optional* *[**str* *]* *,* *optional*) – Specified field used as the id + * **data** (*Iterable* *[* *Any* *]*) – An iterable of objects to store. + * **id_field** (*Optional* *[* *str* *]* *,* *optional*) – Specified field used as the id portion of the redis key (after the prefix) for each object. Defaults to None. - * **keys** (*Optional* *[**Iterable* *[**str* *]* *]* *,* *optional*) – Optional iterable of keys. + * **keys** (*Optional* *[* *Iterable* *[* *str* *]* *]* *,* *optional*) – Optional iterable of keys. Must match the length of objects if provided. Defaults to None. - * **ttl** (*Optional* *[**int* *]* *,* *optional*) – Time-to-live in seconds for each key. + * **ttl** (*Optional* *[* *int* *]* *,* *optional*) – Time-to-live in seconds for each key. Defaults to None. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – An async function to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – An async function to preprocess objects before storage. Defaults to None. - * **concurrency** (*Optional* *[**int* *]* *,* *optional*) – The maximum number of + * **concurrency** (*Optional* *[* *int* *]* *,* *optional*) – The maximum number of concurrent write operations. Defaults to class’s default concurrency level. * **Returns:** @@ -733,7 +733,7 @@ async def add_field(d): keys = await index.load(data, preprocess=add_field) ``` -#### `*async* paginate(query, page_size=30)` +#### `async paginate(query, page_size=30)` Execute a given query against the index and return results in paginated batches. @@ -766,7 +766,7 @@ The page_size parameter controls the number of items each result batch contains. Adjust this value based on performance considerations and the expected volume of search results. -#### `*async* query(query)` +#### `async query(query)` Asynchronously execute a query on the index. @@ -792,7 +792,7 @@ query = VectorQuery( results = await index.query(query) ``` -#### `*async* search(*args, **kwargs)` +#### `async search(*args, **kwargs)` Perform a search on this index. @@ -805,7 +805,7 @@ to the redis-py ft.search() method. * **Return type:** Result -#### `*async* set_client(redis_client)` +#### `async set_client(redis_client)` Manually set the Redis client to use with the search index. @@ -829,25 +829,25 @@ index = AsyncSearchIndex.from_yaml("schemas/schema.yaml") await index.set_client(client) ``` -#### `*property* client *: Redis | None*` +#### `property client: Redis | None` The underlying redis-py client object. -#### `*property* key_separator *: str*` +#### `property key_separator: str` The optional separator between a defined prefix and key value in forming a Redis key. -#### `*property* name *: str*` +#### `property name: str` The name of the Redis search index. -#### `*property* prefix *: str*` +#### `property prefix: str` The optional key prefix that comes before a unique key value in forming a Redis key. -#### `*property* storage_type *: StorageType*` +#### `property storage_type: StorageType` The underlying storage type for the search index; either hash or json. diff --git a/content/integrate/redisvl/api/session_manager.md b/content/integrate/redisvl/api/session_manager.md index 37cd7051f5..02487b5110 100644 --- a/content/integrate/redisvl/api/session_manager.md +++ b/content/integrate/redisvl/api/session_manager.md @@ -9,7 +9,7 @@ type: integration -### `*class* SemanticSessionManager(name, session_tag=None, prefix=None, vectorizer=None, distance_threshold=0.3, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, **kwargs)` +### `class SemanticSessionManager(name, session_tag=None, prefix=None, vectorizer=None, distance_threshold=0.3, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, **kwargs)` Bases: `BaseSessionManager` @@ -22,17 +22,17 @@ responses. * **Parameters:** * **name** (*str*) – The name of the session manager index. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. - * **prefix** (*Optional* *[**str* *]*) – Prefix for the keys for this session data. + * **prefix** (*Optional* *[* *str* *]*) – Prefix for the keys for this session data. Defaults to None and will be replaced with the index name. - * **vectorizer** (*Optional* *[**BaseVectorizer* *]*) – The vectorizer used to create embeddings. + * **vectorizer** (*Optional* *[* *BaseVectorizer* *]*) – The vectorizer used to create embeddings. * **distance_threshold** (*float*) – The maximum semantic distance to be included in the context. Defaults to 0.3. - * **redis_client** (*Optional* *[**Redis* *]*) – A Redis client instance. Defaults to + * **redis_client** (*Optional* *[* *Redis* *]*) – A Redis client instance. Defaults to None. * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. - * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments + * **connection_kwargs** (*Dict* *[* *str* *,* *Any* *]*) – The connection arguments for the redis client. Defaults to empty {}. * **overwrite** (*bool*) – Whether or not to force overwrite the schema for the semantic session index. Defaults to false. @@ -47,8 +47,8 @@ A timestamp is associated with it so that it can be later sorted in sequential ordering after retrieval. * **Parameters:** - * **message** (*Dict* *[**str* *,**str* *]*) – The user prompt or LLM response. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **message** (*Dict* *[* *str* *,**str* *]*) – The user prompt or LLM response. + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. * **Return type:** None @@ -60,8 +60,8 @@ A timestamp is associated with each so that they can be later sorted in sequential ordering after retrieval. * **Parameters:** - * **messages** (*List* *[**Dict* *[**str* *,* *str* *]* *]*) – The list of user prompts and LLM responses. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **messages** (*List* *[* *Dict* *[* *str* *,* *str* *]* *]*) – The list of user prompts and LLM responses. + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. * **Return type:** None @@ -85,7 +85,7 @@ Clear all conversation keys and remove the search index. Remove a specific exchange from the conversation history. * **Parameters:** - **id** (*Optional* *[**str* *]*) – The id of the session entry to delete. + **id** (*Optional* *[* *str* *]*) – The id of the session entry to delete. If None then the last entry is deleted. * **Return type:** None @@ -100,7 +100,7 @@ Retreive the recent conversation history in sequential order. or list of alternating prompts and responses. * **raw** (*bool*) – Whether to return the full Redis hash entry or just the prompt and response - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. * **Returns:** A single string transcription of the session @@ -125,9 +125,9 @@ context to the next LLM call. * **as_text** (*bool*) – Whether to return the prompts and responses as text * **JSON** (*or as*) * **top_k** (*int*) – The number of previous messages to return. Default is 5. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. - * **distance_threshold** (*Optional* *[**float* *]*) – The threshold for semantic + * **distance_threshold** (*Optional* *[* *float* *]*) – The threshold for semantic vector distance. * **fall_back** (*bool*) – Whether to drop back to recent conversation history if no relevant context is found. @@ -150,12 +150,12 @@ in sequential ordering after retrieval. * **Parameters:** * **prompt** (*str*) – The user prompt to the LLM. * **response** (*str*) – The corresponding LLM response. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. * **Return type:** None -#### `*property* messages *: List[str] | List[Dict[str, str]]*` +#### `property messages: List[str] | List[Dict[str, str]]` Returns the full chat history. @@ -163,7 +163,7 @@ Returns the full chat history. -### `*class* StandardSessionManager(name, session_tag=None, prefix=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, **kwargs)` +### `class StandardSessionManager(name, session_tag=None, prefix=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, **kwargs)` Bases: `BaseSessionManager` @@ -176,14 +176,14 @@ responses. * **Parameters:** * **name** (*str*) – The name of the session manager index. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. - * **prefix** (*Optional* *[**str* *]*) – Prefix for the keys for this session data. + * **prefix** (*Optional* *[* *str* *]*) – Prefix for the keys for this session data. Defaults to None and will be replaced with the index name. - * **redis_client** (*Optional* *[**Redis* *]*) – A Redis client instance. Defaults to + * **redis_client** (*Optional* *[* *Redis* *]*) – A Redis client instance. Defaults to None. * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. - * **connection_kwargs** (*Dict* *[**str* *,* *Any* *]*) – The connection arguments + * **connection_kwargs** (*Dict* *[* *str* *,* *Any* *]*) – The connection arguments for the redis client. Defaults to empty {}. The proposed schema will support a single combined vector embedding @@ -196,8 +196,8 @@ A timestamp is associated with it so that it can be later sorted in sequential ordering after retrieval. * **Parameters:** - * **message** (*Dict* *[**str* *,**str* *]*) – The user prompt or LLM response. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **message** (*Dict* *[* *str* *,**str* *]*) – The user prompt or LLM response. + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. * **Return type:** None @@ -209,8 +209,8 @@ A timestamp is associated with each so that they can be later sorted in sequential ordering after retrieval. * **Parameters:** - * **messages** (*List* *[**Dict* *[**str* *,* *str* *]* *]*) – The list of user prompts and LLM responses. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **messages** (*List* *[* *Dict* *[* *str* *,* *str* *]* *]*) – The list of user prompts and LLM responses. + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. * **Return type:** None @@ -234,7 +234,7 @@ Clear all conversation keys and remove the search index. Remove a specific exchange from the conversation history. * **Parameters:** - **id** (*Optional* *[**str* *]*) – The id of the session entry to delete. + **id** (*Optional* *[* *str* *]*) – The id of the session entry to delete. If None then the last entry is deleted. * **Return type:** None @@ -249,7 +249,7 @@ Retrieve the recent conversation history in sequential order. or list of alternating prompts and responses. * **raw** (*bool*) – Whether to return the full Redis hash entry or just the prompt and response - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. * **Returns:** A single string transcription of the session @@ -268,11 +268,11 @@ in sequential ordering after retrieval. * **Parameters:** * **prompt** (*str*) – The user prompt to the LLM. * **response** (*str*) – The corresponding LLM response. - * **session_tag** (*Optional* *[**str* *]*) – Tag to be added to entries to link to a specific + * **session_tag** (*Optional* *[* *str* *]*) – Tag to be added to entries to link to a specific session. Defaults to instance ULID. * **Return type:** None -#### `*property* messages *: List[str] | List[Dict[str, str]]*` +#### `property messages: List[str] | List[Dict[str, str]]` Returns the full chat history. diff --git a/content/integrate/redisvl/api/vectorizer.md b/content/integrate/redisvl/api/vectorizer.md index 4ebb444a67..cba8205b1a 100644 --- a/content/integrate/redisvl/api/vectorizer.md +++ b/content/integrate/redisvl/api/vectorizer.md @@ -9,7 +9,7 @@ type: integration -### `*class* HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2', dtype='float32', *, dims=None)` +### `class HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2', dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -56,7 +56,7 @@ Embed a chunk of text using the Hugging Face sentence transformer. * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -73,8 +73,8 @@ Asynchronously embed many chunks of texts using the Hugging Face sentence transformer. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. Defaults to 10. @@ -99,7 +99,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -107,7 +107,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* OpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', *, dims=None)` +### `class OpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -145,7 +145,7 @@ Initialize the OpenAI vectorizer. * **Parameters:** * **model** (*str*) – Model to use for embedding. Defaults to ‘text-embedding-ada-002’. - * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the + * **api_config** (*Optional* *[* *Dict* *]* *,* *optional*) – Dictionary containing the API key and any additional OpenAI API options. Defaults to None. * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. Used when setting as_buffer=True in calls to embed() and embed_many(). @@ -162,7 +162,7 @@ Asynchronously embed a chunk of text using the OpenAI API. * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -178,8 +178,8 @@ Asynchronously embed a chunk of text using the OpenAI API. Asynchronously embed many chunks of texts using the OpenAI API. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. Defaults to 10. @@ -198,7 +198,7 @@ Embed a chunk of text using the OpenAI API. * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -214,8 +214,8 @@ Embed a chunk of text using the OpenAI API. Embed many chunks of texts using the OpenAI API. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. Defaults to 10. @@ -240,7 +240,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -248,7 +248,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* AzureOpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', *, dims=None)` +### `class AzureOpenAITextVectorizer(model='text-embedding-ada-002', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -292,7 +292,7 @@ Initialize the AzureOpenAI vectorizer. * **model** (*str*) – Deployment to use for embedding. Must be the ‘Deployment name’ not the ‘Model name’. Defaults to ‘text-embedding-ada-002’. - * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the + * **api_config** (*Optional* *[* *Dict* *]* *,* *optional*) – Dictionary containing the API key, API version, Azure endpoint, and any other API options. Defaults to None. * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. @@ -310,7 +310,7 @@ Asynchronously embed a chunk of text using the OpenAI API. * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -326,8 +326,8 @@ Asynchronously embed a chunk of text using the OpenAI API. Asynchronously embed many chunks of texts using the AzureOpenAI API. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. Defaults to 10. @@ -346,7 +346,7 @@ Embed a chunk of text using the AzureOpenAI API. * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -362,8 +362,8 @@ Embed a chunk of text using the AzureOpenAI API. Embed many chunks of texts using the AzureOpenAI API. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. Defaults to 10. @@ -388,7 +388,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -396,7 +396,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* VertexAITextVectorizer(model='textembedding-gecko', api_config=None, dtype='float32', *, dims=None)` +### `class VertexAITextVectorizer(model='textembedding-gecko', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -435,7 +435,7 @@ Initialize the VertexAI vectorizer. * **Parameters:** * **model** (*str*) – Model to use for embedding. Defaults to ‘textembedding-gecko’. - * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the + * **api_config** (*Optional* *[* *Dict* *]* *,* *optional*) – Dictionary containing the API config details. Defaults to None. * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. Used when setting as_buffer=True in calls to embed() and embed_many(). @@ -452,7 +452,7 @@ Embed a chunk of text using the VertexAI API. * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -468,8 +468,8 @@ Embed a chunk of text using the VertexAI API. Embed many chunks of texts using the VertexAI API. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. Defaults to 10. @@ -494,7 +494,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -502,7 +502,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* CohereTextVectorizer(model='embed-english-v3.0', api_config=None, dtype='float32', *, dims=None)` +### `class CohereTextVectorizer(model='embed-english-v3.0', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -542,7 +542,7 @@ Visit [https://cohere.ai/embed](https://cohere.ai/embed) to learn about embeddin * **Parameters:** * **model** (*str*) – Model to use for embedding. Defaults to ‘embed-english-v3.0’. - * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the API key. + * **api_config** (*Optional* *[* *Dict* *]* *,* *optional*) – Dictionary containing the API key. Defaults to None. * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. Used when setting as_buffer=True in calls to embed() and embed_many(). @@ -575,7 +575,7 @@ task downstream, you should set input_type= “classification” or * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -609,8 +609,8 @@ task downstream, you should set input_type= “classification” or “clustering”. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. Defaults to 10. @@ -637,7 +637,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -645,7 +645,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* BedrockTextVectorizer(model='amazon.titan-embed-text-v2:0', api_config=None, dtype='float32', *, dims=None)` +### `class BedrockTextVectorizer(model='amazon.titan-embed-text-v2:0', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -685,7 +685,7 @@ Initialize the AWS Bedrock Vectorizer. * **Parameters:** * **model** (*str*) – The Bedrock model ID to use. Defaults to amazon.titan-embed-text-v2:0 - * **api_config** (*Optional* *[**Dict* *[**str* *,* *str* *]* *]*) – AWS credentials and config. + * **api_config** (*Optional* *[* *Dict* *[* *str* *,* *str* *]* *]*) – AWS credentials and config. Can include: aws_access_key_id, aws_secret_access_key, aws_region If not provided, will use environment variables. * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. @@ -703,7 +703,7 @@ Embed a chunk of text using Amazon Bedrock. * **Parameters:** * **text** (*str*) – Text to embed. - * **preprocess** (*Optional* *[**Callable* *]*) – Optional preprocessing function. + * **preprocess** (*Optional* *[* *Callable* *]*) – Optional preprocessing function. * **as_buffer** (*bool*) – Whether to return as byte buffer. * **Returns:** The embedding vector. @@ -717,8 +717,8 @@ Embed a chunk of text using Amazon Bedrock. Embed multiple texts using Amazon Bedrock. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of texts to embed. - * **preprocess** (*Optional* *[**Callable* *]*) – Optional preprocessing function. + * **texts** (*List* *[* *str* *]*) – List of texts to embed. + * **preprocess** (*Optional* *[* *Callable* *]*) – Optional preprocessing function. * **batch_size** (*int*) – Size of batches for processing. * **as_buffer** (*bool*) – Whether to return as byte buffers. * **Returns:** @@ -740,7 +740,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -748,7 +748,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* CustomTextVectorizer(embed, embed_many=None, aembed=None, aembed_many=None, dtype='float32')` +### `class CustomTextVectorizer(embed, embed_many=None, aembed=None, aembed_many=None, dtype='float32')` Bases: `BaseVectorizer` @@ -779,9 +779,9 @@ Initialize the Custom vectorizer. * **Parameters:** * **embed** (*Callable*) – a Callable function that accepts a string object and returns a list of floats. - * **embed_many** (*Optional* *[**Callable*) – a Callable function that accepts a list of string objects and returns a list containing lists of floats. Defaults to None. - * **aembed** (*Optional* *[**Callable* *]*) – an asyncronous Callable function that accepts a string object and returns a lists of floats. Defaults to None. - * **aembed_many** (*Optional* *[**Callable* *]*) – an asyncronous Callable function that accepts a list of string objects and returns a list containing lists of floats. Defaults to None. + * **embed_many** (*Optional* *[* *Callable*) – a Callable function that accepts a list of string objects and returns a list containing lists of floats. Defaults to None. + * **aembed** (*Optional* *[* *Callable* *]*) – an asyncronous Callable function that accepts a string object and returns a lists of floats. Defaults to None. + * **aembed_many** (*Optional* *[* *Callable* *]*) – an asyncronous Callable function that accepts a list of string objects and returns a list containing lists of floats. Defaults to None. * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. Used when setting as_buffer=True in calls to embed() and embed_many(). Defaults to ‘float32’. @@ -794,7 +794,7 @@ Generate an embedding for a single piece of text using your sync embed function. * **Parameters:** * **text** (*str*) – The text to embed. - * **preprocess** (*Optional* *[**Callable* *]*) – An optional callable to preprocess the text. + * **preprocess** (*Optional* *[* *Callable* *]*) – An optional callable to preprocess the text. * **as_buffer** (*bool*) – If True, return the embedding as a byte buffer. * **Returns:** The embedding of the input text. @@ -808,8 +808,8 @@ Generate an embedding for a single piece of text using your sync embed function. Generate embeddings for multiple pieces of text in batches using your sync embed_many function. * **Parameters:** - * **texts** (*List* *[**str* *]*) – A list of texts to embed. - * **preprocess** (*Optional* *[**Callable* *]*) – Optional preprocessing for each text. + * **texts** (*List* *[* *str* *]*) – A list of texts to embed. + * **preprocess** (*Optional* *[* *Callable* *]*) – Optional preprocessing for each text. * **batch_size** (*int*) – Number of texts per batch. * **as_buffer** (*bool*) – If True, convert each embedding to a byte buffer. * **Returns:** @@ -832,7 +832,7 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. @@ -840,7 +840,7 @@ Configuration for the model, should be a dictionary conforming to [ConfigDict][p -### `*class* VoyageAITextVectorizer(model='voyage-large-2', api_config=None, dtype='float32', *, dims=None)` +### `class VoyageAITextVectorizer(model='voyage-large-2', api_config=None, dtype='float32', *, dims=None)` Bases: `BaseVectorizer` @@ -880,7 +880,7 @@ Visit [https://docs.voyageai.com/docs/embeddings](https://docs.voyageai.com/docs * **Parameters:** * **model** (*str*) – Model to use for embedding. Defaults to “voyage-large-2”. - * **api_config** (*Optional* *[**Dict* *]* *,* *optional*) – Dictionary containing the API key. + * **api_config** (*Optional* *[* *Dict* *]* *,* *optional*) – Dictionary containing the API key. Defaults to None. * **dtype** (*str*) – the default datatype to use when embedding text as byte arrays. Used when setting as_buffer=True in calls to embed() and embed_many(). @@ -907,7 +907,7 @@ querying the database, you should set the input_type=”query”. * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -937,8 +937,8 @@ should be embedded with input_type=”document” and when you are querying the database, you should set the input_type=”query”. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. . @@ -971,7 +971,7 @@ querying the database, you should set the input_type=”query”. * **Parameters:** * **text** (*str*) – Chunk of text to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **as_buffer** (*bool* *,* *optional*) – Whether to convert the raw embedding to a byte string. Defaults to False. @@ -1001,8 +1001,8 @@ should be embedded with input_type=”document” and when you are querying the database, you should set the input_type=”query”. * **Parameters:** - * **texts** (*List* *[**str* *]*) – List of text chunks to embed. - * **preprocess** (*Optional* *[**Callable* *]* *,* *optional*) – Optional preprocessing callable to + * **texts** (*List* *[* *str* *]*) – List of text chunks to embed. + * **preprocess** (*Optional* *[* *Callable* *]* *,* *optional*) – Optional preprocessing callable to perform before vectorization. Defaults to None. * **batch_size** (*int* *,* *optional*) – Batch size of texts to use when creating embeddings. . @@ -1030,6 +1030,6 @@ It takes context as an argument since that’s what pydantic-core passes when ca * **Return type:** None -#### `model_config *: ClassVar[ConfigDict]* *= {}*` +#### `model_config: ClassVar[ConfigDict] = {}` Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict]. From 7534258c52f64b69be7786ec9ae6fe075a8eee50 Mon Sep 17 00:00:00 2001 From: paoloredis Date: Mon, 24 Mar 2025 10:42:50 +0100 Subject: [PATCH 06/13] Fix camel case in rel refs --- content/integrate/redisvl/api/filter.md | 34 ++++++++++---------- content/integrate/redisvl/api/router.md | 22 ++++++------- content/integrate/redisvl/api/schema.md | 4 +-- content/integrate/redisvl/api/searchindex.md | 8 ++--- 4 files changed, 34 insertions(+), 34 deletions(-) diff --git a/content/integrate/redisvl/api/filter.md b/content/integrate/redisvl/api/filter.md index c4e6f80885..8aade64a79 100644 --- a/content/integrate/redisvl/api/filter.md +++ b/content/integrate/redisvl/api/filter.md @@ -51,8 +51,8 @@ built by combining filter statements using the & and | operators. * **Parameters:** * **\_filter** (*str* *|* *None*) * **operator** (*FilterOperator* *|* *None*) - * **left** ([filterexpression](#filterexpression) *|* *None*) - * **right** ([filterexpression](#filterexpression) *|* *None*) + * **left** ([FilterExpression](#filterexpression) *|* *None*) + * **right** ([FilterExpression](#filterexpression) *|* *None*) ## Tag @@ -70,7 +70,7 @@ Create a Tag equality filter expression. * **Parameters:** **other** (*Union* *[* *List* *[* *str* *]* *,* *str* *]*) – The tag(s) to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Tag @@ -85,7 +85,7 @@ Create a Tag inequality filter expression. * **Parameters:** **other** (*Union* *[* *List* *[* *str* *]* *,* *str* *]*) – The tag(s) to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Tag @@ -116,7 +116,7 @@ filters that enforce an exact match on the supplied term(s). * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Text @@ -134,7 +134,7 @@ term(s). * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Text @@ -154,7 +154,7 @@ equality filter expression. * **Parameters:** **other** (*str*) – The text value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Text @@ -185,7 +185,7 @@ Create a Numeric equality filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -199,7 +199,7 @@ Create a Numeric greater than or equal to filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -214,7 +214,7 @@ Create a Numeric greater than filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -229,7 +229,7 @@ Create a Numeric less than or equal to filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -244,7 +244,7 @@ Create a Numeric less than filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -259,7 +259,7 @@ Create a Numeric inequality filter expression. * **Parameters:** **other** (*int*) – The value to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Num @@ -289,9 +289,9 @@ Redis index. Create a geographic filter within a specified GeoRadius. * **Parameters:** - **other** ([georadius](#georadius)) – The geographic spec to filter on. + **other** ([GeoRadius](#georadius)) – The geographic spec to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Geo, GeoRadius @@ -304,9 +304,9 @@ f = Geo("location") == GeoRadius(-122.4194, 37.7749, 1, unit="m") Create a geographic filter outside of a specified GeoRadius. * **Parameters:** - **other** ([georadius](#georadius)) – The geographic spec to filter on. + **other** ([GeoRadius](#georadius)) – The geographic spec to filter on. * **Return type:** - [filterexpression](#filterexpression) + [FilterExpression](#filterexpression) ```python from redisvl.query.filter import Geo, GeoRadius diff --git a/content/integrate/redisvl/api/router.md b/content/integrate/redisvl/api/router.md index cdd46fb7ea..fc938ba543 100644 --- a/content/integrate/redisvl/api/router.md +++ b/content/integrate/redisvl/api/router.md @@ -17,9 +17,9 @@ Initialize the SemanticRouter. * **Parameters:** * **name** (*str*) – The name of the semantic router. - * **routes** (*List* *[*[route](#route) *]*) – List of Route objects. + * **routes** (*List* *[*[Route](#route) *]*) – List of Route objects. * **vectorizer** (*BaseVectorizer* *,* *optional*) – The vectorizer used to embed route references. Defaults to default HFTextVectorizer. - * **routing_config** ([routingconfig](#routingconfig) *,* *optional*) – Configuration for routing behavior. Defaults to the default RoutingConfig. + * **routing_config** ([RoutingConfig](#routingconfig) *,* *optional*) – Configuration for routing behavior. Defaults to the default RoutingConfig. * **redis_client** (*Optional* *[* *Redis* *]* *,* *optional*) – Redis client for connection. Defaults to None. * **redis_url** (*str* *,* *optional*) – The redis url. Defaults to redis://localhost:6379. * **overwrite** (*bool* *,* *optional*) – Whether to overwrite existing index. Defaults to False. @@ -49,7 +49,7 @@ Create a SemanticRouter from a dictionary. * **Returns:** The semantic router instance. * **Return type:** - [semanticrouter](#semanticrouter) + [SemanticRouter](#semanticrouter) * **Raises:** **ValueError** – If required data is missing or invalid. @@ -72,7 +72,7 @@ Create a SemanticRouter from a YAML file. * **Returns:** The semantic router instance. * **Return type:** - [semanticrouter](#semanticrouter) + [SemanticRouter](#semanticrouter) * **Raises:** * **ValueError** – If the file path is invalid. * **FileNotFoundError** – If the file does not exist. @@ -91,7 +91,7 @@ Get a route by its name. * **Returns:** The selected Route object or None if not found. * **Return type:** - Optional[[route](#route)] + Optional[[Route](#route)] #### `model_post_init(context, /)` @@ -123,11 +123,11 @@ Query the semantic router with a given statement or vector for multiple matches. * **vector** (*Optional* *[* *List* *[* *float* *]* *]*) – The input vector to be queried. * **max_k** (*Optional* *[* *int* *]*) – The maximum number of top matches to return. * **distance_threshold** (*Optional* *[* *float* *]*) – The threshold for semantic distance. - * **aggregation_method** (*Optional* *[*[distanceaggregationmethod](#distanceaggregationmethod) *]*) – The aggregation method used for vector distances. + * **aggregation_method** (*Optional* *[*[DistanceAggregationMethod](#distanceaggregationmethod) *]*) – The aggregation method used for vector distances. * **Returns:** The matching routes and their details. * **Return type:** - List[[routematch](#routematch)] + List[[RouteMatch](#routematch)] #### `to_dict()` @@ -171,7 +171,7 @@ router.to_yaml("router.yaml") Update the routing configuration. * **Parameters:** - **routing_config** ([routingconfig](#routingconfig)) – The new routing configuration. + **routing_config** ([RoutingConfig](#routingconfig)) – The new routing configuration. #### `model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True}` @@ -199,11 +199,11 @@ Get the distance thresholds for each route. * **Return type:** Dict[str, float] -#### `routes: `List[[route](#route)] +#### `routes: `List[[Route](#route)] List of Route objects. -#### `routing_config: `[routingconfig](#routingconfig) +#### `routing_config: `[RoutingConfig](#routingconfig) Configuration for routing behavior. @@ -226,7 +226,7 @@ self is explicitly positional-only to allow self as a field name. * **Parameters:** * **max_k** (*Annotated* *[* *int* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=False* *,* *default=1* *,* *metadata=* *[* *Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *]* *)* *]*) - * **aggregation_method** ([distanceaggregationmethod](#distanceaggregationmethod)) + * **aggregation_method** ([DistanceAggregationMethod](#distanceaggregationmethod)) #### `max_k: Annotated[int, FieldInfo(annotation=NoneType, required=False, default=1, metadata=[Strict(strict=True), Gt(gt=0)])]` diff --git a/content/integrate/redisvl/api/schema.md b/content/integrate/redisvl/api/schema.md index 3060e2170e..1e26cb69ec 100644 --- a/content/integrate/redisvl/api/schema.md +++ b/content/integrate/redisvl/api/schema.md @@ -173,7 +173,7 @@ Create an IndexSchema from a dictionary. * **Returns:** The index schema. * **Return type:** - [indexschema](#indexschema) + [IndexSchema](#indexschema) ```python from redisvl.schema import IndexSchema @@ -210,7 +210,7 @@ Create an IndexSchema from a YAML file. * **Returns:** The index schema. * **Return type:** - [indexschema](#indexschema) + [IndexSchema](#indexschema) ```python from redisvl.schema import IndexSchema diff --git a/content/integrate/redisvl/api/searchindex.md b/content/integrate/redisvl/api/searchindex.md index cb5a803731..faeaa23167 100644 --- a/content/integrate/redisvl/api/searchindex.md +++ b/content/integrate/redisvl/api/searchindex.md @@ -183,7 +183,7 @@ Create a SearchIndex from a dictionary. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [searchindex](#searchindex) + [SearchIndex](#searchindex) ```python from redisvl.index import SearchIndex @@ -220,7 +220,7 @@ Create a SearchIndex from a YAML schema file. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [searchindex](#searchindex) + [SearchIndex](#searchindex) ```python from redisvl.index import SearchIndex @@ -600,7 +600,7 @@ Create a SearchIndex from a dictionary. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [searchindex](#searchindex) + [SearchIndex](#searchindex) ```python from redisvl.index import SearchIndex @@ -637,7 +637,7 @@ Create a SearchIndex from a YAML schema file. * **Returns:** A RedisVL SearchIndex object. * **Return type:** - [searchindex](#searchindex) + [SearchIndex](#searchindex) ```python from redisvl.index import SearchIndex From 87ce2f0baaffa25da16ad41fede7fc3076864cb2 Mon Sep 17 00:00:00 2001 From: paoloredis Date: Mon, 24 Mar 2025 11:26:49 +0100 Subject: [PATCH 07/13] Fix parameters formatting --- content/integrate/redisvl/api/router.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/integrate/redisvl/api/router.md b/content/integrate/redisvl/api/router.md index fc938ba543..aefb38a877 100644 --- a/content/integrate/redisvl/api/router.md +++ b/content/integrate/redisvl/api/router.md @@ -225,7 +225,7 @@ validated to form a valid model. self is explicitly positional-only to allow self as a field name. * **Parameters:** - * **max_k** (*Annotated* *[* *int* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=False* *,* *default=1* *,* *metadata=* *[* *Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *]* *)* *]*) + * **max_k** (*Annotated* *[* *int* *,* *FieldInfo* *(* *annotation=NoneType* *,* *required=False* *,* *default=1* *,* *metadata=* *[* *Strict* *(* *strict=True* *)* *,* *Gt* *(* *gt=0* *)* *]* *)* *]*) * **aggregation_method** ([DistanceAggregationMethod](#distanceaggregationmethod)) #### `max_k: Annotated[int, FieldInfo(annotation=NoneType, required=False, default=1, metadata=[Strict(strict=True), Gt(gt=0)])]` @@ -253,7 +253,7 @@ self is explicitly positional-only to allow self as a field name. * **name** (*str*) * **references** (*List* *[* *str* *]*) * **metadata** (*Dict* *[* *str* *,* *Any* *]*) - * **distance_threshold** (*Annotated* *[* *float* *,* *FieldInfo* *(**annotation=NoneType* *,* *required=True* *,* *metadata=* *[* *Strict* *(**strict=True* *)* *,* *Gt* *(**gt=0* *)* *,* *Le* *(**le=1* *)* *]* *)* *]*) + * **distance_threshold** (*Annotated* *[* *float* *,* *FieldInfo* *(* *annotation=NoneType* *,* *required=True* *,* *metadata=* *[* *Strict* *(* *strict=True* *)* *,* *Gt* *(* *gt=0* *)* *,* *Le* *(* *le=1* *)* *]* *)* *]*) #### `distance_threshold: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Strict(strict=True), Gt(gt=0), Le(le=1)])]` From 69b0f247f0caa537b5706ac78b8fadaceaa1b1c6 Mon Sep 17 00:00:00 2001 From: paoloredis Date: Mon, 24 Mar 2025 11:51:44 +0100 Subject: [PATCH 08/13] Add weight frontmatter property to index pages to fix order --- content/integrate/redisvl/api/_index.md | 1 + content/integrate/redisvl/overview/_index.md | 1 + content/integrate/redisvl/user_guide/_index.md | 1 + 3 files changed, 3 insertions(+) diff --git a/content/integrate/redisvl/api/_index.md b/content/integrate/redisvl/api/_index.md index 8a37bc7d01..50d1272e06 100644 --- a/content/integrate/redisvl/api/_index.md +++ b/content/integrate/redisvl/api/_index.md @@ -2,6 +2,7 @@ linkTitle: RedisVL API title: RedisVL API type: integration +weight: 5 --- diff --git a/content/integrate/redisvl/overview/_index.md b/content/integrate/redisvl/overview/_index.md index 0e63a665d7..4abd9ef244 100644 --- a/content/integrate/redisvl/overview/_index.md +++ b/content/integrate/redisvl/overview/_index.md @@ -2,6 +2,7 @@ linkTitle: Overview title: Overview type: integration +weight: 3 --- diff --git a/content/integrate/redisvl/user_guide/_index.md b/content/integrate/redisvl/user_guide/_index.md index 4bcd38f865..42bdd904ac 100644 --- a/content/integrate/redisvl/user_guide/_index.md +++ b/content/integrate/redisvl/user_guide/_index.md @@ -2,6 +2,7 @@ linkTitle: User Guides title: User Guides type: integration +weight: 4 --- From 220a43946a8f9fd1080101189675e4c37ad7500c Mon Sep 17 00:00:00 2001 From: paoloredis Date: Mon, 24 Mar 2025 12:00:55 +0100 Subject: [PATCH 09/13] Fix linkTitle --- content/integrate/redisvl/user_guide/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redisvl/user_guide/_index.md b/content/integrate/redisvl/user_guide/_index.md index 42bdd904ac..d2667f4b6a 100644 --- a/content/integrate/redisvl/user_guide/_index.md +++ b/content/integrate/redisvl/user_guide/_index.md @@ -1,5 +1,5 @@ --- -linkTitle: User Guides +linkTitle: User guides title: User Guides type: integration weight: 4 From 89c33ec4109ecd66cec2d065f9165e8269c8f62a Mon Sep 17 00:00:00 2001 From: paoloredis Date: Mon, 24 Mar 2025 14:58:03 +0100 Subject: [PATCH 10/13] Fix linkTitle for user_guides and add weight property for sorting --- content/integrate/redisvl/api/cache.md | 2 +- content/integrate/redisvl/api/router.md | 2 +- content/integrate/redisvl/api/searchindex.md | 2 +- content/integrate/redisvl/api/session_manager.md | 2 +- content/integrate/redisvl/user_guide/01_getting_started.md | 3 ++- content/integrate/redisvl/user_guide/02_hybrid_queries.md | 1 + content/integrate/redisvl/user_guide/03_llmcache.md | 3 ++- content/integrate/redisvl/user_guide/04_vectorizers.md | 1 + content/integrate/redisvl/user_guide/05_hash_vs_json.md | 3 ++- content/integrate/redisvl/user_guide/06_rerankers.md | 1 + content/integrate/redisvl/user_guide/07_session_manager.md | 3 ++- content/integrate/redisvl/user_guide/08_semantic_router.md | 3 ++- 12 files changed, 17 insertions(+), 9 deletions(-) diff --git a/content/integrate/redisvl/api/cache.md b/content/integrate/redisvl/api/cache.md index 788e266731..b15bc6c3f1 100644 --- a/content/integrate/redisvl/api/cache.md +++ b/content/integrate/redisvl/api/cache.md @@ -1,5 +1,5 @@ --- -linkTitle: LLM Cache +linkTitle: LLM cache title: LLM Cache type: integration --- diff --git a/content/integrate/redisvl/api/router.md b/content/integrate/redisvl/api/router.md index aefb38a877..50e5de3cf4 100644 --- a/content/integrate/redisvl/api/router.md +++ b/content/integrate/redisvl/api/router.md @@ -1,5 +1,5 @@ --- -linkTitle: Semantic Router +linkTitle: Semantic router title: Semantic Router type: integration --- diff --git a/content/integrate/redisvl/api/searchindex.md b/content/integrate/redisvl/api/searchindex.md index faeaa23167..3d805d1477 100644 --- a/content/integrate/redisvl/api/searchindex.md +++ b/content/integrate/redisvl/api/searchindex.md @@ -1,5 +1,5 @@ --- -linkTitle: Search Index Classes +linkTitle: Search index classes title: Search Index Classes type: integration --- diff --git a/content/integrate/redisvl/api/session_manager.md b/content/integrate/redisvl/api/session_manager.md index 02487b5110..6ea469eb69 100644 --- a/content/integrate/redisvl/api/session_manager.md +++ b/content/integrate/redisvl/api/session_manager.md @@ -1,5 +1,5 @@ --- -linkTitle: LLM Session Manager +linkTitle: LLM session manager title: LLM Session Manager type: integration --- diff --git a/content/integrate/redisvl/user_guide/01_getting_started.md b/content/integrate/redisvl/user_guide/01_getting_started.md index 739cf87890..56f71a6e5d 100644 --- a/content/integrate/redisvl/user_guide/01_getting_started.md +++ b/content/integrate/redisvl/user_guide/01_getting_started.md @@ -1,7 +1,8 @@ --- -linkTitle: Getting Started with RedisVL +linkTitle: Getting started with RedisVL title: Getting Started with RedisVL type: integration +weight: 01 --- `redisvl` is a versatile Python library with an integrated CLI, designed to enhance AI applications using Redis. This guide will walk you through the following steps: diff --git a/content/integrate/redisvl/user_guide/02_hybrid_queries.md b/content/integrate/redisvl/user_guide/02_hybrid_queries.md index 205e95cae7..8dbdaaeb93 100644 --- a/content/integrate/redisvl/user_guide/02_hybrid_queries.md +++ b/content/integrate/redisvl/user_guide/02_hybrid_queries.md @@ -2,6 +2,7 @@ linkTitle: Querying with RedisVL title: Querying with RedisVL type: integration +weight: 02 --- diff --git a/content/integrate/redisvl/user_guide/03_llmcache.md b/content/integrate/redisvl/user_guide/03_llmcache.md index da54e52af3..acdbaa6b20 100644 --- a/content/integrate/redisvl/user_guide/03_llmcache.md +++ b/content/integrate/redisvl/user_guide/03_llmcache.md @@ -1,7 +1,8 @@ --- -linkTitle: Semantic Caching for LLMs +linkTitle: Semantic caching for LLMs title: Semantic Caching for LLMs type: integration +weight: 03 --- diff --git a/content/integrate/redisvl/user_guide/04_vectorizers.md b/content/integrate/redisvl/user_guide/04_vectorizers.md index 0ceb56e6c1..516861aca3 100644 --- a/content/integrate/redisvl/user_guide/04_vectorizers.md +++ b/content/integrate/redisvl/user_guide/04_vectorizers.md @@ -2,6 +2,7 @@ linkTitle: Vectorizers title: Vectorizers type: integration +weight: 04 --- diff --git a/content/integrate/redisvl/user_guide/05_hash_vs_json.md b/content/integrate/redisvl/user_guide/05_hash_vs_json.md index 0b3ba1bedd..ade92835b1 100644 --- a/content/integrate/redisvl/user_guide/05_hash_vs_json.md +++ b/content/integrate/redisvl/user_guide/05_hash_vs_json.md @@ -1,7 +1,8 @@ --- -linkTitle: Hash vs JSON Storage +linkTitle: Hash vs JSON storage title: Hash vs JSON Storage type: integration +weight: 05 --- diff --git a/content/integrate/redisvl/user_guide/06_rerankers.md b/content/integrate/redisvl/user_guide/06_rerankers.md index be54805d9a..5747a0078c 100644 --- a/content/integrate/redisvl/user_guide/06_rerankers.md +++ b/content/integrate/redisvl/user_guide/06_rerankers.md @@ -2,6 +2,7 @@ linkTitle: Rerankers title: Rerankers type: integration +weight: 06 --- diff --git a/content/integrate/redisvl/user_guide/07_session_manager.md b/content/integrate/redisvl/user_guide/07_session_manager.md index 05b1b292e1..57eac77620 100644 --- a/content/integrate/redisvl/user_guide/07_session_manager.md +++ b/content/integrate/redisvl/user_guide/07_session_manager.md @@ -1,7 +1,8 @@ --- -linkTitle: LLM Session Memory +linkTitle: LLM session memory title: LLM Session Memory type: integration +weight: 07 --- diff --git a/content/integrate/redisvl/user_guide/08_semantic_router.md b/content/integrate/redisvl/user_guide/08_semantic_router.md index b144455506..91b8ed66bb 100644 --- a/content/integrate/redisvl/user_guide/08_semantic_router.md +++ b/content/integrate/redisvl/user_guide/08_semantic_router.md @@ -1,7 +1,8 @@ --- -linkTitle: Semantic Routing +linkTitle: Semantic routing title: Semantic Routing type: integration +weight: 08 --- From ed7ab0feea56acb8f04a476a94de23bd417338e0 Mon Sep 17 00:00:00 2001 From: paoloredis Date: Mon, 24 Mar 2025 15:00:24 +0100 Subject: [PATCH 11/13] Rename user_guides --- .../user_guide/{01_getting_started.md => getting_started.md} | 0 .../redisvl/user_guide/{05_hash_vs_json.md => hash_vs_json.md} | 0 .../user_guide/{02_hybrid_queries.md => hybrid_queries.md} | 0 .../integrate/redisvl/user_guide/{03_llmcache.md => llmcache.md} | 0 .../redisvl/user_guide/{06_rerankers.md => rerankers.md} | 0 .../user_guide/{08_semantic_router.md => semantic_router.md} | 0 .../user_guide/{07_session_manager.md => session_manager.md} | 0 .../redisvl/user_guide/{04_vectorizers.md => vectorizers.md} | 0 8 files changed, 0 insertions(+), 0 deletions(-) rename content/integrate/redisvl/user_guide/{01_getting_started.md => getting_started.md} (100%) rename content/integrate/redisvl/user_guide/{05_hash_vs_json.md => hash_vs_json.md} (100%) rename content/integrate/redisvl/user_guide/{02_hybrid_queries.md => hybrid_queries.md} (100%) rename content/integrate/redisvl/user_guide/{03_llmcache.md => llmcache.md} (100%) rename content/integrate/redisvl/user_guide/{06_rerankers.md => rerankers.md} (100%) rename content/integrate/redisvl/user_guide/{08_semantic_router.md => semantic_router.md} (100%) rename content/integrate/redisvl/user_guide/{07_session_manager.md => session_manager.md} (100%) rename content/integrate/redisvl/user_guide/{04_vectorizers.md => vectorizers.md} (100%) diff --git a/content/integrate/redisvl/user_guide/01_getting_started.md b/content/integrate/redisvl/user_guide/getting_started.md similarity index 100% rename from content/integrate/redisvl/user_guide/01_getting_started.md rename to content/integrate/redisvl/user_guide/getting_started.md diff --git a/content/integrate/redisvl/user_guide/05_hash_vs_json.md b/content/integrate/redisvl/user_guide/hash_vs_json.md similarity index 100% rename from content/integrate/redisvl/user_guide/05_hash_vs_json.md rename to content/integrate/redisvl/user_guide/hash_vs_json.md diff --git a/content/integrate/redisvl/user_guide/02_hybrid_queries.md b/content/integrate/redisvl/user_guide/hybrid_queries.md similarity index 100% rename from content/integrate/redisvl/user_guide/02_hybrid_queries.md rename to content/integrate/redisvl/user_guide/hybrid_queries.md diff --git a/content/integrate/redisvl/user_guide/03_llmcache.md b/content/integrate/redisvl/user_guide/llmcache.md similarity index 100% rename from content/integrate/redisvl/user_guide/03_llmcache.md rename to content/integrate/redisvl/user_guide/llmcache.md diff --git a/content/integrate/redisvl/user_guide/06_rerankers.md b/content/integrate/redisvl/user_guide/rerankers.md similarity index 100% rename from content/integrate/redisvl/user_guide/06_rerankers.md rename to content/integrate/redisvl/user_guide/rerankers.md diff --git a/content/integrate/redisvl/user_guide/08_semantic_router.md b/content/integrate/redisvl/user_guide/semantic_router.md similarity index 100% rename from content/integrate/redisvl/user_guide/08_semantic_router.md rename to content/integrate/redisvl/user_guide/semantic_router.md diff --git a/content/integrate/redisvl/user_guide/07_session_manager.md b/content/integrate/redisvl/user_guide/session_manager.md similarity index 100% rename from content/integrate/redisvl/user_guide/07_session_manager.md rename to content/integrate/redisvl/user_guide/session_manager.md diff --git a/content/integrate/redisvl/user_guide/04_vectorizers.md b/content/integrate/redisvl/user_guide/vectorizers.md similarity index 100% rename from content/integrate/redisvl/user_guide/04_vectorizers.md rename to content/integrate/redisvl/user_guide/vectorizers.md From e318f9bef4e6efc8fae4191983c83b19192fcf50 Mon Sep 17 00:00:00 2001 From: paoloredis Date: Mon, 24 Mar 2025 15:18:28 +0100 Subject: [PATCH 12/13] Fix docs.redisvl.com links --- content/integrate/redisvl/user_guide/getting_started.md | 2 +- content/integrate/redisvl/user_guide/hybrid_queries.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/integrate/redisvl/user_guide/getting_started.md b/content/integrate/redisvl/user_guide/getting_started.md index 56f71a6e5d..6032bdb897 100644 --- a/content/integrate/redisvl/user_guide/getting_started.md +++ b/content/integrate/redisvl/user_guide/getting_started.md @@ -275,7 +275,7 @@ print(keys) ## Creating `VectorQuery` Objects -Next we will create a vector query object for our newly populated index. This example will use a simple vector to demonstrate how vector similarity works. Vectors in production will likely be much larger than 3 floats and often require Machine Learning models (i.e. Huggingface sentence transformers) or an embeddings API (Cohere, OpenAI). `redisvl` provides a set of [Vectorizers](https://docs.redisvl.com/en/latest/user_guide/vectorizers_04.html#openai) to assist in vector creation. +Next we will create a vector query object for our newly populated index. This example will use a simple vector to demonstrate how vector similarity works. Vectors in production will likely be much larger than 3 floats and often require Machine Learning models (i.e. Huggingface sentence transformers) or an embeddings API (Cohere, OpenAI). `redisvl` provides a set of [Vectorizers]({{< relref "vectorizers#openai" >}}) to assist in vector creation. ```python diff --git a/content/integrate/redisvl/user_guide/hybrid_queries.md b/content/integrate/redisvl/user_guide/hybrid_queries.md index 8dbdaaeb93..15bfd11ccb 100644 --- a/content/integrate/redisvl/user_guide/hybrid_queries.md +++ b/content/integrate/redisvl/user_guide/hybrid_queries.md @@ -635,7 +635,7 @@ result_print(index.query(range_query)) ## Advanced Query Modifiers -See all modifier options available on the query API docs: https://docs.redisvl.com/en/latest/api/query.html +See all modifier options available on the query API docs: https://redis.io/docs/latest/integrate/redisvl/api/query ```python From b40397044eb9b3603d7c619330fcfd88764bedb4 Mon Sep 17 00:00:00 2001 From: paoloredis Date: Mon, 24 Mar 2025 15:36:02 +0100 Subject: [PATCH 13/13] Add hideListLinks frontmatter property to _index.md pages --- content/integrate/redisvl/api/_index.md | 1 + content/integrate/redisvl/overview/_index.md | 1 + content/integrate/redisvl/user_guide/_index.md | 1 + 3 files changed, 3 insertions(+) diff --git a/content/integrate/redisvl/api/_index.md b/content/integrate/redisvl/api/_index.md index 50d1272e06..48f3e79e8e 100644 --- a/content/integrate/redisvl/api/_index.md +++ b/content/integrate/redisvl/api/_index.md @@ -3,6 +3,7 @@ linkTitle: RedisVL API title: RedisVL API type: integration weight: 5 +hideListLinks: true --- diff --git a/content/integrate/redisvl/overview/_index.md b/content/integrate/redisvl/overview/_index.md index 4abd9ef244..602ca30ff3 100644 --- a/content/integrate/redisvl/overview/_index.md +++ b/content/integrate/redisvl/overview/_index.md @@ -3,6 +3,7 @@ linkTitle: Overview title: Overview type: integration weight: 3 +hideListLinks: true --- diff --git a/content/integrate/redisvl/user_guide/_index.md b/content/integrate/redisvl/user_guide/_index.md index d2667f4b6a..a133211c3b 100644 --- a/content/integrate/redisvl/user_guide/_index.md +++ b/content/integrate/redisvl/user_guide/_index.md @@ -3,6 +3,7 @@ linkTitle: User guides title: User Guides type: integration weight: 4 +hideListLinks: true ---