@@ -9,7 +9,7 @@ type: integration
99
1010<a id =" semantic-cache-api " ></a >
1111
12- ### ` * class* SemanticCache(name='llmcache', distance_threshold=0.1, ttl=None, vectorizer=None, filterable_fields=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, **kwargs)`
12+ ### ` class SemanticCache(name='llmcache', distance_threshold=0.1, ttl=None, vectorizer=None, filterable_fields=None, redis_client=None, redis_url='redis://localhost:6379', connection_kwargs={}, overwrite=False, **kwargs) `
1313
1414Bases: ` BaseLLMCache `
1515
@@ -22,16 +22,16 @@ Semantic Cache for Large Language Models.
2222 Defaults to “llmcache”.
2323 * ** distance_threshold** (* float* * ,* * optional* ) – Semantic threshold for the
2424 cache. Defaults to 0.1.
25- * ** ttl** (* Optional* * [ ** int* * ] * * ,* * optional* ) – The time-to-live for records cached
25+ * ** ttl** (* Optional* * [ * * int* * ] * * ,* * optional* ) – The time-to-live for records cached
2626 in Redis. Defaults to None.
27- * ** vectorizer** (* Optional* * [ ** BaseVectorizer* * ] * * ,* * optional* ) – The vectorizer for the cache.
27+ * ** vectorizer** (* Optional* * [ * * BaseVectorizer* * ] * * ,* * optional* ) – The vectorizer for the cache.
2828 Defaults to HFTextVectorizer.
29- * ** filterable_fields** (* Optional* * [ ** List* * [ ** Dict* * [ ** str* * ,* * Any* * ] * * ] * * ] * ) – An optional list of RedisVL fields
29+ * ** filterable_fields** (* Optional* * [ * * List* * [ * * Dict* * [ * * str* * ,* * Any* * ] * * ] * * ] * ) – An optional list of RedisVL fields
3030 that can be used to customize cache retrieval with filters.
31- * ** redis_client** (* Optional* * [ ** Redis* * ] * * ,* * optional* ) – A redis client connection instance.
31+ * ** redis_client** (* Optional* * [ * * Redis* * ] * * ,* * optional* ) – A redis client connection instance.
3232 Defaults to None.
3333 * ** redis_url** (* str* * ,* * optional* ) – The redis url. Defaults to redis://localhost:6379.
34- * ** connection_kwargs** (* Dict* * [ ** str* * ,* * Any* * ] * ) – The connection arguments
34+ * ** connection_kwargs** (* Dict* * [ * * str* * ,* * Any* * ] * ) – The connection arguments
3535 for the redis client. Defaults to empty {}.
3636 * ** overwrite** (* bool* ) – Whether or not to force overwrite the schema for
3737 the semantic cache index. Defaults to false.
@@ -41,7 +41,7 @@ Semantic Cache for Large Language Models.
4141 * ** ValueError** – If the threshold is not between 0 and 1.
4242 * ** ValueError** – If existing schema does not match new schema and overwrite is False.
4343
44- #### ` * async* acheck(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None)`
44+ #### ` async acheck(prompt=None, vector=None, num_results=1, return_fields=None, filter_expression=None, distance_threshold=None) `
4545
4646Async check the semantic cache for results similar to the specified prompt
4747or vector.
@@ -52,19 +52,19 @@ input. It checks for semantically similar prompts and fetches the cached
5252LLM responses.
5353
5454* ** Parameters:**
55- * ** prompt** (* Optional* * [ ** str* * ] * * ,* * optional* ) – The text prompt to search for in
55+ * ** prompt** (* Optional* * [ * * str* * ] * * ,* * optional* ) – The text prompt to search for in
5656 the cache.
57- * ** vector** (* Optional* * [ ** List* * [ ** float* * ] * * ] * * ,* * optional* ) – The vector representation
57+ * ** vector** (* Optional* * [ * * List* * [ * * float* * ] * * ] * * ,* * optional* ) – The vector representation
5858 of the prompt to search for in the cache.
5959 * ** num_results** (* int* * ,* * optional* ) – The number of cached results to return.
6060 Defaults to 1.
61- * ** return_fields** (* Optional* * [ ** List* * [ ** str* * ] * * ] * * ,* * optional* ) – The fields to include
61+ * ** return_fields** (* Optional* * [ * * List* * [ * * str* * ] * * ] * * ,* * optional* ) – The fields to include
6262 in each returned result. If None, defaults to all available
6363 fields in the cached entry.
6464 * ** filter_expression** (* Optional* * [ * [ * FilterExpression* ] ({{< relref "filter/#filterexpression" >}}) * ] * ) – Optional filter expression
6565 that can be used to filter cache results. Defaults to None and
6666 the full cache will be searched.
67- * ** distance_threshold** (* Optional* * [ ** float* * ] * ) – The threshold for semantic
67+ * ** distance_threshold** (* Optional* * [ * * float* * ] * ) – The threshold for semantic
6868 vector distance.
6969* ** Returns:**
7070 A list of dicts containing the requested
@@ -82,32 +82,32 @@ response = await cache.acheck(
8282)
8383```
8484
85- #### ` * async* adrop(ids=None, keys=None)`
85+ #### ` async adrop(ids=None, keys=None) `
8686
8787Async expire specific entries from the cache by id or specific
8888Redis key.
8989
9090* ** Parameters:**
91- * ** ids** (* Optional* * [ ** str* * ] * ) – The document ID or IDs to remove from the cache.
92- * ** keys** (* Optional* * [ ** str* * ] * ) – The Redis keys to remove from the cache.
91+ * ** ids** (* Optional* * [ * * str* * ] * ) – The document ID or IDs to remove from the cache.
92+ * ** keys** (* Optional* * [ * * str* * ] * ) – The Redis keys to remove from the cache.
9393* ** Return type:**
9494 None
9595
96- #### ` * async* astore(prompt, response, vector=None, metadata=None, filters=None, ttl=None)`
96+ #### ` async astore(prompt, response, vector=None, metadata=None, filters=None, ttl=None) `
9797
9898Async stores the specified key-value pair in the cache along with metadata.
9999
100100* ** Parameters:**
101101 * ** prompt** (* str* ) – The user prompt to cache.
102102 * ** response** (* str* ) – The LLM response to cache.
103- * ** vector** (* Optional* * [ ** List* * [ ** float* * ] * * ] * * ,* * optional* ) – The prompt vector to
103+ * ** vector** (* Optional* * [ * * List* * [ * * float* * ] * * ] * * ,* * optional* ) – The prompt vector to
104104 cache. Defaults to None, and the prompt vector is generated on
105105 demand.
106- * ** metadata** (* Optional* * [ ** Dict* * [ ** str* * ,* * Any* * ] * * ] * * ,* * optional* ) – The optional metadata to cache
106+ * ** metadata** (* Optional* * [ * * Dict* * [ * * str* * ,* * Any* * ] * * ] * * ,* * optional* ) – The optional metadata to cache
107107 alongside the prompt and response. Defaults to None.
108- * ** filters** (* Optional* * [ ** Dict* * [ ** str* * ,* * Any* * ] * * ] * ) – The optional tag to assign to the cache entry.
108+ * ** filters** (* Optional* * [ * * Dict* * [ * * str* * ,* * Any* * ] * * ] * ) – The optional tag to assign to the cache entry.
109109 Defaults to None.
110- * ** ttl** (* Optional* * [ ** int* * ] * ) – The optional TTL override to use on this individual cache
110+ * ** ttl** (* Optional* * [ * * int* * ] * ) – The optional TTL override to use on this individual cache
111111 entry. Defaults to the global TTL setting.
112112* ** Returns:**
113113 The Redis key for the entries added to the semantic cache.
@@ -126,7 +126,7 @@ key = await cache.astore(
126126)
127127```
128128
129- #### ` * async* aupdate(key, **kwargs)`
129+ #### ` async aupdate(key, **kwargs) `
130130
131131Async update specific fields within an existing cache entry. If no fields
132132are passed, then only the document TTL is refreshed.
@@ -158,19 +158,19 @@ input. It checks for semantically similar prompts and fetches the cached
158158LLM responses.
159159
160160* ** Parameters:**
161- * ** prompt** (* Optional* * [ ** str* * ] * * ,* * optional* ) – The text prompt to search for in
161+ * ** prompt** (* Optional* * [ * * str* * ] * * ,* * optional* ) – The text prompt to search for in
162162 the cache.
163- * ** vector** (* Optional* * [ ** List* * [ ** float* * ] * * ] * * ,* * optional* ) – The vector representation
163+ * ** vector** (* Optional* * [ * * List* * [ * * float* * ] * * ] * * ,* * optional* ) – The vector representation
164164 of the prompt to search for in the cache.
165165 * ** num_results** (* int* * ,* * optional* ) – The number of cached results to return.
166166 Defaults to 1.
167- * ** return_fields** (* Optional* * [ ** List* * [ ** str* * ] * * ] * * ,* * optional* ) – The fields to include
167+ * ** return_fields** (* Optional* * [ * * List* * [ * * str* * ] * * ] * * ,* * optional* ) – The fields to include
168168 in each returned result. If None, defaults to all available
169169 fields in the cached entry.
170170 * ** filter_expression** (* Optional* * [ * [ * FilterExpression* ] ({{< relref "filter/#filterexpression" >}}) * ] * ) – Optional filter expression
171171 that can be used to filter cache results. Defaults to None and
172172 the full cache will be searched.
173- * ** distance_threshold** (* Optional* * [ ** float* * ] * ) – The threshold for semantic
173+ * ** distance_threshold** (* Optional* * [ * * float* * ] * ) – The threshold for semantic
174174 vector distance.
175175* ** Returns:**
176176 A list of dicts containing the requested
@@ -209,8 +209,8 @@ Manually expire specific entries from the cache by id or specific
209209Redis key.
210210
211211* ** Parameters:**
212- * ** ids** (* Optional* * [ ** str* * ] * ) – The document ID or IDs to remove from the cache.
213- * ** keys** (* Optional* * [ ** str* * ] * ) – The Redis keys to remove from the cache.
212+ * ** ids** (* Optional* * [ * * str* * ] * ) – The document ID or IDs to remove from the cache.
213+ * ** keys** (* Optional* * [ * * str* * ] * ) – The Redis keys to remove from the cache.
214214* ** Return type:**
215215 None
216216
@@ -231,7 +231,7 @@ Sets the semantic distance threshold for the cache.
231231Set the default TTL, in seconds, for entries in the cache.
232232
233233* ** Parameters:**
234- ** ttl** (* Optional* * [ ** int* * ] * * ,* * optional* ) – The optional time-to-live expiration
234+ ** ttl** (* Optional* * [ * * int* * ] * * ,* * optional* ) – The optional time-to-live expiration
235235 for the cache, in seconds.
236236* ** Raises:**
237237 ** ValueError** – If the time-to-live value is not an integer.
@@ -243,14 +243,14 @@ Stores the specified key-value pair in the cache along with metadata.
243243* ** Parameters:**
244244 * ** prompt** (* str* ) – The user prompt to cache.
245245 * ** response** (* str* ) – The LLM response to cache.
246- * ** vector** (* Optional* * [ ** List* * [ ** float* * ] * * ] * * ,* * optional* ) – The prompt vector to
246+ * ** vector** (* Optional* * [ * * List* * [ * * float* * ] * * ] * * ,* * optional* ) – The prompt vector to
247247 cache. Defaults to None, and the prompt vector is generated on
248248 demand.
249- * ** metadata** (* Optional* * [ ** Dict* * [ ** str* * ,* * Any* * ] * * ] * * ,* * optional* ) – The optional metadata to cache
249+ * ** metadata** (* Optional* * [ * * Dict* * [ * * str* * ,* * Any* * ] * * ] * * ,* * optional* ) – The optional metadata to cache
250250 alongside the prompt and response. Defaults to None.
251- * ** filters** (* Optional* * [ ** Dict* * [ ** str* * ,* * Any* * ] * * ] * ) – The optional tag to assign to the cache entry.
251+ * ** filters** (* Optional* * [ * * Dict* * [ * * str* * ,* * Any* * ] * * ] * ) – The optional tag to assign to the cache entry.
252252 Defaults to None.
253- * ** ttl** (* Optional* * [ ** int* * ] * ) – The optional TTL override to use on this individual cache
253+ * ** ttl** (* Optional* * [ * * int* * ] * ) – The optional TTL override to use on this individual cache
254254 entry. Defaults to the global TTL setting.
255255* ** Returns:**
256256 The Redis key for the entries added to the semantic cache.
@@ -288,7 +288,7 @@ cache.update(key, metadata={"hit_count": 1, "model_name": "Llama-2-7b"})
288288)
289289```
290290
291- #### ` * property* aindex *: [ AsyncSearchIndex]({{< relref "searchindex/#asyncsearchindex" >}}) | None* `
291+ #### ` property aindex: ` [ ` AsyncSearchIndex ` ] ({{< relref "searchindex/#asyncsearchindex" >}})` | None`
292292
293293The underlying AsyncSearchIndex for the cache.
294294
@@ -297,7 +297,7 @@ The underlying AsyncSearchIndex for the cache.
297297* ** Return type:**
298298 [ AsyncSearchIndex] ({{< relref "searchindex/#asyncsearchindex" >}})
299299
300- #### ` * property* distance_threshold * : float* `
300+ #### ` property distance_threshold: float `
301301
302302The semantic distance threshold for the cache.
303303
@@ -306,7 +306,7 @@ The semantic distance threshold for the cache.
306306* ** Return type:**
307307 float
308308
309- #### ` * property* index *: [ SearchIndex]({{< relref "searchindex/#searchindex" >}})* `
309+ #### ` property index: ` [ ` SearchIndex ` ] ({{< relref "searchindex/#searchindex" >}})` `
310310
311311The underlying SearchIndex for the cache.
312312
@@ -315,6 +315,6 @@ The underlying SearchIndex for the cache.
315315* ** Return type:**
316316 [ SearchIndex] ({{< relref "searchindex/#searchindex" >}})
317317
318- #### ` * property* ttl * : int | None* `
318+ #### ` property ttl: int | None `
319319
320320The default TTL, in seconds, for entries in the cache.
0 commit comments