You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/develop/interact/search-and-query/best-practices/scalable-query-best-practices.md
+43-12Lines changed: 43 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,7 +17,7 @@ If you're using Redis Software or Redis Cloud, see the [best practices for scala
17
17
{{< /note >}}
18
18
19
19
## Checklist
20
-
Below are some basic steps to ensure good performance of Redis Query Engine.
20
+
Below are some basic steps to ensure good performance of the Redis Query Engine (RQE).
21
21
22
22
* Create a Redis data model with your query patterns in mind.
23
23
* Ensure the Redis architecture has been sized for the expected load using the [sizing calculator](https://redis.io/redisearch-sizing-calculator/).
@@ -64,19 +64,19 @@ The following informational items are available for analysis:
64
64
65
65
## Anti-patterns
66
66
67
-
The following items are anti-patterns for RQE:
67
+
When designing and querying indexes in RQE, certain practices can hinder performance, scalability, and maintainability. Below are some common anti-patterns to avoid:
68
68
69
-
- Large documents
70
-
- Deeply-nested fields
71
-
- Large result sets
72
-
- Wildcarding
73
-
- Large projections
69
+
-**Large documents**: storing excessively large documents in Redis makes data retrieval slower and increases memory usage. Break data into smaller, focused records whenever possible.
70
+
-**Deeply-nested fields**: retrieving or indexing deeply-nested JSON fields is computationally expensive. Use a flatter schema for better performance.
71
+
-**Large result sets**: fetching unnecessarily large result sets puts a strain on memory and network resources. Limit results to only what is needed.
72
+
-**Wildcarding**: using wildcard patterns indiscriminately in queries can lead to large and inefficient scans, especially if the index size is significant.
73
+
-**Large projections**: including excessive fields in query results increases memory overhead and slows down query execution. Limit projections to essential fields.
74
74
75
-
The following examples depict an anti-pattern index schema and query, followed by a corrected index schema and query, which allows for scalability with the Redis Query Engine.
75
+
The following examples depict an anti-pattern index schema and query, followed by corrected versions designed for scalability with RQE.
76
76
77
77
### Anti-pattern index schema
78
78
79
-
The following index schema is not optimized for vertical scaling:
79
+
The following schema introduces challenges for scalability and performance:
80
80
81
81
```sh
82
82
FT.CREATE jsonidx:profiles ON JSON PREFIX 1 profiles:
- Minimal schema definition: the schema is sparse and lacks fields like `lastName`, `id`, and `version` that might be frequently queried. This results in additional operations to fetch these fields separately, reducing efficiency.
91
+
- Missing `SORTABLE` flag for text fields: sorting operations on unsortable fields require full-text processing, which is slow.
92
+
- Wildcard indexing: `$.tags.*` creates a broad index that can lead to excessive memory usage and reduced query performance.
93
+
88
94
### Anti-pattern query
89
95
90
-
The following query is not optimized for vertical scaling:
96
+
The following query is inefficient and not optimized for vertical scaling:
- Wildcard projection (`LOAD *`): retrieving all fields in the result set is inefficient and increases memory usage, especially if the documents are large.
104
+
- Unnecessary fields: fields that aren't required for the current operation are still fetched, slowing down execution.
105
+
- Lack of advanced query syntax: without specifying a query dialect or leveraging features like tagging, the query may perform unnecessary computations.
95
106
96
107
### Improved index schema
97
108
98
-
Here's an improved index schema that follows best practices for vertical scaling:
109
+
Here’s an optimized schema that adheres to best practices for vertical scaling:
99
110
100
111
```sh
101
112
FT.CREATE jsonidx:profiles ON JSON PREFIX 1 profiles:
-`NOSTEM` for text fields: prevents stemming on fields like `firstName` and `lastName` to allow for exact matches (e.g., "Smith" stays "Smith").
124
+
- Expanded schema: adds commonly queried fields like `lastName`, `id`, and `version`, making queries more efficient by reducing the need for post-query data retrieval.
125
+
-`TAG` fields: `id` and `ver` are defined as `TAG` fields to support fast filtering with exact matches.
126
+
-`SORTABLE` for all relevant fields: ensures that sorting operations are efficient without requiring full-text scanning.
127
+
128
+
You might be wondering why `$.tags.* as t NUMERIC SORTABLE` is acceptable in the improved schema and it wasn't previously.
129
+
The inclusion of `$.tags.*` is acceptable when:
130
+
131
+
- It has a clear purpose: it is actively used in queries, such as filtering on numeric ranges or matching specific values.
132
+
- Other fields in the schema complement it: these fields reduce over-reliance on `$.tags.*` for all query operations, distributing the load more evenly.
133
+
- Projections and limits are managed carefully: queries that use `$.tags.*` should avoid loading unnecessary fields or returning excessively large result sets.
134
+
110
135
### Improved query
111
136
112
-
Here's an improved query that follows best practices for vertical scaling:
137
+
The following query is better suited for vertical scaling:
113
138
114
139
```sh
115
140
FT.AGGREGATE jsonidx:profiles '@t:[1299 1299]'
116
141
LOAD 6 id t name lastname loc ver
117
142
LIMIT 0 10
118
143
DIALECT 3
119
144
```
145
+
146
+
Improvements:
147
+
148
+
- Targeted projection: the `LOAD` clause specifies only essential fields (`id, t, name, lastname, loc, ver`), reducing memory and network overhead.
149
+
- Limited results: the `LIMIT` clause ensures the query retrieves only the first 10 results, avoiding large result sets.
150
+
-[`DIALECT 3`]({{< relref "/develop/interact/search-and-query/advanced-concepts/dialects#dialect-3" >}}): enables the latest RQE syntax and features, ensuring compatibility with modern capabilities.
0 commit comments