diff --git a/docs/command-reference/compatibility.md b/docs/command-reference/compatibility.md index 8d53801..8a115ee 100644 --- a/docs/command-reference/compatibility.md +++ b/docs/command-reference/compatibility.md @@ -344,12 +344,18 @@ sidebar_position: 0 | | JSON.STRLEN | Fully supported | | | JSON.TOGGLE | Fully supported | | | JSON.TYPE | Fully supported | -| Search | FT.CREATE | Fully supported | -| | FT.SEARCH | Fully supported | -| | FT.SYNUPDATE | Fully supported | -| | FT.SYNDUMP | Fully supported | +| Search | FT.CREATE | Partially supported | +| | FT.SEARCH | Partially supported | | | FT.ALTER | Fully supported | +| | FT.DROPINDEX | Partially supported | +| | FT.INFO | Partially supported | +| | FT._LIST | Fully supported | +| | FT.PROFILE | Partially supported | +| | FT.AGGREGATE | Partially supported | | | FT.TAGVALS | Fully supported | +| | FT.SYNUPDATE | Fully supported | +| | FT.SYNDUMP | Fully supported | +| | FT.CONFIG | Fully supported | | Auto Suggest | TBD | Unsupported | | T-Digest | TBD | Unsupported | | Time Series | TBD | Unsupported | diff --git a/docs/command-reference/search/ft.aggregate.md b/docs/command-reference/search/ft.aggregate.md new file mode 100644 index 0000000..230edc7 --- /dev/null +++ b/docs/command-reference/search/ft.aggregate.md @@ -0,0 +1,135 @@ +--- +description: Runs a search query and performs aggregate transformations +--- + +# FT.AGGREGATE + +## Syntax + + FT.AGGREGATE index query + [LOAD count field [field ...]] + [GROUPBY nargs property [property ...] [REDUCE function nargs arg [arg ...]] [REDUCE function nargs arg [arg ...]]] + [SORTBY nargs property [ASC|DESC] [property [ASC|DESC] ...] [MAX num]] + [LIMIT offset num] + [PARAMS nargs name value [name value ...]] + +**Time complexity:** O(N) + +## Description + +Run a search query and perform aggregate transformations on the results. + +This command provides aggregation functionality similar to SQL GROUP BY operations, allowing you to group search results and apply reduction functions. + +## Required arguments + +
+index + +is index name. You must first create the index using [`FT.CREATE`](./ft.create.md). +
+ +
+query + +is text query to search. If it's more than a single word, put it in quotes. +Refer to [query syntax](https://redis.io/docs/latest/operate/oss_and_stack/stack-with-enterprise/search/) for more details. +
+ +## Optional arguments + +
+LOAD count field [field ...] + +loads additional fields from the document that are not part of the index. + +`count` is the number of fields to load. +`field` is the field name to load from the document. +
+ +
+GROUPBY nargs property [property ...] [REDUCE ...] + +groups the results according to one or more properties. + +`nargs` is the number of properties to group by. +`property` is the property name to group by. + +Optionally followed by one or more `REDUCE` clauses that aggregate the grouped results. +
+ +
+REDUCE function nargs arg [arg ...] + +applies an aggregate function to the grouped results. + +Common functions include: +- `COUNT` - counts the number of records in each group +- `SUM property` - sums the values of the given property +- `MIN property` - finds minimum value +- `MAX property` - finds maximum value +- `AVG property` - calculates average value +- `STDDEV property` - calculates standard deviation + +
+ +
+SORTBY nargs property [ASC|DESC] [property [ASC|DESC] ...] [MAX num] + +sorts the results by the given properties. + +`nargs` is the number of properties to sort by. +`property` is the property name to sort by. +`ASC|DESC` specifies sort order (default is ASC). +`MAX num` limits the number of results. +
+ +
+LIMIT offset num + +limits the results to the offset and number of results given. + +The offset is zero-indexed. Default is 0 10. +
+ +
+PARAMS nargs name value [name value ...] + +defines one or more value parameters that can be referenced in the query. + +Similar to [`FT.SEARCH`](./ft.search.md) PARAMS option. +
+ +## Return + +`FT.AGGREGATE` returns an array reply with aggregated results based on the specified grouping and reduction operations. + +:::info Limited support +FT.AGGREGATE has partial support in Dragonfly. Some advanced aggregation functions and options may not be fully implemented. +::: + +## Examples + +
+Group results by category and count + +```bash +dragonfly> FT.AGGREGATE products "*" GROUPBY 1 @category REDUCE COUNT 0 AS count +``` +
+ +
+Calculate average price by category + +```bash +dragonfly> FT.AGGREGATE products "*" GROUPBY 1 @category REDUCE AVG 1 @price AS avg_price +``` +
+ +## See also + +[`FT.SEARCH`](./ft.search.md) | [`FT.CREATE`](./ft.create.md) + +## Related topics + +- [RediSearch](https://redis.io/docs/latest/operate/oss_and_stack/stack-with-enterprise/search/) \ No newline at end of file diff --git a/docs/command-reference/search/ft.config.md b/docs/command-reference/search/ft.config.md new file mode 100644 index 0000000..69d5fbe --- /dev/null +++ b/docs/command-reference/search/ft.config.md @@ -0,0 +1,93 @@ +--- +description: Configure search engine at runtime +--- + +# FT.CONFIG + +## Syntax + + FT.CONFIG GET pattern + FT.CONFIG SET option value + FT.CONFIG HELP [option] + +**Time complexity:** O(1) + +## Description + +Configure search engine at runtime. + +## Required arguments + +
+GET pattern + +returns the current values of configuration parameters matching the given pattern. + +`pattern` is a glob-style pattern for the configuration parameter names. +
+ +
+SET option value + +sets a configuration parameter to the given value. + +`option` is the configuration parameter name. +`value` is the new value for the configuration parameter. +
+ +
+HELP [option] + +returns help information about configuration parameters. + +If `option` is specified, returns help for that specific parameter. +If no option is given, returns help for all available parameters. +
+ +## Return + +- `FT.CONFIG GET` returns an array with parameter names and their values. +- `FT.CONFIG SET` returns a simple string reply `OK` if executed correctly, or an error reply otherwise. +- `FT.CONFIG HELP` returns an array with parameter information including descriptions and current values. + +## Examples + +
+Get all search configuration parameters + +```bash +dragonfly> FT.CONFIG GET * +1) "MAXSEARCHRESULTS" +2) "1000000" +``` +
+ +
+Set maximum search results + +```bash +dragonfly> FT.CONFIG SET MAXSEARCHRESULTS 500000 +OK +``` +
+ +
+Get help for a specific parameter + +```bash +dragonfly> FT.CONFIG HELP MAXSEARCHRESULTS +1) "MAXSEARCHRESULTS" +2) "Description" +3) "Maximum number of results from ft.search command" +4) "Value" +5) "500000" +``` +
+ +## See also + +[`FT.CREATE`](./ft.create.md) | [`FT.SEARCH`](./ft.search.md) + +## Related topics + +- [RediSearch](https://redis.io/docs/stack/search) diff --git a/docs/command-reference/search/ft.create.md b/docs/command-reference/search/ft.create.md index 050a1b5..77dfb9d 100644 --- a/docs/command-reference/search/ft.create.md +++ b/docs/command-reference/search/ft.create.md @@ -9,8 +9,9 @@ description: Creates an index with the given spec FT.CREATE index [ON HASH | JSON] [PREFIX count prefix [prefix ...]] - SCHEMA field_name [AS alias] TEXT | TAG | NUMERIC | VECTOR [ SORTABLE ] - [NOINDEX] [ field_name [AS alias] TEXT | TAG | NUMERIC | VECTOR [ SORTABLE ] [NOINDEX] ...] + [STOPWORDS count [words...]] + SCHEMA field_name [AS alias] TEXT | TAG | NUMERIC | VECTOR | GEO [ SORTABLE ] + [NOINDEX] [ field_name [AS alias] TEXT | TAG | NUMERIC | VECTOR | GEO [ SORTABLE ] [NOINDEX] ...] **Time complexity:** O(K) at creation where K is the number of fields, O(N) if scanning the keyspace is triggered, where N is the number of keys in the keyspace. @@ -43,17 +44,24 @@ after the SCHEMA keyword, declares which fields to index: Field types are: - - `TEXT` - Allows searching for words against the text value in this attribute. + - `TEXT [WITHSUFFIXTRIE]` - Allows searching for words against the text value in this attribute. + * `WITHSUFFIXTRIE` - builds a suffix trie for efficient suffix and infix queries. - - `TAG` - Allows exact-match queries, such as categories or primary keys, against the value in this attribute. + - `TAG [SEPARATOR {char}] [CASESENSITIVE] [WITHSUFFIXTRIE]` - Allows exact-match queries, such as categories or primary keys, against the value in this attribute. + * `SEPARATOR {char}` - indicates how text is split into individual tags. Default is `,`. + * `CASESENSITIVE` - preserve original case for tags. Default is case insensitive. + * `WITHSUFFIXTRIE` - builds a suffix trie for efficient suffix and infix queries. For more information, see [tag fields](https://redis.io/docs/interact/search-and-query/advanced-concepts/tags/). - - `NUMERIC` - Allows numeric range queries against the value in this attribute. + - `NUMERIC [BLOCKSIZE {size}]` - Allows numeric range queries against the value in this attribute. + * `BLOCKSIZE {size}` - block size for the range tree data structure. Default is optimized based on data size. See [query syntax](https://redis.io/docs/interact/search-and-query/query/) for details on how to use numeric ranges. - `VECTOR` - Allows vector similarity queries against the value in this attribute. For more information, see [vector fields](https://redis.io/docs/interact/search-and-query/search/vectors/). + - `GEO` - Allows geographic range queries against the value in this attribute. + :::note About `VECTOR` - Full documentation on vector options is available [here](https://redis.io/docs/interact/search-and-query/advanced-concepts/vectors/). - Currently, Dragonfly has limited support for vector options. @@ -75,6 +83,13 @@ Dragonfly does **not** support sorting without the `SORTABLE` option. - `NOINDEX` - Attributes can have the `NOINDEX` option, which means they will not be indexed. This is useful in conjunction with `SORTABLE`, to create attributes whose update using PARTIAL will not cause full reindexing of the document. If an attribute has NOINDEX and doesn't have SORTABLE, it will just be ignored by the index. +:::note About ignored field options +The following field options are accepted but ignored for compatibility with Redis: +- `UNF`, `NOSTEM` - ignored without arguments +- `WEIGHT`, `PHONETIC` - ignored with their arguments +- `INDEXMISSING`, `INDEXEMPTY` - ignored without warning +::: + ## Optional arguments @@ -99,6 +114,18 @@ Currently, Dragonfly supports only one prefix (i.e., `PREFIX 1 my_prefix`), if t ::: + +
+STOPWORDS count [words...] + +sets the index's stopword list to the given list of stopwords. +Stopwords are common words (like "the", "a", "is") that are typically ignored during indexing and searching. + +If `count` is 0, the index will have no stopwords. +If `count` is greater than 0, the following `count` arguments will be treated as stopwords. + +
+ ## Return `FT.CREATE` returns a simple string reply `OK` if executed correctly, or an error reply otherwise. @@ -119,6 +146,21 @@ Index the `sku` attribute from a hash as both a `TEXT` and as `TAG`: ``` bash dragonfly> FT.CREATE idx ON HASH PREFIX 1 blog:post: SCHEMA sku AS sku_text TEXT sku AS sku_tag TAG SORTABLE +OK +``` + +Create an index with custom TAG separator and case-sensitive matching: + +``` bash +dragonfly> FT.CREATE products_idx ON HASH PREFIX 1 product: SCHEMA categories TAG SEPARATOR "|" CASESENSITIVE SORTABLE price NUMERIC BLOCKSIZE 64 description TEXT WITHSUFFIXTRIE +OK +``` + +Create an index with custom stopwords: + +``` bash +dragonfly> FT.CREATE articles_idx ON HASH PREFIX 1 article: STOPWORDS 3 the a is SCHEMA title TEXT content TEXT NOINDEX +OK ``` diff --git a/docs/command-reference/search/ft.dropindex.md b/docs/command-reference/search/ft.dropindex.md index 0fba929..0afb25a 100644 --- a/docs/command-reference/search/ft.dropindex.md +++ b/docs/command-reference/search/ft.dropindex.md @@ -14,6 +14,10 @@ description: Deletes the index Delete an index. +:::info Future functionality +The `DD` (delete documents) option is planned for future releases but is not currently implemented in Dragonfly. +::: + ## Required arguments
diff --git a/docs/command-reference/search/ft.info.md b/docs/command-reference/search/ft.info.md index e540843..5e242ae 100644 --- a/docs/command-reference/search/ft.info.md +++ b/docs/command-reference/search/ft.info.md @@ -29,7 +29,12 @@ is index name. You must first create the index using [`FT.CREATE`](./ft.create.m Returned values include: - `index_name`: name of the index upon creation by using [`FT.CREATE`](./ft.create.md). -- `fields`: index schema - field names, types, and attributes. +- `index_definition`: contains information about the index configuration, including `key_type` and `prefix`. +- `attributes`: index schema - for each field contains: + - `identifier`: the original field name or JSONPath + - `attribute`: the field alias (or same as identifier if no alias provided) + - `type`: field type (TEXT, TAG, NUMERIC, VECTOR, GEO) + - field-specific options like `SORTABLE` and `NOINDEX` flags when applicable - `num_docs`: Number of documents in the index. ## Examples @@ -47,33 +52,44 @@ OK dragonfly> FT.INFO idx 1) index_name 2) idx -3) fields -4) 1) 1) identifier +3) index_definition +4) 1) key_type + 2) HASH + 3) prefix + 4) blog:post: +5) attributes +6) 1) 1) identifier + 2) description + 3) attribute + 4) description + 5) type + 6) TEXT + 7) NOINDEX + 2) 1) identifier 2) published_at 3) attribute 4) published_at 5) type 6) NUMERIC - 2) 1) identifier + 7) SORTABLE + 8) blocksize + 9) 7000 + 3) 1) identifier 2) title 3) attribute 4) title 5) type 6) TEXT - 3) 1) identifier + 7) SORTABLE + 4) 1) identifier 2) category 3) attribute 4) category 5) type 6) TAG - 4) 1) identifier - 2) description - 3) attribute - 4) description - 5) type - 6) TEXT -5) num_docs -6) (integer) 1 + 7) SORTABLE +7) num_docs +8) (integer) 1 ```
diff --git a/docs/command-reference/search/ft.profile.md b/docs/command-reference/search/ft.profile.md index 44cdb4f..3d0279f 100644 --- a/docs/command-reference/search/ft.profile.md +++ b/docs/command-reference/search/ft.profile.md @@ -6,13 +6,13 @@ description: Uses SEARCH command to collect performance info ## Syntax - FT.PROFILE index SEARCH [LIMITED] QUERY query + FT.PROFILE index SEARCH | AGGREGATE [LIMITED] QUERY query **Time complexity:** O(N) ## Description -Apply the [`FT.SEARCH`](./ft.search.md) command to collect performance details. +Apply the [`FT.SEARCH`](./ft.search.md) or [`FT.AGGREGATE`](./ft.aggregate.md) command to collect performance details. ## Required arguments @@ -22,10 +22,29 @@ Apply the [`FT.SEARCH`](./ft.search.md) command to collect performance details. is index name, created using the [`FT.CREATE`](./ft.create.md) command. +
+SEARCH | AGGREGATE + +specifies which command to profile. +- `SEARCH` profiles a search query +- `AGGREGATE` profiles an aggregation query + +
+ +
+LIMITED + +enables limited profiling mode (planned feature). + +:::info Future functionality +The `LIMITED` option is planned for future releases but full functionality is not currently implemented in Dragonfly. +::: +
+
QUERY query -is query string, sent to the [`FT.SEARCH`](./ft.search.md) command to collect performance details. +is query string, sent to the [`FT.SEARCH`](./ft.search.md) or [`FT.AGGREGATE`](./ft.aggregate.md) command to collect performance details.
## Return @@ -88,7 +107,7 @@ dragonfly> FT.PROFILE idx SEARCH QUERY "@category:{default}" ## See also -[`FT.SEARCH`](./ft.search.md) +[`FT.SEARCH`](./ft.search.md) | [`FT.AGGREGATE`](./ft.aggregate.md) ## Related topics diff --git a/docs/command-reference/search/ft.search.md b/docs/command-reference/search/ft.search.md index 09ad8c3..bfa0cb1 100644 --- a/docs/command-reference/search/ft.search.md +++ b/docs/command-reference/search/ft.search.md @@ -8,10 +8,12 @@ description: Searches the index with a query, returning docs or just IDs FT.SEARCH index query [NOCONTENT] + [LOAD count identifier [AS property] [ identifier [AS property] ...]] [RETURN count identifier [AS property] [ identifier [AS property] ...]] [SORTBY sortby [ ASC | DESC] [WITHCOUNT]] [LIMIT offset num] [PARAMS nargs name value [ name value ...]] + [FILTER field min max] **Time complexity:** O(N) @@ -45,6 +47,20 @@ returns the document IDs and not the content. This is useful if Dragonfly is storing an index on an external document collection. +
+LOAD num identifier AS property ... + +loads specific attributes from documents instead of all content. Similar to `RETURN` but for pre-loading fields during search. + +`num` is the number of attributes following the keyword. +`identifier` is either an attribute name (for Hash and JSON) or a JSONPath expression (for JSON). +`property` is an optional name used in the result. If not provided, the `identifier` is used in the result. + +:::note About `LOAD` vs `RETURN` +`LOAD` and `RETURN` cannot be used together. `LOAD` is used for pre-loading fields, while `RETURN` filters the final output. +::: +
+
RETURN num identifier AS property ... @@ -90,6 +106,17 @@ For example, with parameter definition `PARAMS 4 start 2020 end 2021`, the expre You cannot reference parameters in the query string where concrete values are not allowed, such as in field names, for example, `@published_at`.
+
+FILTER field min max + +applies a numeric filter to the results based on the value of a numeric field. + +`field` is the name of a numeric field in the index. +`min` and `max` define the numeric range (inclusive) that matching documents must have for the specified field. + +Multiple `FILTER` clauses can be used to apply filters on different fields. +
+ ## Return `FT.SEARCH` returns an array reply, where the first element is an integer reply of the total number of results, and then array reply pairs of document IDs, and array replies of attribute/value pairs. @@ -171,6 +198,16 @@ dragonfly> FT.SEARCH books-idx "python" RETURN 3 $.book.price AS price ``` +
+Search with numeric filter + +Search for books with "python" in any TEXT attribute, filtering by price range between 10 and 50. + +``` bash +dragonfly> FT.SEARCH books-idx "python" FILTER price 10 50 +``` +
+ ## See also [`FT.CREATE`](./ft.create.md)