Merge pull request #829 from KarthikSubbarao/1.2

Merge from main into 1.2 branch

Author: KarthikSubbarao

.github/benchmark_configs/fts-benchmarks-arm.json

@@ -12,10 +12,8 @@ Example
 
 ```
 ft._list
-1) b
-2) a
-3) x
-4) index
-5) aa
-6) bb
-```
+1) index
+2) products
+3) users
+4) transactions
+```

docs/commands/ft._list.md

@@ -73,18 +73,11 @@ If the MAX clause is present, then the output is trimmed after the first N recor
 The `GROUPBY` stage organizes the input records into buckets based on the values of the specified fields.
 For each unique combination of values a separate bucket is created to hold all records that have that combination of values.
 
-Each bucket of records is processed into a single output record, discarding the bucket contents. That output record has two sections. The first section has one value for each of the specified groupby fields. This section provides the values that formed (named) this unique bucket.
-
-The second section is the output of the reducers for that bucket. Reducers provide an efficient mechanism for reducing (summarizing) the contents of a bucket. Each reducer function processes each record of the bucket and generates a single output value which is inserted into the second section of groupby output record for this bucket.
-
-The `GROUPBY` stage organizes the input records into buckets based on the values of the specified fields.
-For each unique combination of values a separate bucket is created to hold all records that have that combination of values.
-
 Each bucket of records is processed into a single output record, discarding the bucket contents. That output record has two sections. The first section has one value for each of the specified `GROUPBY` fields. This section provides the values that formed (named) this unique bucket.
 
 The second section is the output of the reducers for that bucket. Reducers provide an efficient mechanism for reducing (summarizing) the contents of a bucket. Each reducer function processes each record of the bucket and generates a single output value which is inserted into the second section of `GROUPBY` output record for this bucket.
 
-The output of the `GROUPBY` stage is one record for each unique bucket.
+The output of the `GROUPBY`stage is one record for each unique bucket.
 
 ### Reducers
 

docs/commands/ft.aggregate.md

@@ -37,13 +37,13 @@ FT.CREATE <index-name>
 
 - `LANGUAGE <language>` (optional): For text fields, the language used to control lexical parsing and stemming. Currently only the value `ENGLISH` is supported.
 
-- `MINSTEMSIZE <min_stem_size>` (optional): For text fields with stemming enabled. This controls the minimum length of a word before it is subjected to stemming. The default value is 4.
+- `MINSTEMSIZE <min_stem_size>` (optional): For text fields with stemming enabled. This controls the minimum length of a word required for it to be subjected to stemming. The default value is 4.
 
 - `WITHOFFSETS | NOOFFSETS` (optional): Enables/Disables the retention of per-word offsets within a text field. Offsets are required to perform exact phrase matching and slop-based proximity matching. Thus if offsets are disabled, those query operations will be rejected with an error. The default is `WITHOFFSETS`.
 
-- `NOSTOPWORDS | STOPWORDS <count> <word1> <word2>...` (optional): Stop words are words which are not put into the indexes. The default value of `STOPWORDS`is language dependent. For`LANGUAGE ENGLISH` the default is: <?>.
+- `NOSTOPWORDS | STOPWORDS <count> <word1> <word2>...` (optional): Stop words are words which are not put into the indexes. The default value of `STOPWORDS`is language dependent. For`LANGUAGE ENGLISH` the default is: [a, an, and, are, as, at, be, but, by, for, if, in, into, is, it, no, not, of, on, or, such, that, their, then, there, these, they, this, to, was, will, with].
 
-- `PUNCTUATION <punctuation>` (optional): A string of characters that are used to define words in the text field. The default value is `,.<>{}[]"':;!@#$%^&\*()-+=~/\|`.
+- `PUNCTUATION <punctuation>` (optional): A string of characters that define the separation points between words, in addition to whitespace characters (spaces, tabs, newlines, carriage returns, and control characters) which always break words. The default value is `,.<>{}[]"':;!@#$%^&\*()-+=~/\|?`.
 
 - `SKIPINITIALSCAN` (optional): If specified, this option skips the normal backfill operation for an index. If this option is specified, pre-existing keys which match the `PREFIX` clause will not be loaded into the index during a backfill operation. This clause has no effect on processing of key mutations _after_ an index is created, i.e., keys which are mutated after an index is created and satisfy the data type and `PREFIX` clause will be inserted into that index.
 

docs/commands/ft.create.md

@@ -40,17 +40,16 @@ An array of key value pairs.
   - Type-specific extension (see below)
 - `num_docs` (integer) Total keys in the index
 - `num_records` (integer) Total number of fields indexed.
-- `num_total_terms` (integer) Total number of terms in all text fields in this index.
-- `num_unique_terms` (integer) Total number of unique terms in all text fields in this index.
-- `total_postings` (integer) Total number of postings entries in all text fields in this index.
+- `total_term_occurrences` (integer) Total number of terms in all text fields in this index.
+- `num_terms` (integer) Total number of unique terms in all text fields in this index.
 - `hash_indexing_failures` (integer) Count of unsuccessful indexing attempts
 - `backfill_in_progress` (string). "1" if a backfill is currently running. "0" if not.
 - `backfill_complete_percent` (string) Estimated progress of background indexing. Percentage is expressed as a fractional value from 0 to 1.0.
 - `mutation_queue_size` (string) Number of keys contained in the mutation queue.
 - `recent_mutations_queue_delay` (string) 0 if the mutation queue is empty. Otherwise it is the mutation queue occupancy of the of the last key to be ingested in seconds.
-- `state` (string) Backfill state. `ready` indicates not backfill is in progress. `backfill_in_progress` backfill operation proceeding normally. `backfill_paused_by_oom` backfill is paused because the Valkey instance is out of memory.
+- `state` (string) Current backfill state. `ready` indicates not backfill is in progress. `backfill_in_progress` backfill operation proceeding normally. `backfill_paused_by_oom` backfill is paused because the Valkey instance is out of memory.
 - `punctuation` (string) list of punctuation characters.
-- `stopwords` (array of strings) list of stopwords.
+- `stopwords` (array of strings) list of `stopwords`.
 - `with_offsets` (string) "1" if offsets are included. "0" if offsets are not included
 - `min_stem_size` (integer) Minimum stemming size for this field.
 

docs/commands/ft.info.md

@@ -22,17 +22,17 @@ FT.SEARCH <index> <query>
 - `ALLSHARDS` (Optional): If specified, the command is terminated with a timeout error if a valid response from all shards is not received within the timeout interval. This is the default.
 - `CONSISTENT` (Optional): If specified, the command is terminated with an error if the cluster is in an inconsistent state. This is the default.
 - `DIALECT <dialect>` (optional): Specifies your dialect. The only supported dialect is 2.
-- `INORDER` (optional): Indicates that proximity matching of terms must be inorder.
 - `INCONSISTENT` (Optional): If specified, the command will generate a best-effort reply if the cluster remains inconsistent within the timeout interval.
 - `LIMIT <offset> <count>` (optional): Lets you choose a portion of the result. The first `<offset>` keys are skipped and only a maximum of `<count>` keys are included. The default is LIMIT 0 10, which returns at most 10 keys.
 - `NOCONTENT` (optional): When present, only the resulting key names are returned, no key values are included.
 - `PARAMS <count> <name> <value> [<name> <value> ...]` (optional): `count` is of the number of arguments, i.e., twice the number of value/name pairs. [Search - query language](../topics/search-query.md) for details.
 - `RETURN <count> <field> [AS <name>] <field> [AS <name>] ...` (options): `count` is the number of fields to return. Specifies the fields you want to retrieve from your documents, along with any renaming for the returned values. By default, all fields are returned unless the `NOCONTENT` option is set, in which case no fields are returned. If num is set to 0, it behaves the same as `NOCONTENT`.
-- `SLOP <slop>` (Optional): Specifies a slop value for proximity matching of terms.
+- `INORDER` (optional): Indicates that proximity matching of text terms in the query must be in order.
+- `SLOP <slop>` (Optional): Specifies a slop value for proximity matching of text terms in the query.
+- `VERBATIM` (Optional): If specified, stemming is not applied to text terms in the query.
 - `SOMESHARDS` (Optional): If specified, the command will generate a best-effort reply if all shards have not responded within the timeout interval.
 - `SORTBY <field> [ASC | DESC]` (Optional): If present, results are sorted according the value of the specified field and the optional sort-direction instruction. By default, vector results are sorted in distance order and non-vector results are not sorted in any particular order. Sorting is applied before the `LIMIT` clause is applied.
 - `TIMEOUT <timeout>` (optional): Lets you set a timeout value for the search command. This must be an integer in milliseconds.
-- `VERBATIM` (Optional): If specified stemming is not applied to term searches.
 - `WITHSORTKEYS` (Optional): If `SORTBY` is specified then enabling this option augments the output with the value of the field used for sorting.
 
 Response

docs/commands/ft.search.md

@@ -1,6 +1,6 @@
 ---
-title: "Search - Configuration"
-description: Search Module Configurable Settings
+title: "Valkey Search - Configuration"
+description: Valkey Search Module Configurable Settings
 ---
 
 # Configurables

docs/topics/search-configurables.md

@@ -1,7 +1,4 @@
 ---
-title: "Search - Data Formats"
-description: Search Module Formats for Input Field Data
----
 title: "Valkey Search - Data Formats"
 description: Valkey Search Module Formats for Input Field Data
 ---
@@ -260,15 +257,15 @@ Removed stop words do not occupy a position in the token sequence. For example,
 
 ### Stemming
 
-Stemming reduces words to their root form so that morphological variants match each other. For example, "running", "runs", and "runner" all stem to "run". The stemming algorithm is language-specific; currently only English (Snowball stemmer) is supported.
+Stemming reduces words to their root form so that morphological variants match each other. For example, "running", "runs", and "run" all have the same stem: "run". The stemming algorithm is language-specific; currently only English (Snowball stemmer) is supported.
 
 Stemming is controlled by these options:
 
 - `LANGUAGE ENGLISH` (schema-level): Specifies the stemming language. Currently only `ENGLISH` is supported.
 - `NOSTEM` (per-field): Disables stemming for a specific text field.
 - `MINSTEMSIZE <size>` (schema-level): Words shorter than this length are not stemmed. The default is 4.
 
-When stemming is enabled, the original word (not the stemmed form) is stored in the index. The stem mapping is recorded separately so that a search for a stemmed root expands to match all words that share that root.
+When stemming is enabled, the original word (not the stemmed form) is stored in the index. The stem mapping is recorded separately so that a search on a term expands to match all terms that share the same stem, as well as the stem itself.
 
 ## Text Ingestion Examples
 
@@ -309,4 +306,4 @@ Given English stemming with the default `MINSTEMSIZE` of 4 and input `"The Runni
 | Indexed tokens          | `running` (position 0), `searches` (position 1), `cat` (position 2) |
 | Stem mappings           | `run` -> {`running`}, `search` -> {`searches`}       |
 
-The word `the` is removed as a stop word. The word `cat` (3 characters) is below the minimum stem size and is not stemmed. A search for `run` will match `running` through the stem mapping.
+The word `the` is removed as a stop word. The word `cat` (3 characters) is below the minimum stem size and is not stemmed. A search for `run` or `runs` will match `running` through the stem mapping.

docs/topics/search-data-formats.md

@@ -1,6 +1,6 @@
 ---
-title: "Search Aggregation Expressions"
-description: Aggregation Expression Language
+title: "Valkey Search - Aggregation Expressions"
+description: Valkey Search Module Aggregation Expression Language
 ---
 
 The `FILTER`, `APPLY`, `SORTBY` and `GROUPBY` stages of `FT.AGGREGATE` use expressions to compute values.

docs/topics/search-expressions.md

@@ -1,6 +1,6 @@
 ---
-title: "Search - Info Fields"
-description: Search Module INFO Fields
+title: "Valkey Search - Info Fields"
+description: Valkey Search Module INFO Fields
 ---
 
 # INFO Fields