Skip to content

Commit e4d775d

Browse files
[9.0] [DOCS] Fixes formatting issue on dense vector reference page. (#127214) (#127341)
* [DOCS] Fixes formatting issue on dense vector reference page. (#127214) (cherry picked from commit 1e7c6ab) # Conflicts: # docs/reference/elasticsearch/mapping-reference/dense-vector.md * delete rescore_vector info— 9.1.0 only --------- Co-authored-by: István Zoltán Szabó <[email protected]>
1 parent 275d640 commit e4d775d

File tree

1 file changed

+4
-18
lines changed

1 file changed

+4
-18
lines changed

docs/reference/elasticsearch/mapping-reference/dense-vector.md

Lines changed: 4 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ mapped_pages:
66

77
# Dense vector field type [dense-vector]
88

9-
109
The `dense_vector` field type stores dense vectors of numeric values. Dense vector fields are primarily used for [k-nearest neighbor (kNN) search](docs-content://deploy-manage/production-guidance/optimize-performance/approximate-knn-search.md).
1110

1211
The `dense_vector` type does not support aggregations or sorting.
@@ -46,7 +45,6 @@ PUT my-index/_doc/2
4645
Unlike most other data types, dense vectors are always single-valued. It is not possible to store multiple values in one `dense_vector` field.
4746
::::
4847

49-
5048
## Index vectors for kNN search [index-vectors-knn-search]
5149

5250
A *k-nearest neighbor* (kNN) search finds the *k* nearest vectors to a query vector, as measured by a similarity metric.
@@ -78,7 +76,6 @@ PUT my-index-2
7876
Indexing vectors for approximate kNN search is an expensive process. It can take substantial time to ingest documents that contain vector fields with `index` enabled. See [k-nearest neighbor (kNN) search](docs-content://deploy-manage/production-guidance/optimize-performance/approximate-knn-search.md) to learn more about the memory requirements.
7977
::::
8078

81-
8279
You can disable indexing by setting the `index` parameter to `false`:
8380

8481
```console
@@ -98,7 +95,6 @@ PUT my-index-2
9895

9996
{{es}} uses the [HNSW algorithm](https://arxiv.org/abs/1603.09320) to support efficient kNN search. Like most kNN algorithms, HNSW is an approximate method that sacrifices result accuracy for improved speed.
10097

101-
10298
## Automatically quantize vectors for kNN search [dense-vector-quantization]
10399

104100
The `dense_vector` type supports quantization to reduce the memory footprint required when [searching](docs-content://solutions/search/vector/knn.md#approximate-knn) `float` vectors. The three following quantization strategies are supported:
@@ -117,17 +113,14 @@ Quantized vectors can use [oversampling and rescoring](docs-content://solutions/
117113
Quantization will continue to keep the raw float vector values on disk for reranking, reindexing, and quantization improvements over the lifetime of the data. This means disk usage will increase by ~25% for `int8`, ~12.5% for `int4`, and ~3.1% for `bbq` due to the overhead of storing the quantized and raw vectors.
118114
::::
119115

120-
121116
::::{note}
122117
`int4` quantization requires an even number of vector dimensions.
123118
::::
124119

125-
126120
::::{note}
127121
`bbq` quantization only supports vector dimensions that are greater than 64.
128122
::::
129123

130-
131124
Here is an example of how to create a byte-quantized index:
132125

133126
```console
@@ -188,7 +181,6 @@ PUT my-byte-quantized-index
188181
}
189182
```
190183

191-
192184
## Parameters for dense vector fields [dense-vector-params]
193185

194186
The following mapping parameters are accepted:
@@ -210,7 +202,6 @@ $$$dense-vector-element-type$$$
210202

211203
::::
212204

213-
214205
`dims`
215206
: (Optional, integer) Number of vector dimensions. Can’t exceed `4096`. If `dims` is not specified, it will be set to the length of the first vector added to the field.
216207

@@ -228,7 +219,6 @@ $$$dense-vector-similarity$$$
228219
`bit` vectors only support `l2_norm` as their similarity metric.
229220
::::
230221

231-
232222
::::{dropdown} Valid values for `similarity`
233223
`l2_norm`
234224
: Computes similarity based on the L2 distance (also known as Euclidean distance) between the vectors. The document `_score` is computed as `1 / (1 + l2_norm(query, vector)^2)`.
@@ -242,7 +232,6 @@ For `bit` vectors, instead of using `l2_norm`, the `hamming` distance between th
242232

243233
When `element_type` is `byte`, all vectors must have the same length including both document and query vectors or results will be inaccurate. The document `_score` is computed as `0.5 + (dot_product(query, vector) / (32768 * dims))` where `dims` is the number of dimensions per vector.
244234

245-
246235
`cosine`
247236
: Computes the cosine similarity. During indexing {{es}} automatically normalizes vectors with `cosine` similarity to unit length. This allows to internally use `dot_product` for computing similarity, which is more efficient. Original un-normalized vectors can be still accessed through scripts. The document `_score` is computed as `(1 + cosine(query, vector)) / 2`. The `cosine` similarity does not allow vectors with zero magnitude, since cosine is not defined in this case.
248237

@@ -251,12 +240,10 @@ For `bit` vectors, instead of using `l2_norm`, the `hamming` distance between th
251240

252241
::::
253242

254-
255243
::::{note}
256244
Although they are conceptually related, the `similarity` parameter is different from [`text`](/reference/elasticsearch/mapping-reference/text.md) field [`similarity`](/reference/elasticsearch/mapping-reference/similarity.md) and accepts a distinct set of options.
257245
::::
258246

259-
260247
$$$dense-vector-index-options$$$
261248

262249
`index_options`
@@ -267,7 +254,6 @@ $$$dense-vector-index-options$$$
267254
::::{dropdown} Properties of `index_options`
268255
`type`
269256
: (Required, string) The type of kNN algorithm to use. Can be either any of:
270-
271257
* `hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) for scalable approximate kNN search. This supports all `element_type` values.
272258
* `int8_hnsw` - The default index type for float vectors. This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 4x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
273259
* `int4_hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 8x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
@@ -276,17 +262,17 @@ $$$dense-vector-index-options$$$
276262
* `int8_flat` - This utilizes a brute-force search algorithm in addition to automatically scalar quantization. Only supports `element_type` of `float`.
277263
* `int4_flat` - This utilizes a brute-force search algorithm in addition to automatically half-byte scalar quantization. Only supports `element_type` of `float`.
278264
* `bbq_flat` - This utilizes a brute-force search algorithm in addition to automatically binary quantization. Only supports `element_type` of `float`.
279-
280-
265+
281266
`m`
282267
: (Optional, integer) The number of neighbors each node will be connected to in the HNSW graph. Defaults to `16`. Only applicable to `hnsw`, `int8_hnsw`, `int4_hnsw` and `bbq_hnsw` index types.
283-
268+
284269
`ef_construction`
285270
: (Optional, integer) The number of candidates to track while assembling the list of nearest neighbors for each new node. Defaults to `100`. Only applicable to `hnsw`, `int8_hnsw`, `int4_hnsw` and `bbq_hnsw` index types.
286-
271+
287272
`confidence_interval`
288273
: (Optional, float) Only applicable to `int8_hnsw`, `int4_hnsw`, `int8_flat`, and `int4_flat` index types. The confidence interval to use when quantizing the vectors. Can be any value between and including `0.90` and `1.0` or exactly `0`. When the value is `0`, this indicates that dynamic quantiles should be calculated for optimized quantization. When between `0.90` and `1.0`, this value restricts the values used when calculating the quantization thresholds. For example, a value of `0.95` will only use the middle 95% of the values when calculating the quantization thresholds (e.g. the highest and lowest 2.5% of values will be ignored). Defaults to `1/(dims + 1)` for `int8` quantized vectors and `0` for `int4` for dynamic quantile calculation.
289274

275+
:::::
290276
::::
291277

292278

0 commit comments

Comments
 (0)