Skip to content

Commit fb96c4f

Browse files
authored
Merge branch 'main' into refactor_sparsevectorquerybuilder
2 parents d9d4397 + e522df6 commit fb96c4f

File tree

168 files changed

+5573
-975
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

168 files changed

+5573
-975
lines changed

docs/changelog/134320.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 134320
2+
summary: Add CHUNK function
3+
area: ES|QL
4+
type: enhancement
5+
issues: []

docs/reference/elasticsearch/mapping-reference/pattern-text.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,14 +46,16 @@ In both cases, all queries return a constant score of 1.0.
4646

4747
## Index sorting for improved compression
4848
The compression provided by `pattern_text` can be significantly improved if the index is sorted by the `template_id` field.
49-
For example, a typical approach would be to sort first by `message.template_id`, then by `@timestamp`, as shown in the following example.
49+
This sorting is not applied by default, but can be enabled for the `message` field of LogsDB indices (assuming it is of type `pattern_text`) by setting the index setting `index.logsdb.default_sort_on_message_template` to `true`.
50+
This will cause the index to be sorted by `host.name` (if present), then `message.template_id`, and finally by `@timestamp`.
51+
If the index is not LogsDB or the `pattern_text` field is named something other than `message`, index sorting can still be manually applied as shown in the following example.
5052

5153
```console
5254
PUT logs
5355
{
5456
"settings": {
5557
"index": {
56-
"sort.field": [ "message.template_id", "@timestamp" ],
58+
"sort.field": [ "notice.template_id", "@timestamp" ],
5759
"sort.order": [ "asc", "desc" ]
5860
}
5961
},
@@ -62,7 +64,7 @@ PUT logs
6264
"@timestamp": {
6365
"type": "date"
6466
},
65-
"message": {
67+
"notice": {
6668
"type": "pattern_text"
6769
}
6870
}

docs/reference/elasticsearch/mapping-reference/semantic-text.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -424,20 +424,20 @@ POST test-index/_search
424424

425425
## Updates and partial updates for `semantic_text` fields [semantic-text-updates]
426426

427-
When updating documents that contain `semantic_text` fields, its important to understand how inference is triggered:
427+
When updating documents that contain `semantic_text` fields, it's important to understand how inference is triggered:
428428

429-
* **Full document updates**
430-
When you perform a full document update, **all `semantic_text` fields will re-run inference** even if their values did not change. This ensures that the embeddings are always consistent with the current document state but can increase ingestion costs.
429+
Full document updates
430+
: Full document updates re-run inference on all `semantic_text` fields, even if their values did not change. This ensures that embeddings remain consistent with the current document state but can increase ingestion costs.
431431

432-
* **Partial updates using the Bulk API**
433-
Partial updates that **omit `semantic_text` fields** and are submitted through the [Bulk API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) will **reuse the existing embeddings** stored in the index. In this case, inference is **not triggered** for fields that were not updated, which can significantly reduce processing time and cost.
432+
Partial updates using the Bulk API
433+
: Partial updates submitted through the [Bulk API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) reuse existing embeddings when you omit `semantic_text` fields. Inference does not run for omitted fields, which can significantly reduce processing time and cost.
434434

435-
* **Partial updates using the Update API**
436-
When using the [Update API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-update) with a `doc` object that **omits `semantic_text` fields**, inference **will still run** on all `semantic_text` fields. This means that even if the field values are not changed, embeddings will be re-generated.
435+
Partial updates using the Update API
436+
: Partial updates submitted through the [Update API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-update) re-run inference on all `semantic_text` fields, even when you omit them from the `doc` object. Embeddings are re-generated regardless of whether field values changed.
437437

438-
If you want to avoid unnecessary inference and keep existing embeddings:
438+
To preserve existing embeddings and avoid unnecessary inference costs:
439439

440-
* Use **partial updates through the Bulk API**.
440+
* Use partial updates with the Bulk API.
441441
* Omit any `semantic_text` fields that did not change from the `doc` object in your request.
442442

443443
### Scripted updates

docs/reference/query-languages/esql/_snippets/functions/description/chunk.md

Lines changed: 10 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/query-languages/esql/_snippets/functions/examples/chunk.md

Lines changed: 22 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/query-languages/esql/_snippets/functions/examples/v_cosine.md

Lines changed: 12 additions & 12 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/query-languages/esql/_snippets/functions/examples/v_dot_product.md

Lines changed: 13 additions & 13 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/query-languages/esql/_snippets/functions/examples/v_hamming.md

Lines changed: 13 additions & 13 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/query-languages/esql/_snippets/functions/examples/v_l1_norm.md

Lines changed: 3 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/reference/query-languages/esql/_snippets/functions/examples/v_l2_norm.md

Lines changed: 3 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)