Skip to content

Commit e143066

Browse files
Merge branch 'main' into esql/fix_generative_tests_20250702
2 parents 5cbb64c + 8ffbf4a commit e143066

File tree

7 files changed

+448
-37
lines changed

7 files changed

+448
-37
lines changed

docs/changelog/129725.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 129725
2+
summary: Throw a 400 when sorting for all types of range fields
3+
area: Search
4+
type: bug
5+
issues: []
Lines changed: 264 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,264 @@
1+
---
2+
mapped_pages:
3+
- https://www.elastic.co/guide/en/elasticsearch/reference/8.18/docs-update.html
4+
applies_to:
5+
stack: all
6+
navigation_title: Update a document
7+
---
8+
9+
# Update a document [update-document]
10+
11+
The [Update API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-update) enables you to script document updates, allowing you to update, delete, or skip modifying a document.
12+
13+
The following examples show common use cases, such as incrementing a counter and adding or removing elements from a list, as well as how to:
14+
15+
- [Update part of a document](#update-part-document)
16+
- [Detect noop updates](#detect-noop-updates)
17+
- [Insert or update documents with upsert](#upsert)
18+
- [Simplify upsert with doc_as_upsert](#doc-as-upsert)
19+
20+
First, let's index a simple doc:
21+
22+
```console
23+
PUT test/_doc/1
24+
{
25+
"counter" : 1,
26+
"tags" : ["red"]
27+
}
28+
```
29+
% TESTSETUP
30+
31+
To increment the counter, you can submit an update request with the
32+
following script:
33+
34+
```console
35+
POST test/_update/1
36+
{
37+
"script" : {
38+
"source": "ctx._source.counter += params.count",
39+
"lang": "painless",
40+
"params" : {
41+
"count" : 4
42+
}
43+
}
44+
}
45+
```
46+
47+
Similarly, you could use and update script to add a tag to the list of tags
48+
(this is just a list, so the tag is added even it exists):
49+
50+
```console
51+
POST test/_update/1
52+
{
53+
"script": {
54+
"source": "ctx._source.tags.add(params.tag)",
55+
"lang": "painless",
56+
"params": {
57+
"tag": "blue"
58+
}
59+
}
60+
}
61+
```
62+
63+
You could also remove a tag from the list of tags. The Painless
64+
function to `remove` a tag takes the array index of the element
65+
you want to remove. To avoid a possible runtime error, you first need to
66+
make sure the tag exists. If the list contains duplicates of the tag, this
67+
script just removes one occurrence.
68+
69+
```console
70+
POST test/_update/1
71+
{
72+
"script": {
73+
"source": "if (ctx._source.tags.contains(params.tag)) { ctx._source.tags.remove(ctx._source.tags.indexOf(params.tag)) }",
74+
"lang": "painless",
75+
"params": {
76+
"tag": "blue"
77+
}
78+
}
79+
}
80+
```
81+
82+
You can also add and remove fields from a document. For example, this script
83+
adds the field `new_field`:
84+
85+
```console
86+
POST test/_update/1
87+
{
88+
"script" : "ctx._source.new_field = 'value_of_new_field'"
89+
}
90+
```
91+
92+
Conversely, this script removes the field `new_field`:
93+
94+
```console
95+
POST test/_update/1
96+
{
97+
"script" : "ctx._source.remove('new_field')"
98+
}
99+
```
100+
% TEST[continued]
101+
102+
The following script removes a subfield from an object field:
103+
104+
```console
105+
PUT test/_doc/1?refresh
106+
{
107+
"my-object": {
108+
"my-subfield": true
109+
}
110+
}
111+
```
112+
113+
```console
114+
POST test/_update/1
115+
{
116+
"script": "ctx._source['my-object'].remove('my-subfield')"
117+
}
118+
```
119+
% TEST[continued]
120+
121+
Instead of updating the document, you can also change the operation that is
122+
executed from within the script. For example, this request deletes the doc if
123+
the `tags` field contains `green`, otherwise it does nothing (`noop`):
124+
125+
```console
126+
POST test/_update/1
127+
{
128+
"script": {
129+
"source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = 'delete' } else { ctx.op = 'noop' }",
130+
"lang": "painless",
131+
"params": {
132+
"tag": "green"
133+
}
134+
}
135+
}
136+
```
137+
138+
## Update part of a document [update-part-document]
139+
140+
The following partial update adds a new field to the
141+
existing document:
142+
143+
```console
144+
POST test/_update/1
145+
{
146+
"doc": {
147+
"name": "new_name"
148+
}
149+
}
150+
```
151+
152+
If both `doc` and `script` are specified, then `doc` is ignored. If you
153+
specify a scripted update, include the fields you want to update in the script.
154+
155+
156+
## Detect noop updates [detect-noop-updates]
157+
158+
By default updates that don't change anything detect that they don't change
159+
anything and return `"result": "noop"`:
160+
161+
```console
162+
POST test/_update/1
163+
{
164+
"doc": {
165+
"name": "new_name"
166+
}
167+
}
168+
```
169+
% TEST[continued]
170+
171+
If the value of `name` is already `new_name`, the update
172+
request is ignored and the `result` element in the response returns `noop`:
173+
174+
```console
175+
176+
{
177+
"_shards": {
178+
"total": 0,
179+
"successful": 0,
180+
"failed": 0
181+
},
182+
"_index": "test",
183+
"_id": "1",
184+
"_version": 2,
185+
"_primary_term": 1,
186+
"_seq_no": 1,
187+
"result": "noop"
188+
}
189+
```
190+
191+
You can disable this behavior by setting `"detect_noop": false`:
192+
193+
```console
194+
POST test/_update/1
195+
{
196+
"doc": {
197+
"name": "new_name"
198+
},
199+
"detect_noop": false
200+
}
201+
```
202+
203+
## Insert or update documents with upsert [upsert]
204+
205+
An upsert operation lets you update an existing document or insert a new one if it doesn't exist, in a single request.
206+
207+
In this example, if the product with ID `1` exists, its price will be updated to `100`. If the product does not exist, a new document with ID `1` and a price of `50` will be inserted.
208+
209+
```console
210+
POST /test/_update/1
211+
{
212+
"doc": {
213+
"product_price": 100
214+
},
215+
"upsert": {
216+
"product_price": 50
217+
}
218+
}
219+
```
220+
221+
## Run a scripted upsert [scripted-upsert]
222+
223+
To run the script whether or not the document exists, set `scripted_upsert` to
224+
`true`:
225+
226+
```console
227+
POST test/_update/1
228+
{
229+
"scripted_upsert": true,
230+
"script": {
231+
"source": """
232+
if ( ctx.op == 'create' ) {
233+
ctx._source.counter = params.count
234+
} else {
235+
ctx._source.counter += params.count
236+
}
237+
""",
238+
"params": {
239+
"count": 4
240+
}
241+
},
242+
"upsert": {}
243+
}
244+
```
245+
246+
## Simplify upsert with doc_as_upsert [doc-as-upsert]
247+
248+
Instead of sending a partial `doc` plus an `upsert` doc, you can set
249+
`doc_as_upsert` to `true` to use the contents of `doc` as the `upsert`
250+
value:
251+
252+
```console
253+
POST test/_update/1
254+
{
255+
"doc": {
256+
"name": "new_name"
257+
},
258+
"doc_as_upsert": true
259+
}
260+
```
261+
262+
::::{note}
263+
Using [ingest pipelines](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/ingest.html) with `doc_as_upsert` is not supported.
264+
::::

docs/reference/elasticsearch/toc.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,7 @@ toc:
105105
- file: rest-apis/searching-with-query-rules.md
106106
- file: rest-apis/shard-request-cache.md
107107
- file: rest-apis/term-vectors-examples.md
108+
- file: rest-apis/update-document.md
108109
- file: rest-apis/update-cc-api-key-examples.md
109110
- file: rest-apis/vector-tile-search.md
110111
- file: mapping-reference/index.md

docs/reference/search-connectors/index.md

Lines changed: 49 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,13 @@ mapped_pages:
88
- https://www.elastic.co/guide/en/enterprise-search/current/connectors.html
99
---
1010

11-
# Search connectors
11+
# Content connectors
1212

1313
$$$es-connectors-native$$$
1414

1515

1616
:::{note}
17-
This page is about Search connectors that synchronize third-party data into {{es}}. If you’re looking for Kibana connectors to integrate with services like generative AI model providers, refer to [Kibana Connectors](docs-content://deploy-manage/manage-connectors.md).
17+
This page is about Content connectors that synchronize third-party data into {{es}}. If you’re looking for Kibana connectors to integrate with services like generative AI model providers, refer to [Kibana Connectors](docs-content://deploy-manage/manage-connectors.md).
1818
:::
1919

2020
A _connector_ is an Elastic integration that syncs data from an original data source to {{es}}. Use connectors to create searchable, read-only replicas of your data in {{es}}.
@@ -30,46 +30,60 @@ These connectors are written in Python and the source code is available in the [
3030
As of Elastic 9.0, managed connectors on Elastic Cloud Hosted are no longer available. All connectors must be [self-managed](/reference/search-connectors/self-managed-connectors.md).
3131
::::
3232

33-
34-
Connectors are available for the following third-party data sources:
35-
36-
- [Azure Blob Storage](/reference/search-connectors/es-connectors-azure-blob.md)
37-
- [Box](/reference/search-connectors/es-connectors-box.md)
38-
- [Confluence](/reference/search-connectors/es-connectors-confluence.md)
39-
- [Dropbox](/reference/search-connectors/es-connectors-dropbox.md)
40-
- [GitHub](/reference/search-connectors/es-connectors-github.md)
41-
- [Gmail](/reference/search-connectors/es-connectors-gmail.md)
42-
- [Google Cloud Storage](/reference/search-connectors/es-connectors-google-cloud.md)
43-
- [Google Drive](/reference/search-connectors/es-connectors-google-drive.md)
44-
- [GraphQL](/reference/search-connectors/es-connectors-graphql.md)
45-
- [Jira](/reference/search-connectors/es-connectors-jira.md)
46-
- [MicrosoftSQL](/reference/search-connectors/es-connectors-ms-sql.md)
47-
- [MongoDB](/reference/search-connectors/es-connectors-mongodb.md)
48-
- [MySQL](/reference/search-connectors/es-connectors-mysql.md)
49-
- [Network drive](/reference/search-connectors/es-connectors-network-drive.md)
50-
- [Notion](/reference/search-connectors/es-connectors-notion.md)
51-
- [OneDrive](/reference/search-connectors/es-connectors-onedrive.md)
52-
- [OpenText Documentum](/reference/search-connectors/es-connectors-opentext.md)
53-
- [Oracle](/reference/search-connectors/es-connectors-oracle.md)
54-
- [Outlook](/reference/search-connectors/es-connectors-outlook.md)
55-
- [PostgreSQL](/reference/search-connectors/es-connectors-postgresql.md)
56-
- [Redis](/reference/search-connectors/es-connectors-redis.md)
57-
- [S3](/reference/search-connectors/es-connectors-s3.md)
58-
- [Salesforce](/reference/search-connectors/es-connectors-salesforce.md)
59-
- [ServiceNow](/reference/search-connectors/es-connectors-servicenow.md)
60-
- [SharePoint Online](/reference/search-connectors/es-connectors-sharepoint-online.md)
61-
- [SharePoint Server](/reference/search-connectors/es-connectors-sharepoint.md)
62-
- [Slack](/reference/search-connectors/es-connectors-slack.md)
63-
- [Teams](/reference/search-connectors/es-connectors-teams.md)
64-
- [Zoom](/reference/search-connectors/es-connectors-zoom.md)
33+
This table provides an overview of our available connectors, their current support status, and the features they support.
34+
35+
The columns provide specific information about each connector:
36+
37+
- **Status**: Indicates whether the connector is in General Availability (GA), Technical Preview, Beta, or is an Example connector.
38+
- **Advanced sync rules**: Specifies the versions in which advanced sync rules are supported, if applicable.
39+
- **Local binary extraction service**: Specifies the versions in which the local binary extraction service is supported, if applicable.
40+
- **Incremental syncs**: Specifies the version in which incremental syncs are supported, if applicable.
41+
- **Document level security**: Specifies the version in which document level security is supported, if applicable.
42+
43+
44+
45+
| Connector | Status | [Advanced sync rules](./es-sync-rules.md#es-sync-rules-advanced) | [Local binary extraction service](./es-connectors-content-extraction.md#es-connectors-content-extraction-local) | [Incremental syncs](./content-syncs.md#es-connectors-sync-types-incremental) | [Document level security](./document-level-security.md) | Source code |
46+
| ------- | --------------- | -- | -- | -- | -- | -- |
47+
| [Azure Blob](/reference/search-connectors/es-connectors-azure-blob.md) | **GA** | - | 8.11+ | 8.13+ | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/azure_blob_storage.py) |
48+
| [Box](/reference/search-connectors/es-connectors-box.md) | **Preview** | - | - | 8.13+ | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/box.py) |
49+
| [Confluence Cloud](/reference/search-connectors/es-connectors-confluence.md) | **GA** | 8.9+ | 8.11+ | 8.13+ | 8.10 | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/confluence.py) |
50+
| [Confluence Data Center](/reference/search-connectors/es-connectors-confluence.md) | **Preview** | 8.13+ | 8.13+ | 8.13+ | 8.14+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/confluence.py) |
51+
| [Confluence Server](/reference/search-connectors/es-connectors-confluence.md)| **GA** | 8.9+ | 8.11+ | 8.13+ | 8.14+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/confluence.py) |
52+
| [Dropbox](/reference/search-connectors/es-connectors-dropbox.md)| **GA** | - | 8.11+ | 8.13+ | 8.12+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/dropbox.py) |
53+
| [GitHub](/reference/search-connectors/es-connectors-github.md)| **GA** | 8.10+ | 8.11+ | 8.13+ | 8.12+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/github.py) |
54+
| [Gmail](/reference/search-connectors/es-connectors-gmail.md)| **GA** | - | - | 8.13+ | 8.10+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/gmail.py) |
55+
| [Google Cloud Storage](/reference/search-connectors/es-connectors-google-cloud.md)| **GA** | - | 8.11+ | 8.13+ | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/google_cloud_storage.py) |
56+
| [Google Drive](/reference/search-connectors/es-connectors-google-drive.md)| **GA** | - | 8.11+ | 8.13+ | 8.10+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/google_drive.py) |
57+
| [GraphQL](/reference/search-connectors/es-connectors-graphql.md)| **Preview** | - | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/graphql.py) |
58+
| [Jira Cloud](/reference/search-connectors/es-connectors-jira.md)| **GA** | 8.9+ | 8.11+ | 8.13+ | 8.10+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/jira.py) |
59+
| [Jira Data Center](/reference/search-connectors/es-connectors-jira.md)| **Preview** | 8.13+ | 8.13+ | 8.13+ | 8.13+*| [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/jira.py) |
60+
| [Jira Server](/reference/search-connectors/es-connectors-jira.md)| **GA** | 8.9+ | 8.11+ | 8.13+ | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/jira.py) |
61+
| [Microsoft SQL Server](/reference/search-connectors/es-connectors-ms-sql.md)| **GA** | 8.11+ | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/mssql.py) |
62+
| [MongoDB](/reference/search-connectors/es-connectors-mongodb.md)| **GA** | 8.8 native/ 8.12 self-managed | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/mongo.py) |
63+
| [MySQL](/reference/search-connectors/es-connectors-mysql.md)| **GA** | 8.8+ | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/mysql.py) |
64+
| [Network drive](/reference/search-connectors/es-connectors-network-drive.md)| **GA** | 8.10+ | 8.14+ | 8.13+ | 8.11+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/network_drive.py) |
65+
| [Notion](/reference/search-connectors/es-connectors-notion.md)| **GA** | 8.14+ | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/notion.py) |
66+
| [OneDrive](/reference/search-connectors/es-connectors-onedrive.md)| **GA** | 8.11+ | 8.11+ | 8.13+ | 8.11+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/onedrive.py) |
67+
| [Opentext Documentum](/reference/search-connectors/es-connectors-opentext.md)| **Example** | n/a | n/a | n/a | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/opentext_documentum.py) |
68+
| [Oracle](/reference/search-connectors/es-connectors-oracle.md)| **GA** | - | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/oracle.py) |
69+
| [Outlook](/reference/search-connectors/es-connectors-outlook.md)| **GA** | - | 8.11+ | 8.13+ | 8.14+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/outlook.py) |
70+
| [PostgreSQL](/reference/search-connectors/es-connectors-postgresql.md)| **GA** | 8.11+ | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/postgresql.py) |
71+
| [Redis](/reference/search-connectors/es-connectors-redis.md)| **Preview** | - | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/redis.py) |
72+
| [Amazon S3](/reference/search-connectors/es-connectors-s3.md)| **GA** | 8.12+ | 8.11+ | - | - |[View code](https://github.com/elastic/connectors/tree/main/connectors/sources/s3.py) |
73+
| [Salesforce](/reference/search-connectors/es-connectors-salesforce.md)| **GA** | 8.12+ | 8.11+ | 8.13+ | 8.13+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/salesforce.py) |
74+
| [ServiceNow](/reference/search-connectors/es-connectors-servicenow.md)| **GA** | 8.10+ | 8.11+ | 8.13+ | 8.13+ | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/servicenow.py) |
75+
| [Sharepoint Online](/reference/search-connectors/es-connectors-sharepoint-online.md)| **GA** | 8.9+ | 8.9+ | 8.9+ | 8.9+ |[View code](https://github.com/elastic/connectors/tree/main/connectors/sources/sharepoint_online.py) |
76+
| [Sharepoint Server](/reference/search-connectors/es-connectors-sharepoint.md)| **Beta** | - | 8.11+ | 8.13+ | 8.15+ |[View code](https://github.com/elastic/connectors/tree/main/connectors/sources/sharepoint_server.py) |
77+
| [Slack](/reference/search-connectors/es-connectors-slack.md)| **Preview** | - | - | - | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/slack.py) |
78+
| [Teams](/reference/search-connectors/es-connectors-teams.md)| **Preview** | - | - | 8.13+ | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/teams.py) |
79+
| [Zoom](/reference/search-connectors/es-connectors-zoom.md)| **Preview** | - | 8.11+ | 8.13+ | - | [View code](https://github.com/elastic/connectors/tree/main/connectors/sources/zoom.py) |
6580

6681
:::{tip}
6782
Because prerequisites and configuration details vary by data source, you’ll need to refer to the individual connector references for specific details.
6883
:::
6984

7085
## Overview
7186

72-
7387
Because connectors are self-managed on your own infrastructure, they run outside of your Elastic deployment.
7488

7589
You can run them from source or in a Docker container.

docs/reference/search-connectors/toc.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
project: 'Search connectors reference'
1+
project: 'Content connectors reference'
22
toc:
33
- file: index.md
44
- file: connector-reference.md

0 commit comments

Comments
 (0)