Skip to content

Replace usages of _source.mode in documentation #114743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 16, 2024
Merged

Replace usages of _source.mode in documentation #114743

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
02cee8b
fix: replace usages of _source.mode in documentation
salvatore-campagna Oct 14, 2024
c1efeb2
Merge branch 'main' into doc-source-mode-setting
elasticmachine Oct 16, 2024
5fb607b
doc: binary mapper uses IndexMode.isSyntheticSourceEnabled
salvatore-campagna Oct 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions docs/plugins/mapper-annotated-text.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,16 @@ duplicates removed. So:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"text": {
"type": "annotated_text",
Expand Down Expand Up @@ -215,8 +223,16 @@ are preserved.
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"text": { "type": "annotated_text", "store": true }
}
Expand Down
28 changes: 19 additions & 9 deletions docs/reference/mapping/fields/synthetic-source.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
==== Synthetic `_source`

IMPORTANT: Synthetic `_source` is Generally Available only for TSDB indices
(indices that have `index.mode` set to `time_series`). For other indices
(indices that have `index.mode` set to `time_series`). For other indices,
synthetic `_source` is in technical preview. Features in technical preview may
be changed or removed in a future release. Elastic will work to fix
any issues, but features in technical preview are not subject to the support SLA
Expand All @@ -11,15 +11,19 @@ of official GA features.
Though very handy to have around, the source field takes up a significant amount
of space on disk. Instead of storing source documents on disk exactly as you
send them, Elasticsearch can reconstruct source content on the fly upon retrieval.
Enable this by setting `mode: synthetic` in `_source`:
Enable this by using the value `synthetic` for the index setting `index.mapping.source.mode`:

[source,console,id=enable-synthetic-source-example]
----
PUT idx
{
"mappings": {
"_source": {
"mode": "synthetic"
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
}
}
Expand All @@ -38,7 +42,7 @@ properties when used with synthetic `_source`.
<<synthetic-source-fields-native-list, Most field types>> construct synthetic `_source` using existing data, most
commonly <<doc-values,`doc_values`>> and <<stored-fields, stored fields>>. For these field types, no additional space
is needed to store the contents of `_source` field. Due to the storage layout of <<doc-values,`doc_values`>>, the
generated `_source` field undergoes <<synthetic-source-modifications, modifications>> compared to original document.
generated `_source` field undergoes <<synthetic-source-modifications, modifications>> compared to the original document.

For all other field types, the original value of the field is stored as is, in the same way as the `_source` field in
non-synthetic mode. In this case there are no modifications and field data in `_source` is the same as in the original
Expand Down Expand Up @@ -227,10 +231,16 @@ For instance:
----
PUT idx_keep
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": {
"mode": "synthetic"
},
"properties": {
"path": {
"type": "object",
Expand Down
10 changes: 9 additions & 1 deletion docs/reference/mapping/types/aggregate-metric-double.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,8 +267,16 @@ For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"agg_metric": {
"type": "aggregate_metric_double",
Expand Down
44 changes: 26 additions & 18 deletions docs/reference/mapping/types/binary.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,24 +63,32 @@ be changed or removed in a future release. Elastic will work to fix
any issues, but features in technical preview are not subject to the support SLA
of official GA features.

`binary` fields support <<synthetic-source,synthetic `_source`>> only when <<doc-values,`doc_values`>> are enabled. Synthetic source always sorts `binary` values in order of their byte representation. For example:
[source,console,id=synthetic-source-binary-example]
----
PUT idx
{
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"binary": { "type": "binary", "doc_values": true }
}
}
}
PUT idx/_doc/1
{
"binary": ["IAA=", "EAA="]
}
----
// TEST[s/$/\nGET idx\/_doc\/1?filter_path=_source\n/]
`binary` fields support <<synthetic-source,synthetic `_source`>> only when <<doc-values,`doc_values`>> are enabled. Synthetic source always sorts `binary` values in order of their byte representation. For example:
[source,console,id=synthetic-source-binary-example]
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"properties": {
"binary": { "type": "binary", "doc_values": true }
}
}
}
PUT idx/_doc/1
{
"binary": ["IAA=", "EAA="]
}
----
// TEST[s/$/\nGET idx\/_doc\/1?filter_path=_source\n/]

Will become:
[source,console-result]
Expand Down
10 changes: 9 additions & 1 deletion docs/reference/mapping/types/boolean.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,8 +249,16 @@ Synthetic source always sorts `boolean` fields. For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"bool": { "type": "boolean" }
}
Expand Down
12 changes: 10 additions & 2 deletions docs/reference/mapping/types/date.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ The following parameters are accepted by `date` fields:
<<ignore-malformed,`ignore_malformed`>>::

If `true`, malformed numbers are ignored. If `false` (default), malformed
numbers throw an exception and reject the whole document. Note that this
numbers throw an exception and reject the whole document. Note that this
cannot be set if the `script` parameter is used.

<<mapping-index,`index`>>::
Expand Down Expand Up @@ -248,8 +248,16 @@ Synthetic source always sorts `date` fields. For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"date": { "type": "date" }
}
Expand Down
10 changes: 9 additions & 1 deletion docs/reference/mapping/types/date_nanos.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,16 @@ Synthetic source always sorts `date_nanos` fields. For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"date": { "type": "date_nanos" }
}
Expand Down
30 changes: 27 additions & 3 deletions docs/reference/mapping/types/flattened.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,8 +334,16 @@ For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"flattened": { "type": "flattened" }
}
Expand Down Expand Up @@ -367,8 +375,16 @@ For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"flattened": { "type": "flattened" }
}
Expand Down Expand Up @@ -407,8 +423,16 @@ For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully understand why this failures happens for flattened field doc tests: https://buildkite.com/elastic/elasticsearch-pull-request/builds/36289#01928be4-ffbb-4def-8006-256e92e1e5d5/6-13640

foo is returned in an extra array. Strange that this failure happens with a doc change.

"properties": {
"flattened": { "type": "flattened" }
}
Expand Down
10 changes: 9 additions & 1 deletion docs/reference/mapping/types/geo-point.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,8 +229,16 @@ longitude) and reduces them to their stored precision. For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"point": { "type": "geo_point" }
}
Expand Down
10 changes: 9 additions & 1 deletion docs/reference/mapping/types/ip.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,16 @@ Synthetic source always sorts `ip` fields and removes duplicates. For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"ip": { "type": "ip" }
}
Expand Down
30 changes: 27 additions & 3 deletions docs/reference/mapping/types/keyword.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,8 +188,16 @@ For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"kwd": { "type": "keyword" }
}
Expand Down Expand Up @@ -218,8 +226,16 @@ are preserved. For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"kwd": { "type": "keyword", "store": true }
}
Expand Down Expand Up @@ -248,8 +264,16 @@ For example:
----
PUT idx
{
"settings": {
"index": {
"mapping": {
"source": {
"mode": "synthetic"
}
}
}
},
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"kwd": { "type": "keyword", "ignore_above": 3 }
}
Expand Down
Loading
Loading