Skip to content

Commit 64b2bd5

Browse files
parkertimminsparkertimmins
committed
Split text family types into separate docs pages
1 parent 5fd13e8 commit 64b2bd5

File tree

5 files changed

+145
-12
lines changed

5 files changed

+145
-12
lines changed

‎docs/reference/elasticsearch/mapping-reference/field-data-types.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,8 +77,8 @@ Dates
7777

7878
### Text search types [text-search-types]
7979

80-
[`text` fields](/reference/elasticsearch/mapping-reference/text.md)
81-
: The text family, including `text` and `match_only_text`. Analyzed, unstructured text.
80+
[`text` fields](/reference/elasticsearch/mapping-reference/text-type-family.md)
81+
: The text family, including `text`, `match_only_text`, and `pattern_text`. Analyzed, unstructured text.
8282

8383
[`annotated-text`](/reference/elasticsearch-plugins/mapper-annotated-text.md)
8484
: Text containing special markup. Used for identifying named entities.

‎docs/reference/elasticsearch/mapping-reference/match-only-text.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
navigation_title: "Match Only Text"
3+
mapped_pages:
4+
- https://www.elastic.co/guide/en/elasticsearch/reference/current/match-only-text.html
5+
---
6+
7+
## Match-only text field type [match-only-text-field-type]
8+
9+
A variant of [`text`](#text-field-type) that trades scoring and efficiency of positional queries for space efficiency. This field effectively stores data the same way as a `text` field that only indexes documents (`index_options: docs`) and disables norms (`norms: false`). Term queries perform as fast if not faster as on `text` fields, however queries that need positions such as the [`match_phrase` query](/reference/query-languages/query-dsl/query-dsl-match-query-phrase.md) perform slower as they need to look at the `_source` document to verify whether a phrase matches. All queries return constant scores that are equal to 1.0.
10+
11+
Analysis is not configurable: text is always analyzed with the [default analyzer](docs-content://manage-data/data-store/text-analysis/specify-an-analyzer.md#specify-index-time-default-analyzer) ([`standard`](/reference/text-analysis/analysis-standard-analyzer.md) by default).
12+
13+
[span queries](/reference/query-languages/query-dsl/span-queries.md) are not supported with this field, use [interval queries](/reference/query-languages/query-dsl/query-dsl-intervals-query.md) instead, or the [`text`](#text-field-type) field type if you absolutely need span queries.
14+
15+
Other than that, `match_only_text` supports the same queries as `text`. And like `text`, it does not support sorting and has only limited support for aggregations.
16+
17+
```console
18+
PUT logs
19+
{
20+
"mappings": {
21+
"properties": {
22+
"@timestamp": {
23+
"type": "date"
24+
},
25+
"message": {
26+
"type": "match_only_text"
27+
}
28+
}
29+
}
30+
}
31+
```
32+
33+
34+
## Parameters for match-only text fields [match-only-text-params]
35+
36+
The following mapping parameters are accepted:
37+
38+
[`fields`](/reference/elasticsearch/mapping-reference/multi-fields.md)
39+
: Multi-fields allow the same string value to be indexed in multiple ways for different purposes, such as one field for search and a multi-field for sorting and aggregations, or the same string value analyzed by different analyzers.
40+
41+
[`meta`](/reference/elasticsearch/mapping-reference/mapping-field-meta.md)
42+
: Metadata about the field.
43+

‎docs/reference/elasticsearch/mapping-reference/pattern-text.md‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
---
2+
navigation_title: "Pattern Text"
3+
mapped_pages:
4+
- https://www.elastic.co/guide/en/elasticsearch/reference/current/pattern-text.html
5+
---
6+
7+
## Pattern text field type [pattern-text-field-type]
8+
```{applies_to}
9+
serverless: preview
10+
stack: preview 9.2
11+
```
12+
:::{note}
13+
This feature requires a [subscription](https://www.elastic.co/subscriptions).
14+
:::
15+
16+
The `pattern_text` field type is a variant of [`text`](#text-field-type) with improved space efficiency for log data.
17+
Internally, it decomposes values into static parts that are likely to be shared among many values, and dynamic parts that tend to vary.
18+
The static parts usually come from the explanatory text of a log message, while the dynamic parts are the variables that were interpolated into the logs.
19+
This decomposition allows for improved compression on log-like data.
20+
21+
We call the static portion of the value the `template`.
22+
Although the template cannot be accessed directly, a separate field called `<field_name>.template_id` is accessible.
23+
This field is a hash of the template and can be used to group similar values.
24+
25+
Analysis is configurable but defaults to a delimiter-based analyzer.
26+
This analyzer applies a lowercase filter and then splits on whitespace and the following delimiters: `=`, `?`, `:`, `[`, `]`, `{`, `}`, `"`, `\`, `'`.
27+
28+
## Limitations
29+
30+
Unlike most mapping types, `pattern_text` does not support multiple values for a given field per document.
31+
If a document is created with multiple values for a pattern_text field, an error will be returned.
32+
33+
[span queries](/reference/query-languages/query-dsl/span-queries.md) are not supported with this field, use [interval queries](/reference/query-languages/query-dsl/query-dsl-intervals-query.md) instead, or the [`text`](#text-field-type) field type if you absolutely need span queries.
34+
35+
Like `text`, `pattern_text` does not support sorting and has only limited support for aggregations.
36+
37+
## Phrase matching
38+
Pattern text supports an `index_options` parameter with valid values of `docs` and `positions`.
39+
The default value is `docs`, which makes `pattern_text` behave similarly to `match_only_text` for phrase queries.
40+
Specifically, positions are not stored, which reduces the index size at the cost of slowing down phrase queries.
41+
If `index_options` is set to `positions`, positions are stored and `pattern_text` will support fast phrase queries.
42+
In both cases, all queries return a constant score of 1.0.
43+
44+
## Index sorting for improved compression
45+
The compression provided by `pattern_text` can be significantly improved if the index is sorted by the `template_id` field.
46+
For example, a typical approach would be to sort first by `message.template_id`, then by `@timestamp`, as shown in the following example.
47+
48+
```console
49+
PUT logs
50+
{
51+
"settings": {
52+
"index": {
53+
"sort.field": [ "message.template_id", "@timestamp" ],
54+
"sort.order": [ "asc", "desc" ]
55+
}
56+
},
57+
"mappings": {
58+
"properties": {
59+
"@timestamp": {
60+
"type": "date"
61+
},
62+
"message": {
63+
"type": "pattern_text"
64+
}
65+
}
66+
}
67+
}
68+
```
69+
70+
71+
## Parameters for pattern text fields [pattern-text-params]
72+
73+
The following mapping parameters are accepted:
74+
75+
[`analyzer`](/reference/elasticsearch/mapping-reference/analyzer.md)
76+
: The [analyzer](docs-content://manage-data/data-store/text-analysis.md) which should be used for the `pattern_text` field, both at index-time and at search-time (unless overridden by the [`search_analyzer`](/reference/elasticsearch/mapping-reference/search-analyzer.md)).
77+
Supports a delimiter-based analyzer and the standard analyzer, as is used in `match_only_text` mappings.
78+
Defaults to the delimiter-based analyzer, which applies a lowercase filter and then splits on whitespace and the following delimiters: `=`, `?`, `:`, `[`, `]`, `{`, `}`, `"`, `\`, `'`.
79+
80+
[`index_options`](/reference/elasticsearch/mapping-reference/index-options.md)
81+
: What information should be stored in the index, for search and highlighting purposes. Valid values are `docs` and `positions`. Defaults to `docs`.
82+
83+
[`meta`](/reference/elasticsearch/mapping-reference/mapping-field-meta.md)
84+
: Metadata about the field.
85+

‎docs/reference/elasticsearch/mapping-reference/text-type-family.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
---
2+
navigation_title: "Text type family"
3+
mapped_pages:
4+
- https://www.elastic.co/guide/en/elasticsearch/reference/current/text-type-family.html
5+
---
6+
7+
# Text type family [text]
8+
9+
10+
The text family includes the following field types:
11+
12+
* [`text`](/reference/elasticsearch/mapping-reference/text.md), the traditional field type for full-text content such as the body of an email or the description of a product.
13+
* [`match_only_text`](/reference/elasticsearch/mapping-reference/match-only-text.md), a space-optimized variant of `text` that disables scoring and performs slower on queries that need positions. It is best suited for indexing log messages.
14+
* [`pattern_text`](/reference/elasticsearch/mapping-reference/pattern-text.md), a variant of `text` with improved space efficiency when storing log messages.
15+

‎docs/reference/elasticsearch/mapping-reference/text.md‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,6 @@ mapped_pages:
44
- https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html
55
---
66

7-
# Text type family [text]
8-
9-
10-
The text family includes the following field types:
11-
12-
* [`text`](#text-field-type), the traditional field type for full-text content such as the body of an email or the description of a product.
13-
* [`match_only_text`](#match-only-text-field-type), a space-optimized variant of `text` that disables scoring and performs slower on queries that need positions. It is best suited for indexing log messages.
14-
* [`pattern_text`](#pattern-text-field-type), a variant of `text` with improved space efficiency when storing log messages.
15-
16-
177
## Text field type [text-field-type]
188

199
A field to index full-text values, such as the body of an email or the description of a product. These fields are `analyzed`, that is they are passed through an [analyzer](docs-content://manage-data/data-store/text-analysis.md) to convert the string into a list of individual terms before being indexed. The analysis process allows Elasticsearch to search for individual words *within* each full text field. Text fields are not used for sorting and seldom used for aggregations (although the [significant text aggregation](/reference/aggregations/search-aggregations-bucket-significanttext-aggregation.md) is a notable exception).

0 commit comments

Comments
 (0)