Skip to content
Open
Changes from 5 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8698e45
First version of pattern_text docs
parkertimmins Oct 1, 2025
657670c
Some syntax fixes
parkertimmins Oct 2, 2025
9ec090f
Incorrect sort settings
parkertimmins Oct 2, 2025
db7491f
broken link
parkertimmins Oct 2, 2025
fa2d731
Add applies_to badge
parkertimmins Oct 2, 2025
67aa9dd
Update docs/reference/elasticsearch/mapping-reference/text.md
parkertimmins Oct 7, 2025
cd99011
Update docs/reference/elasticsearch/mapping-reference/text.md
parkertimmins Oct 7, 2025
180ba1e
Update docs/reference/elasticsearch/mapping-reference/text.md
parkertimmins Oct 7, 2025
fe11cdf
Update docs/reference/elasticsearch/mapping-reference/text.md
parkertimmins Oct 7, 2025
75aa09d
Update docs/reference/elasticsearch/mapping-reference/text.md
parkertimmins Oct 7, 2025
5426984
Reorder so limitations are in separate section
parkertimmins Oct 7, 2025
4e6f29c
Add mention of subscription
parkertimmins Oct 7, 2025
5fd13e8
mention standard analyzer is supported
parkertimmins Oct 7, 2025
64b2bd5
Split text family types into separate docs pages
parkertimmins Oct 7, 2025
1f2c619
Remove match_only_text and pattern_text from text docs page
parkertimmins Oct 7, 2025
438454b
Merge branch 'main' into parker/pattern-text-docs
parkertimmins Oct 7, 2025
dd93135
Fix a few build errors
parkertimmins Oct 7, 2025
90933ca
Add redirects, fix some anchors
parkertimmins Oct 7, 2025
db1ede3
Change redirect syntax
parkertimmins Oct 7, 2025
acb57e2
Add to toc.yml
parkertimmins Oct 7, 2025
823863f
Fix incorrect file name
parkertimmins Oct 7, 2025
ab3d63f
Another incorrect anchor
parkertimmins Oct 7, 2025
a6d1c80
Change pattern_text description
parkertimmins Oct 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions docs/reference/elasticsearch/mapping-reference/text.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ The text family includes the following field types:

* [`text`](#text-field-type), the traditional field type for full-text content such as the body of an email or the description of a product.
* [`match_only_text`](#match-only-text-field-type), a space-optimized variant of `text` that disables scoring and performs slower on queries that need positions. It is best suited for indexing log messages.
* [`pattern_text`](#pattern-text-field-type), a variant of `text` with improved space efficiency when storing log messages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could use some clarification--which is better for log messages, match_only_text or pattern_text?

Copy link
Contributor

@leemthompo leemthompo Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should consider splitting these pages up for readability, if you're game go ahead, otherwise could create an issue to do this in a follow-up. We'd need to ensure redirects are set up in /docs/redirects.yml.

  • Text family overview
    • text field type
    • match_only_text
    • pattern_text



## Text field type [text-field-type]
Expand Down Expand Up @@ -341,3 +342,82 @@ The following mapping parameters are accepted:
: Metadata about the field.


## Pattern text field type [pattern-text-field-type]
```{applies_to}
serverless: preview
stack: preview 9.2
```

::::{warning}
This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
::::

A variant of [`text`](#text-field-type) with improved space efficiency for log data.
Internally, it decomposes values into static parts that are likely to be shared among many values, and dynamic parts that tend to vary.
The static parts usually come from the explanatory text of a log message, while the dynamic parts are the variables that were interpolated into the logs.
This decomposition allows for improved compression on log-like data.

We call the static portion of the value the `template`.
Although the template cannot be accessed directly, a separate field called `<field_name>.template_id` is accessible.
This field is a hash of the template and can be used to group similar values.
As this feature is in technical preview, the internal structure of the template is subject to change.
Because of this, `<field_name>.template_id` is also subject to future changes.

Unlike most mapping types, `pattern_text` does not support multiple values for a given field per document.
If a document is created with multiple values for a pattern_text field, an error will be returned.

Analysis is configurable but defaults to a delimiter-based analyzer.
This analyzer applies a lowercase filter and then splits on whitespace and the following delimiters: `=`, `?`, `:`, `[`, `]`, `{`, `}`, `"`, `\`, `'`.

[span queries](/reference/query-languages/query-dsl/span-queries.md) are not supported with this field, use [interval queries](/reference/query-languages/query-dsl/query-dsl-intervals-query.md) instead, or the [`text`](#text-field-type) field type if you absolutely need span queries.

Like `text`, `pattern_text` does not support sorting and has only limited support for aggregations.

### Phrase matching
Pattern text supports an `index_options` parameter with valid values of `docs` and `positions`.
The default value is `docs`, which makes `pattern_text` behave similarly to `match_only_text` for phrase queries.
Specifically, positions are not stored, which reduces the index size at the cost of slowing down phrase queries.
If `index_options` is set to `positions`, positions are stored and `pattern_text` will support fast phrase queries.
In both cases, all queries return a constant score of 1.0.

### Index sorting for improved compression
The compression provided by `pattern_text` can be significantly improved if the index is sorted by the `template_id` field.
For example, a typical approach would be to sort first by `message.template_id`, then by `@timestamp`, as shown in the following example.

```console
PUT logs
{
"settings": {
"index": {
"sort.field": [ "message.template_id", "@timestamp" ],
"sort.order": [ "asc", "desc" ]
}
},
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"message": {
"type": "pattern_text"
}
}
}
}
```


### Parameters for pattern text fields [pattern-text-params]

The following mapping parameters are accepted:

[`analyzer`](/reference/elasticsearch/mapping-reference/analyzer.md)
: The [analyzer](docs-content://manage-data/data-store/text-analysis.md) which should be used for the `pattern_text` field, both at index-time and at search-time (unless overridden by the [`search_analyzer`](/reference/elasticsearch/mapping-reference/search-analyzer.md)). Defaults to a custom delimiter-based analyzer.
This analyzer applies a lowercase filter and then splits on whitespace and the following delimiters: `=`, `?`, `:`, `[`, `]`, `{`, `}`, `"`, `\`, `'`.

[`index_options`](/reference/elasticsearch/mapping-reference/index-options.md)
: What information should be stored in the index, for search and highlighting purposes. Valid values are `docs` and `positions`. Defaults to `docs`.

[`meta`](/reference/elasticsearch/mapping-reference/mapping-field-meta.md)
: Metadata about the field.

Loading