diff --git a/explore-analyze/discover.md b/explore-analyze/discover.md index b32440aaf9..ac5f9c413f 100644 --- a/explore-analyze/discover.md +++ b/explore-analyze/discover.md @@ -6,17 +6,59 @@ applies_to: serverless: ga products: - id: kibana +description: Explore and analyze your Elasticsearch data with Discover in Kibana. Search, filter, visualize, and investigate documents to answer questions about your data. --- -# Discover [discover] +# Explore and analyze data with Discover [discover] -You have questions about your data. What pages on your website contain a specific word or phrase? What events were logged most recently? What processes take longer than 500 milliseconds to respond? +**Discover** in {{kib}} is your primary tool for exploring and analyzing data stored in {{es}}. Use Discover to search and filter your data, investigate document structure and field values, create visualizations, and save your analysis for later use or sharing with your team. -$$$save-your-search$$$ -With **Discover**, you can quickly search and filter your data, get information about the structure of the fields, and display your findings in a visualization. You can also customize and save your Discover sessions and place them on a dashboard. +Discover helps you answer questions about your data: What pages on your website contain specific terms? What events occurred most recently? Which processes exceed performance thresholds? With flexible querying using KQL, Lucene, or {{esql}}, you can quickly find the information you need. :::{image} /explore-analyze/images/kibana-hello-field.png :alt: A view of the Discover app :screenshot: ::: +## What you can do with Discover + +**Search and explore** +: Search through your data using KQL, Lucene, or {{esql}}. Filter results to focus on what matters. Discover adapts its interface based on the type of data you're exploring, providing specialized experiences for logs, metrics, and other data types. + +**Analyze fields and documents** +: View field statistics, examine individual documents, compare multiple documents side by side, and find patterns in your log data. + +**Visualize on the fly** +: Create quick visualizations from aggregatable fields, or use {{esql}} to build charts directly from your queries. + +**Save and share** +: Save your Discover sessions to reuse later, add them to dashboards, or share them with your team. You can also generate reports and create alerts based on your searches. + +## Get started + +New to Discover? Start with these resources: + +* **[Get started with Discover](discover/discover-get-started.md)** - A hands-on tutorial that walks you through exploring data, from loading data to filtering and visualizing your findings. +* **[Using {{esql}}](discover/try-esql.md)** - Learn how to use the {{es}} Query Language for powerful data exploration. + +## Common tasks + +Once you're familiar with the basics, explore these guides for specific tasks: + +* **[Search and filter data](discover/search-and-filter.md)** - Build queries and apply filters to narrow down your results. +* **[Customize the Discover view](discover/document-explorer.md)** - Adjust the layout, columns, and display options to suit your needs. +* **[Save a search for reuse](discover/save-open-search.md)** - Save your Discover sessions and add them to dashboards. + +## Advanced features + +For more sophisticated use cases, see **[Advanced Discover features](discover/discover-advanced-guides.md)**: + +* Compare documents side by side +* Add runtime fields to your {{data-source}} +* Work with multiple tabs +* Understand context-aware experiences +* Run queries in the background +* Generate alerts +* Analyze field statistics and patterns +* Search for relevance + diff --git a/explore-analyze/discover/add-fields-to-data-views.md b/explore-analyze/discover/add-fields-to-data-views.md new file mode 100644 index 0000000000..3c57b357f2 --- /dev/null +++ b/explore-analyze/discover/add-fields-to-data-views.md @@ -0,0 +1,92 @@ +--- +navigation_title: Add fields +mapped_pages: + - https://www.elastic.co/guide/en/kibana/current/discover.html#add-field-in-discover +applies_to: + stack: ga + serverless: ga +products: + - id: kibana +description: Create runtime fields in Discover to extend your data views without reindexing. Compute values on the fly, combine fields, or extract new data from existing fields. +--- + +# Add runtime fields to {{data-sources}} from Discover [add-field-in-discover] + +Create [runtime fields](../../manage-data/data-store/mapping/runtime-fields.md) directly from **Discover** to extend your {{data-source}} without reindexing your data. Runtime fields are computed on the fly from your source data, allowing you to combine existing fields, extract new values, or perform calculations without modifying your indices. + +**Technical summary**: In **Discover**, click **Add a field** from the fields sidebar, select the field type, write a {{product.painless}} script using `emit(value)` to compute the field value from source data, and save to the {{data-source}}. Runtime fields are stored in the {{data-source}} definition and computed at query time. + +Use runtime fields when you need to add missing fields, combine data from multiple fields, or create calculated values for analysis and visualization. + +## Prerequisites + +* You need sufficient privileges to modify the {{data-source}}. Refer to [Granting access to {{kib}}](elasticsearch://reference/elasticsearch/roles.md). +* You should understand [runtime fields](../../manage-data/data-store/mapping/runtime-fields.md) and the [{{product.painless}} scripting language](../scripting/modules-scripting-painless.md). + +## Add a runtime field + +1. In **Discover**, open the {{data-source}} you want to modify. +2. In the fields sidebar, select **Add a field**. +3. Select the **Type** of the new field from the dropdown menu (for example, `Keyword`, `Long`, `Boolean`, `Date`, or `IP`). +4. **Name** the field. Choose a name that corresponds to the naming convention of other fields in the {{data-source}}. +5. Optionally, set a **Custom label** and **Description** for the field to make it more recognizable in your {{data-source}}. The custom label appears in **Discover** and other applications, while the field name is used in queries. +6. Define the field value using one of these options: + + * **Set value**: Define a script that determines the value to show for the field. This is required for computed fields. + * **Set format**: Set your preferred format for displaying the value. Changing the format can affect the value and prevent highlighting in **Discover**. + + By default, if you don't enable **Set value**, the field value is retrieved from the source data if it already contains a field with the same name. + +7. In the **Advanced settings**, you can adjust the field popularity to make it appear higher or lower in the fields list. By default, **Discover** orders popular fields from most selected to least selected. +8. Select **Save** to add the field to your {{data-source}}. + +The new field now appears in the fields list and can be added to the document table, used in queries, and visualized like any other field. + +## Usage examples + +### Example 1: Simple "Hello World" field + +This example creates a simple static text field: + +* **Name**: `hello` +* **Type**: `Keyword` +* **Set value**: enabled +* **Script**: + +```ts +emit("Hello World!"); +``` + +### Example 2: Combine and convert fields + +This example combines first and last name fields from the ecommerce sample data, creating a "Last, First Initial" format: + +* **Name**: `customer` +* **Type**: `Keyword` +* **Set value**: enabled +* **Script**: + +```ts +String str = doc['customer_first_name.keyword'].value; +char ch1 = str.charAt(0); +emit(doc['customer_last_name.keyword'].value + ", " + ch1); +``` + +This creates a computed field that displays as "Smith, J" for a customer named John Smith. + +## Edit or remove a runtime field + +To modify or remove a runtime field you created: + +1. Find the field in the fields list in **Discover**. +2. Hover over the field name and select the gear icon. +3. Choose to edit the field definition or remove it from the {{data-source}}. + +Changes to runtime fields affect all applications using the same {{data-source}}. + +## Learn more + +* For more information on adding fields and {{product.painless}} scripting language examples, refer to [Explore your data with runtime fields](../find-and-organize/data-views.md#runtime-fields). +* For advanced runtime field concepts, see [Runtime fields](../../manage-data/data-store/mapping/runtime-fields.md). +* To learn about {{product.painless}} scripting, refer to [{{product.painless}} scripting language](../scripting/modules-scripting-painless.md). + diff --git a/explore-analyze/discover/background-search.md b/explore-analyze/discover/background-search.md index ecb741cfa4..77b00e583e 100644 --- a/explore-analyze/discover/background-search.md +++ b/explore-analyze/discover/background-search.md @@ -20,6 +20,8 @@ If you have been using search sessions and upgrade to 9.2, your search sessions Sometimes you might need to search through large amounts of data, no matter how long the search takes. Consider a threat hunting scenario, where you need to search through years of data. +**Technical summary**: Enable by setting `data.search.sessions.enabled: true` in `kibana.yml`. From **Discover** or **Dashboards**, click **Send to background** to run long queries asynchronously. Monitor status and restore results using the background searches toolbar button. + You can send your long-running searches to the background from **Discover** or **Dashboards** and let them run while you continue your work. You can access your list of background searches at any time to check their status and manage them from the {icon}`background_task` **Background searches** button in the toolbar. diff --git a/explore-analyze/discover/compare-documents.md b/explore-analyze/discover/compare-documents.md new file mode 100644 index 0000000000..f8b6cf6f03 --- /dev/null +++ b/explore-analyze/discover/compare-documents.md @@ -0,0 +1,79 @@ +--- +navigation_title: Compare documents +mapped_pages: + - https://www.elastic.co/guide/en/kibana/current/discover.html#compare-documents-in-discover +applies_to: + stack: ga + serverless: ga +products: + - id: kibana +description: Compare field values across multiple documents in Discover to identify differences, troubleshoot issues, and understand how values change across records. +--- + +# Compare documents in Discover [compare-documents-in-discover] + +Compare multiple documents side by side in **Discover** to identify differences in field values. This feature helps you troubleshoot issues by spotting variations between similar documents, track how values change across records, or identify patterns in your data. + +## Compare documents side by side + +1. In **Discover**, run your search to display the documents you want to compare. +2. Select the results you want to compare from the **Documents** or **Results** tab. You can select multiple documents by clicking the checkbox next to each one. +3. From the **Selected** menu in the table toolbar, choose **Compare selected**. + + The comparison view opens and shows the selected results next to each other. + +4. Compare the values of each field. By default, the first result you selected serves as the reference for displaying differences in the other results: + + * When the value remains the same for a given field across all documents, it's displayed in **green**. + * When the value differs from the reference document, it's displayed in **red**. + + ::::{tip} + You can change the result used as reference by selecting **Pin for comparison** from the contextual menu of any other result. + :::: + + ![Comparison view in Discover](/explore-analyze/images/kibana-discover-compare-rows.png "") + +5. Optionally, customize the **Comparison settings** to adjust how differences are displayed: + + * Choose to not highlight differences at all + * Show differences more granularly at the line, word, or character level + * Hide fields where the value matches across all results to focus only on differences + +6. Exit the comparison view at any time using the **Exit comparison mode** button at the top of the screen. + +## Copy selected documents + +After comparing documents, you may want to export the selected results for further analysis or record-keeping. + +1. Select the results you want to copy from the table. +2. Open the **Selected** menu in the table toolbar. +3. Choose one of the copy options: + + * **Copy selection as text** - Copies the visible fields in a human-readable text format + * **Copy documents as JSON** - Copies the complete document data in JSON format + +The content is copied to your clipboard in the selected format. Only fields that are currently added to the table as columns are included in the text format. The JSON format includes all fields. + +:::{tip} +You can also copy the content of a single cell to your clipboard using the quick actions that appear when hovering over the cell. +::: + +## Filter to show only selected documents + +If you want to temporarily narrow your view to only the documents you've selected: + +1. Select the documents you want to focus on. +2. Click the **Selected** menu in the table toolbar. +3. Select **Show selected documents only**. + +**Discover** applies a filter to show only those documents. You can remove this filter at any time to return to your full result set. + +## Use cases + +Document comparison is particularly useful for: + +* **Troubleshooting**: Compare error logs to identify patterns or differences in failures +* **Configuration analysis**: Check how settings vary across different hosts or environments +* **Version comparison**: See what changed between different versions of a document +* **Pattern recognition**: Identify common fields and values across similar events + diff --git a/explore-analyze/discover/context-aware-discover.md b/explore-analyze/discover/context-aware-discover.md new file mode 100644 index 0000000000..7b92770172 --- /dev/null +++ b/explore-analyze/discover/context-aware-discover.md @@ -0,0 +1,57 @@ +--- +navigation_title: Context-aware experiences +mapped_pages: + - https://www.elastic.co/guide/en/kibana/current/discover.html#context-aware-discover +applies_to: + stack: ga + serverless: ga +products: + - id: kibana +description: Discover provides specialized interfaces for logs, metrics, traces, and security data. Learn how context-aware experiences adapt to your data type and solution context. +--- + +# Context-aware experiences in Discover [context-aware-discover] + +**Discover** adapts its interface and features based on your data type and solution context. When you explore logs, metrics, traces, or security data within {{observability}} or Security solutions, **Discover** provides specialized views and capabilities tailored to that specific type of data. + +Context-aware experiences combine the right tools, visualizations, and workflows for your data type, making exploration more efficient and intuitive. + +## Available context-aware experiences + +**Discover** currently offers specialized experiences for the following data types: + +* **{{observability}}:** + * **[Logs exploration](/solutions/observability/logs/discover-logs.md)** - Tailored for exploring log data with log-specific features and UI elements. + * **[Metrics exploration](/solutions/observability/infra-and-hosts/discover-metrics.md)** {applies_to}`stack: preview 9.2` {applies_to}`serverless: preview` - Optimized for metrics data with metric-specific visualizations and analysis tools. +% LINK/PAGE TBD * **Traces exploration** - Specialized interface for distributed tracing data. +% * **Security:** +% LINK/PAGE TBD * **Security data exploration** - Enhanced features for security event analysis. + +When you access **Discover** outside of a specific solution context, or when working with data types that don't have specialized experiences, you get the default **Discover** interface with all its core functionality for general-purpose data exploration. + +## Working with multiple data types + +Your query may include multiple data types that each have tailored experiences. For example, if you query both `logs-*` and `traces-*` indices within an {{observability}} context. + +In this case, **Discover** provides the default experience until it detects that you're interacting with a single type of data. For example, when you [expand a document to view its details](discover-get-started.md#look-inside-a-document), **Discover** recognizes the data type and switches to the appropriate context-aware experience for that document. + +## Check which experience is active + +You can verify which experience is currently active for your current Discover session. This helps you confirm whether the type of data you're exploring is properly detected or if Discover is using its default experience. + +1. Select **Inspect** from Discover's toolbar. +2. Open the **View** dropdown, then select **Profiles**. + +The various profiles listed show details such as the active solution and data source contexts, which determine Discover's context-aware experiences. + +## Benefits of context-aware experiences + +Context-aware experiences provide several advantages: + +* **Optimized UI**: Field layouts, visualizations, and controls are tailored to the data type. +* **Relevant features**: Only the features that make sense for your data type are surfaced. +* **Solution integration**: Quick access to related applications and workflows within your solution area. +* **Specialized queries**: Query suggestions and filters appropriate for the data type. + +By adapting to your context, **Discover** reduces complexity and helps you work more efficiently with your specific type of data. + diff --git a/explore-analyze/discover/discover-advanced-guides.md b/explore-analyze/discover/discover-advanced-guides.md new file mode 100644 index 0000000000..91667bd903 --- /dev/null +++ b/explore-analyze/discover/discover-advanced-guides.md @@ -0,0 +1,34 @@ +--- +navigation_title: Advanced features +mapped_pages: + - https://www.elastic.co/guide/en/kibana/current/discover.html +applies_to: + stack: ga + serverless: ga +products: + - id: kibana +description: Advanced features for data exploration in Discover including document comparison, runtime fields, tabs, background queries, alerts, and pattern analysis. +--- + +# Advanced data exploration features in Discover + +After mastering the basics of **Discover**, these advanced features help you work more efficiently with complex data exploration tasks. Compare documents to identify differences, create runtime fields without reindexing, run multiple explorations simultaneously, and set up automated monitoring with alerts. + +## Advanced data manipulation + +* **[Compare documents](compare-documents.md)** - Compare field values across multiple documents side by side to identify differences and patterns. +* **[Add fields to your {{data-source}}](add-fields-to-data-views.md)** - Create runtime fields on the fly to extend your data model without reindexing. + +## Specialized exploration + +* **[Work with tabs](work-with-tabs.md)** - Run multiple explorations simultaneously in separate tabs to compare queries, time periods, or data sources. +* **[Context-aware experiences](context-aware-discover.md)** - Understand how Discover adapts its interface for logs, metrics, traces, and security data. +* **[Run queries in the background](background-search.md)** - Send long-running queries to the background while you continue working. + +## Integration and analysis + +* **[Generate alerts from Discover](generate-alerts-from-discover.md)** - Create rules that periodically check your data against conditions and send notifications. +* **[View field statistics](show-field-statistics.md)** - Explore field distributions, top values, and statistical summaries. +* **[Run pattern analysis](run-pattern-analysis-discover.md)** - Find patterns in unstructured log messages with log pattern analysis. +* **[Search for relevance](discover-search-for-relevance.md)** - Sort documents by relevance score to find the most relevant results. + diff --git a/explore-analyze/discover/discover-get-started.md b/explore-analyze/discover/discover-get-started.md index 4392fd27a4..07fa4aa6ee 100644 --- a/explore-analyze/discover/discover-get-started.md +++ b/explore-analyze/discover/discover-get-started.md @@ -6,395 +6,203 @@ applies_to: serverless: ga products: - id: kibana +description: Learn how to explore and analyze data in Discover. Select data sources, explore fields, create visualizations, and save your exploration sessions. --- -# Explore fields and data with Discover [discover-get-started] +# Get started with Discover [discover-get-started] -Learn how to use **Discover** to: +This tutorial teaches you the fundamentals of exploring data in **Discover**. You'll work through the core workflows for data exploration: selecting data sources, investigating field values, creating visualizations, and saving your work. By the end, you'll be comfortable with the essential features of **Discover** and ready to explore more advanced capabilities. -* **Select** and **filter** your {{es}} data. -* **Explore** the fields and content of your data in depth. -* **Present** your findings in a visualization. +You'll learn how to: -**Prerequisites:** +* Select data and set the time range +* Explore fields and their values +* Add fields to the document table +* Create quick visualizations from fields +* Save your work for later use -* If you don’t already have {{kib}}, [start a free trial](https://www.elastic.co/cloud/elasticsearch-service/signup?baymax=docs-body&elektra=docs) on Elastic Cloud. -* You must have data in {{es}}. Examples on this page use the [ecommerce sample data set](../index.md#gs-get-data-into-kibana), but you can use your own data. -* You should have an understanding of [{{es}} documents and indices](../../manage-data/data-store/index-basics.md). - -## Context-aware data exploration [context-aware-discover] - -**Discover** provides tailored interfaces and features for the following data types when accessed from Observability or Security project types or {{kib}} solution views: - -* Observability: - * **[Logs exploration](/solutions/observability/logs/discover-logs.md)** - * **[Metrics exploration](/solutions/observability/infra-and-hosts/discover-metrics.md)** {applies_to}`stack: preview 9.2` {applies_to}`serverless: preview` -% LINK/PAGE TBD * **Traces exploration** -% * Security: -% LINK/PAGE TBD * **Security data exploration** - -This context-aware experience is determined by both your solution context and the type of data you query. When both conditions align, **Discover** provides specific capabilities useful for exploring that specific type of data, and integrates features or paths to other relevant solution applications. - -When you access **Discover** outside of a specific solution context, or when working with data types that don't have specialized experiences, you get the default **Discover** interface with all its core functionality for general-purpose data exploration. +## Prerequisites -### Context-awareness with multiple data types - -Your query may include multiple data types that each have tailored experiences; for example, if you query both `logs-*` and `traces-*` indices within an Observability context. - -In this case **Discover** provides the default experience until it detects that you're interacting with a single type of data. For example, when you [](#look-inside-a-document). - -### View active context-aware experience - -You can check which experience is currently active for your current Discover session. This can help you confirm whether the type of data you're currently exploring is properly detected or if Discover is currently using its default experience. +* If you don't already have {{kib}}, [start a free trial](https://www.elastic.co/cloud/elasticsearch-service/signup?baymax=docs-body&elektra=docs) on {{ecloud}}. +* You must have data in {{es}}. This tutorial uses the [ecommerce sample data set](../index.md#gs-get-data-into-kibana), but you can use your own data. +* You should have an understanding of [{{es}} documents and indices](../../manage-data/data-store/index-basics.md). -1. Select **Inspect** from Discover's toolbar. -1. Open the **View** dropdown, then select **Profiles**. +::::{note} +**Discover** adapts its interface based on the type of data you're exploring (logs, metrics, traces, security events) and your solution context. For details about these specialized experiences, see [Context-aware experiences in Discover](context-aware-discover.md). This tutorial focuses on the core **Discover** features available in all contexts. +:::: -The various profiles listed show details such as the active solution and data source contexts, which determine Discover's context-aware experiences. +## Step 1: Load data into Discover [find-the-data-you-want-to-use] -## Load data into Discover [find-the-data-you-want-to-use] +First, select the data you want to explore and set the time range. -Select the data you want to explore, and then specify the time range in which to view that data. +1. Open **Discover** from the navigation menu or by using the [global search field](../../explore-analyze/find-and-organize/find-apps-and-objects.md). -1. Find **Discover** in the navigation menu or by using the [global search field](../../explore-analyze/find-and-organize/find-apps-and-objects.md). -2. Select the data view that contains the data you want to explore. +2. Select the {{data-source}} that contains the data you want to explore. For this tutorial, select **{{kib}} Sample Data eCommerce** if you've installed the sample data. + ::::{tip} - By default, {{kib}} requires a [{{data-source}}](../find-and-organize/data-views.md) to access your Elasticsearch data. A {{data-source}} can point to one or more indices, [data streams](../../manage-data/data-store/data-streams.md), or [index aliases](/manage-data/data-store/aliases.md). When adding data to {{es}} using one of the many integrations available, sometimes data views are created automatically, but you can also create your own. - - You can also [try {{esql}}](try-esql.md), that lets you query any data you have in {{es}} without specifying a {{data-source}} first. + By default, {{kib}} requires a {{data-source}} to access your {{es}} data. A {{data-source}} can point to one or more indices, [data streams](../../manage-data/data-store/data-streams.md), or [index aliases](/manage-data/data-store/aliases.md). + + Alternatively, you can try {{esql}}, which lets you query any data you have in {{es}} without selecting a {{data-source}} first. :::: - If you’re using sample data, data views are automatically created and are ready to use. + :::{image} /explore-analyze/images/kibana-discover-data-view.png :alt: How to set the {{data-source}} in Discover :screenshot: :width: 300px ::: -3. If needed, adjust the [time range](../query-filter/filtering.md), for example by setting it to the **Last 7 days**. - The range selection is based on the default time field in your data view. If you are using the sample data, this value was set when the data view was created. If you are using your own data view, and it does not have a time field, the range selection is not available. - +3. Adjust the [time range](../query-filter/filtering.md) to **Last 7 days** using the time picker in the upper right. -**Discover** is populated with your data and you can view various areas with different information: +**Discover** now displays your data with three main areas: -* All fields detected are listed in a dedicated panel. -* A chart allows you to visualize your data. -* A table displays the results of your search. By default, the table includes a column for the time field and a **Summary** column with an overview of each result. You can modify the document table to display your fields of interest. +* **Fields panel** (left sidebar): Lists all fields detected in your data +* **Chart** (top): Visualizes the distribution of your data over time +* **Document table** (bottom): Shows individual results with a time column and a **Summary** column by default -You can later filter the data that shows in the chart and in the table by specifying a query and changing the time range. +You can modify these areas as you explore. Next, let's dive into the fields in your data. -## Explore the fields in your data [explore-fields-in-your-data] +## Step 2: Explore fields in your data [explore-fields-in-your-data] -**Discover** provides utilities designed to help you make sense of your data: +Now that you have data loaded, explore the fields to understand your data's structure. -1. In the sidebar, check the available fields. It's common to have hundreds of fields. Use the search at the top of that sidebar to look for specific terms in the field names. - In this example, we’ve entered `ma` in the search field to find the `manufacturer` field. +1. Look at the **fields panel** on the left. You'll see hundreds of fields. Use the search box at the top to find specific fields. + + Try searching for `ma` to find the `manufacturer` field. + ![Fields list that displays the top five search results](/explore-analyze/images/kibana-discover-sidebar-available-fields.png "title =40%") + ::::{tip} - You can combine multiple keywords or characters. For example, `geo dest` finds `geo.dest` and `geo.src.dest`. + You can combine multiple keywords. For example, `geo dest` finds both `geo.dest` and `geo.src.dest`. :::: - {applies_to}`stack: ga 9.2` For some searches, Discover suggests recommended fields to explore. These suggestions are based on the data you query and are managed by Elastic. - -2. Select a field to view its most frequent values. - **Discover** shows the top 10 values and the number of records used to calculate those values. +2. Click on a field name to view its most frequent values. + + **Discover** shows the top 10 values and the number of documents containing each value. -3. Select the **Plus** icon to add fields to the results table. You can also drag them from the list into the table. +3. Add fields to the document table to see them as columns: + + * Click the **+** icon next to a field name, or + * Drag a field from the list directly into the table ![How to add a field as a column in the table](/explore-analyze/images/kibana-discover-add-field.png "title =50%") - When you add fields to the table, the **Summary** column is replaced. + When you add fields, the default **Summary** column is replaced with your selected fields. + ![Document table with fields for manufacturer](/explore-analyze/images/kibana-document-table.png "") -4. Arrange the view to your liking to display the fields and data you care most about using the various display options of **Discover**. For example, you can change the order and size of columns, expand the table to be in full screen or collapse the chart and the list of fields. Check [Customize the Discover view](document-explorer.md) for more information. -5. **Save** your changes to be able to open the same view later on and explore your data further. - - -### Add a field to your {{data-source}} [add-field-in-discover] - -What happens if you forgot to define an important value as a separate field? Or, what if you want to combine two fields and treat them as one? This is where [runtime fields](../../manage-data/data-store/mapping/runtime-fields.md) come into play. You can add a runtime field to your {{data-source}} from inside of **Discover**, and then use that field for analysis and visualizations the same way you do with other fields. - -1. In the sidebar, select **Add a field**. -2. Select the **Type** of the new field. -3. **Name** the field. Name it in a way that corresponds to the way other fields of the data view are named. You can set a custom label and description for the field to make it more recognizable in your data view. -4. Define the value that you want the field to show. By default, the field value is retrieved from the source data if it already contains a field with the same name. You can customize this with the following options: - - **Set value**: Define a script that will determine the value to show for the field. For more information on adding fields and Painless scripting language examples, refer to [Explore your data with runtime fields](../find-and-organize/data-views.md#runtime-fields). - - **Set format**: Set your preferred format for displaying the value. Changing the format can affect the value and prevent highlighting in Discover. - -5. In the advanced settings, you can adjust the field popularity to make it appear higher or lower in the fields list. By default, Discover orders popular fields from most selected to least selected. -6. **Save** your new field. - -You can now find it in the list of fields and add it to the table. - -In the following example, we’re adding 2 fields: A simple "Hello world" field, and a second field that combines and transforms the `customer_first_name` and `customer_last_name` fields of the sample data into a single "customer" field: - -**Hello world field example**: - -* **Name**: `hello` -* **Type**: `Keyword` -* **Set value**: enabled -* **Script**: +4. Try adding these fields from the ecommerce sample data: + * `manufacturer.keyword` + * `products.product_name.keyword` + * `customer_first_name.keyword` + * `total_quantity` - ```ts - emit("Hello World!"); - ``` +5. Rearrange columns by dragging their headers to new positions. -**Customer field example**: +## Step 3: Visualize aggregated fields [_visualize_aggregated_fields] -* **Name**: `customer` -* **Type**: `Keyword` -* **Set value**: enabled -* **Script**: +**Discover** lets you create quick visualizations from aggregatable fields without leaving the application. - ```ts - String str = doc['customer_first_name.keyword'].value; - char ch1 = str.charAt(0); - emit(doc['customer_last_name.keyword'].value + ", " + ch1); - ``` - - - -### Visualize aggregated fields [_visualize_aggregated_fields] - -If a field can be [aggregated](../query-filter/aggregations.md), you can quickly visualize it in detail by opening it in **Lens** from **Discover**. **Lens** is the default visualization editor in {{kib}}. - -1. In the list of fields, find an aggregatable field. For example, with the sample data, you can look for `day_of_week`. +1. In the fields list, find an aggregatable field such as `day_of_week`. ![Top values for the day_of_week field](/explore-analyze/images/kibana-discover-day-of-week.png "title =60%") -2. In the popup, click **Visualize**. +2. In the field popup, click **Visualize**. + {{kib}} creates a **Lens** visualization best suited for this field. -3. In **Lens**, from the **Available fields** list, drag and drop more fields to refine the visualization. In this example, we’re adding the `manufacturer.keyword` field onto the workspace, which automatically adds a breakdown of the top values to the visualization. +3. In **Lens**, drag and drop more fields from the **Available fields** list to refine the visualization. Try adding the `manufacturer.keyword` field to the workspace, which automatically adds a breakdown of the top values. + ![Visualization that opens from Discover based on your data](/explore-analyze/images/kibana-discover-from-visualize.png "") -4. Save the visualization if you’d like to add it to a dashboard or keep it in the Visualize library for later use. +4. Save the visualization to add it to a dashboard or keep it in the Visualize library for later use. -For geo point fields (![Geo point field icon](/explore-analyze/images/kibana-geoip-icon.png "kibana-geoip-icon =4%x4%")), if you click **Visualize**, your data appears in a map. +::::{tip} +For geo point fields (![Geo point field icon](/explore-analyze/images/kibana-geoip-icon.png "kibana-geoip-icon =4%x4%")), clicking **Visualize** opens your data in a map. ![Map containing documents](/explore-analyze/images/kibana-discover-maps.png "") +:::: -### Find and highlight values in the table [in-table-search] - -Use the in-table search to find and highlight specific values beyond what's currently visible on your screen. - -The in-table search looks for all matching values in all results and pages currently loaded in the table. The number of results loaded depends on the [Sample size](document-explorer.md#document-explorer-sample-size). If you load more results, the search automatically updates and reflects the new number of matching values, if any more are found. - -```{tip} -You can navigate between results with your keyboard by pressing "Enter" to go to the next result, and "Shift + Enter" to go to the previous result. -``` - -![Using the in-table search and navigating through the matches](https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt30bf5f8b9a45ab74/67c234a787966d9fbc994ce0/in-table-search-demo.gif) - - -### Compare documents [compare-documents-in-discover] - -You can use **Discover** to compare and diff the field values of multiple results or documents in the table. - -1. Select the results you want to compare from the Documents or Results tab in Discover. -2. From the **Selected** menu in the table toolbar, choose **Compare selected**. The comparison view opens and shows the selected results next to each other. -3. Compare the values of each field. By default the first result selected shows as the reference for displaying differences in the other results. When the value remains the same for a given field, it’s displayed in green. When the value differs, it’s displayed in red. - ::::{tip} - You can change the result used as reference by selecting **Pin for comparison** in the contextual menu of any other result. - :::: - - - ![Comparison view in Discover](/explore-analyze/images/kibana-discover-compare-rows.png "") - -4. Optionally, customize the **Comparison settings** to your liking. You can for example choose to not highlight the differences, to show them more granularly at the line, word, or character level, or even to hide fields where the value matches for all results. -5. Exit the comparison view at any time using the **Exit comparison mode** button. - - -### Copy results as text or JSON [copy-row-content] - -You can quickly copy the content currently displayed in the table for one or several results to your clipboard. - -1. Select the results you want to copy. -2. Open the **Selected** menu in the table toolbar, and select **Copy selection as text** or **Copy documents as JSON**. - -The content is copied to your clipboard in the selected format. Fields that are not currently added to the table are ignored. - -:::{tip} -You can copy the content of a single cell to your clipboard from the quick actions that appear when hovering over the cell. -::: +## Step 4: Explore individual documents [look-inside-a-document] +Dive deeper into individual documents to view all their fields and values. -### Explore individual result or document details in depth [look-inside-a-document] - -$$$document-explorer-expand-documents$$$ -Dive into an individual document to view its fields and the documents that occurred before and after it. - -1. In the document table, click the expand icon ![double arrow icon to open a flyout with the document details](/explore-analyze/images/kibana-expand-icon-2.png "") to show document details. +1. In the document table, click the expand icon (![double arrow icon to open a flyout with the document details](/explore-analyze/images/kibana-expand-icon-2.png "")) next to any document. ![Table view with document expanded](/explore-analyze/images/kibana-document-table-expanded.png "") -2. Scan through the fields and their values. You can filter the table in several ways: - - * If you find a field of interest, hover your mouse over the **Field** or **Value** columns for filters and additional options. - * Use the search above the table to filter for specific fields or values, or filter by field type using the options to the right of the search field. - * You can pin some fields by clicking the left column to keep them displayed even if you filter the table. - - ::::{tip} - You can restrict the fields listed in the detailed view to the fields that you explicitly added to the **Discover** table, using the **Selected only** toggle. In ES|QL mode, you also have an option to hide fields with null values. This toggle isn't available from the **View single document** page. - :::: - -3. To navigate to a view of the document that you can bookmark and share, select **View single document**. -4. To view documents that occurred before or after the event you are looking at, select **View surrounding documents**. - - -## Search and filter data [search-in-discover] - - -### Default mode: Search and filter using KQL [_default_mode_search_and_filter_using_kql] - -One of the unique capabilities of **Discover** is the ability to combine free text search with filtering based on structured data. To search all fields, enter a simple string in the query bar. - -![Search field in Discover](/explore-analyze/images/kibana-discover-search-field.png "") - -:::{note} -Free text searches that don't specify a field may not return expected results depending on how the [`index.query.default_field` index setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md#index-query-default-field) is configured for the indices matching the current data view. -::: - -To search particular fields and build more complex queries, use the [Kibana Query language](../query-filter/languages/kql.md). As you type, KQL prompts you with the fields you can search and the operators you can use to build a structured query. - -For example, search the ecommerce sample data for documents where the country matches US: - -1. Enter `g`, and then select **geoip.country_iso_code**. -2. Select **:** for equals, and **US** for the value, and then click the refresh button or press the Enter key. -3. For a more complex search, try: - - ```ts - geoip.country_iso_code : US and products.taxless_price >= 75 - ``` - - -$$$filter-in-discover$$$ -With the query input, you can filter data using the KQL or Lucene languages. You can also use the **Add filter** function available next to the query input to build your filters one by one or define them as Query DSL. - -For example, exclude results from the ecommerce sample data view where day of week is not Wednesday: - -1. Click ![Add icon](/explore-analyze/images/kibana-add-icon.png "") next to the query bar. -2. In the **Add filter** pop-up, set the field to **day_of_week**, the operator to **is not**, and the value to **Wednesday**. - - ![Add filter dialog in Discover](/explore-analyze/images/kibana-discover-add-filter.png "") - -3. Click **Add filter**. -4. Continue your exploration by adding more filters. -5. To remove a filter, click the close icon (x) next to its name in the filter bar. +2. Scan through the fields and their values in the flyout: -#### Filter pill actions + * Hover over a **Field** or **Value** to see filter options and other actions + * Use the search box to find specific fields or values + * Pin important fields by clicking the pin icon to keep them visible when filtering -:::{include} ../_snippets/global-filters.md -::: +3. Try these additional actions: + * Click **View single document** to open a standalone view you can bookmark and share + * Click **View surrounding documents** to see documents that occurred before and after this one in time -### Search and filter using ES|QL [_search_and_filter_using_esql] +## Step 5: Search and filter your data [search-in-discover] -You can use **Discover** with the Elasticsearch Query Language, ES|QL. When using ES|QL, you don’t have to select a data view. It’s your query that determines the data to explore and display in Discover. +Now let's narrow down your results with a search query and filters. -You can switch to the ES|QL mode of Discover from the application menu bar. -If you've entered a KQL or Lucene query in the default mode of Discover, it automatically converts to ES|QL. +1. In the query bar at the top, try a simple search. For the ecommerce sample data, search for documents where the country is US: + + ```ts + geoip.country_iso_code : US + ``` -In ES|QL mode, the **Documents** tab is named **Results**. + Press Enter to run the query. -:::{important} -{applies_to}`stack: ga 9.1` When an ES|QL query times out, partial results that are available are shown. The timeout is defined by the `search:timeout` advanced setting, which is set to 10 minutes (600000 ms) by default. In serverless projects, this advanced setting is not customizable and the timeout is set to 10 minutes. -::: +2. Add a filter to further refine your results: + + * Click the **Add filter** button (![Add icon](/explore-analyze/images/kibana-add-icon.png "")) next to the query bar + * Set **Field** to `day_of_week` + * Set **Operator** to `is not` + * Set **Value** to `Wednesday` + * Click **Add filter** -Learn more about how to use ES|QL queries in [Using ES|QL](try-esql.md). + ![Add filter dialog in Discover](/explore-analyze/images/kibana-discover-add-filter.png "") -### Run multiple explorations with tabs -```{applies_to} -stack: preview 9.2 -serverless: preview -``` -**Discover** supports multiple tabs to help you explore different aspects of your data simultaneously. Each tab maintains its own independent state, including the query ({{esql}} or classic mode), time range, filters, selected data source, columns and sort order you defined, and the active [context-aware experience](#context-aware-discover). +3. Notice how the results update to match your query and filters. The chart also adjusts to show only the matching data. -This allows you to pivot quickly between different explorations without losing your place. For example: +::::{tip} +As you type in the query bar, KQL suggests fields and operators to help you build structured queries. You can also create filters by clicking the **+** or **-** icons next to field values in the fields panel. +:::: -* **Compare time periods:** Open multiple tabs with the same query but different time ranges -* **Test query variations:** Duplicate a tab to experiment with different {{esql}} queries or filters -* **Switch contexts:** Keep separate tabs for logs, metrics, and traces explorations -* **Test a hypothesis:** Switch between different data sources or field combinations +For more detailed information on searching and filtering, see [Search and filter data](search-and-filter.md). -#### Manage Discover tabs +## Step 6: Save your session [save-discover-search] -You can open new tabs or duplicate existing ones to compare different queries: -- To start a fresh exploration in a new tab, select the {icon}`plus` icon next to the existing tabs. -- To test variations of your current query in a new tab, hover over a tab and select the {icon}`boxes_vertical` **Actions** icon, then select **Duplicate**. +Save your Discover session so you can return to it later or share it with others. -To manage and organize your tabs, you can: -- Rename them: Double-click its label or hover over a tab and select the {icon}`boxes_vertical` **Actions** icon, then select **Rename**. -- Reorder them: Drag and drop a tab to move it. -- Close them: Hover over a tab and select the {icon}`cross` icon. -- Close several tabs at once: When you hover over a tab and select the {icon}`boxes_vertical` **Actions** icon, options let you **Close other tabs** to keep only the active tab open or **Close tabs to the right** to only keep your first tabs and discard any subsequent tabs. - - :::{tip} - If you want to discard all open tabs, you can also start a {icon}`plus` **New session** from the toolbar. When you use this option, any unsaved changes to your current session are lost. - ::: -- Reopen recently closed tabs: If you close a tab by mistake, you can retrieve it by selecting the {icon}`boxes_vertical` **Tabs bar menu** icon located at the end of the tab bar. - -To keep all of your tabs for later, you can [Save your Discover session](#save-discover-search). All currently open tabs are saved within the session and will be there when you open it again. - -### Inspect your Discover queries - -:::{include} ../_snippets/inspect-request.md -::: - -### Run long-running queries in the background -```{applies_to} -stack: ga 9.2 -serverless: unavailable -``` - -You can send your long-running KQL or {{esql}} queries to the background from **Discover** and let them run while you continue exploring your data. Refer to [Run queries in the background](/explore-analyze/discover/background-search.md). - - -### Save your Discover session for later use [save-discover-search] - -Save your Discover session so you can use it later, generate a CSV report, or use it to create visualizations, dashboards, and Canvas workpads. Saving a Discover session saves all open tabs, along with their query text, filters, and current view of **Discover**, including the columns selected in the document table, the sort order, and the {{data-source}}. - -1. In the application menu bar, click **Save**. -2. Give your session a title and a description. -3. Optionally store [tags](../find-and-organize/tags.md) and the time range with the session. +1. Click **Save** in the toolbar. +2. Give your session a meaningful name like "US ecommerce purchases". +3. Optionally add a description and [tags](../find-and-organize/tags.md). 4. Click **Save**. - -### Share your Discover session [share-your-findings] - -To share your search and **Discover** view with a larger audience, click {icon}`share` **Share** in the application menu bar. For detailed information about the sharing options, refer to [Reporting](../report-and-share.md). - - -## Generate alerts [alert-from-Discover] - -From **Discover**, you can create a rule to periodically check when data goes above or below a certain threshold within a given time interval. - -1. Ensure that your data view, query, and filters fetch the data for which you want an alert. -2. In the application menu bar, click **Alerts > Create search threshold rule**. - - The **Create rule** form is pre-filled with the latest query sent to {{es}}. - -3. [Configure your query](../alerts-cases/alerts/rule-type-es-query.md) and [select a connector type](../../deploy-manage/manage-connectors.md). -4. Click **Save**. - -For more about this and other rules provided in {{alert-features}}, go to [Alerting](../alerts-cases/alerts.md). - - -## What’s next? [_whats_next_4] - -* [Search for relevance](discover-search-for-relevance.md). -* [Configure the chart and document table](document-explorer.md) to better meet your needs. +Your session is now saved with all your settings: the query, filters, selected fields, time range, and {{data-source}}. You can reopen it anytime by clicking **Open** in the toolbar. -## Troubleshooting [_troubleshooting] +## What's next -This section references common questions and issues encountered when using Discover. Also check the following blog post: [Learn how to resolve common issues with Discover.](https://www.elastic.co/blog/troubleshooting-guide-common-issues-kibana-discover-load) +Now that you're familiar with the basics of **Discover**, explore these guides to learn more: -**Some fields show as empty while they should not be, why is that?** +**Common tasks** +* **[Search and filter data](search-and-filter.md)** - Learn advanced query techniques and filtering strategies +* **[Customize the Discover view](document-explorer.md)** - Adjust the layout, table, and display options +* **[Compare documents](compare-documents.md)** - Compare field values across multiple documents +* **[Add fields to your {{data-source}}](add-fields-to-data-views.md)** - Create runtime fields to extend your data -This can happen in several cases: +**Advanced features** +* **[Using {{esql}}](try-esql.md)** - Query your data with the {{es}} Query Language +* **[Work with tabs](work-with-tabs.md)** - Run multiple explorations simultaneously +* **[Run queries in the background](background-search.md)** - Send long-running queries to the background +* **[Generate alerts from Discover](generate-alerts-from-discover.md)** - Create rules to monitor your data -* With runtime fields and regular keyword fields, when the string exceeds the value set for the [ignore_above](elasticsearch://reference/elasticsearch/mapping-reference/ignore-above.md) setting used when indexing the data into {{es}}. -* Due to the structure of nested fields, a leaf field added to the table as a column will not contain values in any of its cells. Instead, add the root field as a column to view a JSON representation of its values. Learn more in [this blog post](https://www.elastic.co/de/blog/discover-uses-fields-api-in-7-12). +**Analysis tools** +* **[View field statistics](show-field-statistics.md)** - Explore field distributions and statistics +* **[Run pattern analysis](run-pattern-analysis-discover.md)** - Find patterns in log messages +* **[Search for relevance](discover-search-for-relevance.md)** - Sort documents by relevance score diff --git a/explore-analyze/discover/discover-search-for-relevance.md b/explore-analyze/discover/discover-search-for-relevance.md index 18dcc5bb2b..e8245aa7c4 100644 --- a/explore-analyze/discover/discover-search-for-relevance.md +++ b/explore-analyze/discover/discover-search-for-relevance.md @@ -6,17 +6,18 @@ applies_to: serverless: ga products: - id: kibana +description: Sort Discover results by relevance score to find the most relevant documents for your query. Learn how to use the _score field to rank search results. --- -# Search for relevance [discover-search-for-relevance] +# Sort results by relevance score in Discover [discover-search-for-relevance] -{{es}} assigns a relevancy, or score to each document, so you can can narrow your search to the documents with the most relevant results. The higher the score, the better it matches your query. +{{es}} assigns a relevance score to each document based on how well it matches your search query. Sort documents by their `_score` field in **Discover** to display the most relevant results first, helping you quickly find the information that best matches your search criteria. This example shows how to use **Discover** to list your documents from most relevant to least relevant. This example uses the [sample flights data set](../index.md#gs-get-data-into-kibana), or you can use your own data. 1. In **Discover**, open the {{data-source}} dropdown, and select the data that you want to work with. - For the sample flights data, set the {{data-source}} to **Kibana Sample Data Flights**. + For the sample flights data, set the {{data-source}} to **{{kib}} Sample Data Flights**. 2. Run your search. For the sample data, try: diff --git a/explore-analyze/discover/document-explorer.md b/explore-analyze/discover/document-explorer.md index e36d513104..a84a4f7d81 100644 --- a/explore-analyze/discover/document-explorer.md +++ b/explore-analyze/discover/document-explorer.md @@ -6,14 +6,15 @@ applies_to: serverless: ga products: - id: kibana +description: Customize the Discover interface to optimize your data exploration. Adjust columns, table density, row height, sample size, and sorting options. --- # Customize the Discover view [document-explorer] -Fine tune your explorations by customizing **Discover** to bring out the the best view of your documents. +Adjust the **Discover** interface to match your workflow and data exploration needs. Customize table columns, density, and row height to display your data effectively. Configure sample sizes, sorting, and layout to focus on the information that matters most. :::{tip} -Discover provides default [context-aware experiences](/explore-analyze/discover/discover-get-started.md#context-aware-discover) tailored to the type of data that you're exploring, and you can further customize your Discover view on top of them. +**Discover** provides default [context-aware experiences](context-aware-discover.md) tailored to your data type. You can further customize these views to suit your specific needs. ::: :::{image} /explore-analyze/images/kibana-hello-field.png @@ -92,7 +93,7 @@ To sort by multiple fields: Change how {{kib}} displays a field. -1. Click the column header for the field, and then select **Edit data view field.** +1. Click the column header for the field, and then select **Edit {{data-source}} field.** 2. In the **Edit field** form, change the field name and format. For detailed information on formatting options, refer to [Format data fields](../find-and-organize/data-views.md#managing-fields). @@ -111,7 +112,7 @@ Narrow your results to a subset of documents so you're comparing the data of int ::: -You can also compare individual field values using the [**Compare selected** option](discover-get-started.md#compare-documents-in-discover). +You can also compare individual field values using the [**Compare selected** option](compare-documents.md). ### Set the number of results per page [document-explorer-configure-table] diff --git a/explore-analyze/discover/generate-alerts-from-discover.md b/explore-analyze/discover/generate-alerts-from-discover.md new file mode 100644 index 0000000000..0d5311d75d --- /dev/null +++ b/explore-analyze/discover/generate-alerts-from-discover.md @@ -0,0 +1,81 @@ +--- +navigation_title: Generate alerts from Discover +mapped_pages: + - https://www.elastic.co/guide/en/kibana/current/discover.html#alert-from-Discover +applies_to: + stack: ga + serverless: ga +products: + - id: kibana +description: Create alerting rules from Discover searches to monitor data conditions. Get notified when values exceed thresholds or match specific criteria in your data. +--- + +# Create alerts from Discover searches [alert-from-Discover] + +Create alerting rules directly from **Discover** to monitor your data automatically. Set up rules that check your data at regular intervals and send notifications when values exceed thresholds, match specific conditions, or deviate from expected patterns. + +**Technical summary**: From **Discover** with your query configured, select **Alerts > Create search threshold rule**, configure threshold conditions and time window, select a connector (email, Slack, PagerDuty, {{webhook}}), and save the rule. Manage rules in **{{stack-manage-app}} > {{rules-ui}}**. + +## Prerequisites + +* You must have the appropriate privileges to create rules. Refer to [Alerting setup](../alerts-cases/alerts/alerting-setup.md). +* Your query should be tested and refined to return the data you want to monitor. + +## Create a search threshold rule + +1. In **Discover**, ensure that your {{data-source}}, query, and filters fetch the data for which you want an alert. +2. Test your search to confirm it returns the expected results. +3. In the application menu bar, click **Alerts > Create search threshold rule**. + + The **Create rule** form opens, pre-filled with the latest query sent to {{es}}. + +4. [Configure your query](../alerts-cases/alerts/rule-type-es-query.md) by setting: + * **Threshold conditions**: Define when the alert should fire (for example, when the count is above, below, or between certain values) + * **Time window**: Specify the time interval to check (for example, last 5 minutes) + * **Check frequency**: How often to run the query (for example, every 1 minute) + +5. [Select a connector type](../../deploy-manage/manage-connectors.md) to determine how you'll be notified when the rule fires. Options include: + * Email + * Slack + * PagerDuty + * {{webhook}} + * Other notification methods + +6. Configure the action details for your chosen connector. +7. Click **Save** to create the rule. + +The rule now runs in the background at the specified frequency, checking your data against the threshold conditions you defined. + +## Manage your rules + +After creating a rule from **Discover**, you can manage it in the {{rules-ui}} interface: + +1. Go to **{{stack-manage-app}} > Alerts and Insights > {{rules-ui}}**. +2. Find your rule in the list. +3. Click on it to view details, edit conditions, or disable/enable it. + +You can also view the history of when the rule fired and what actions were taken. + +## Usage example + +Suppose you're monitoring application logs and want to be alerted when error rates spike: + +1. In **Discover**, create a query that filters for error-level logs: + ``` + log.level : "error" AND service.name : "checkout" + ``` + +2. Click **Alerts > Create search threshold rule**. +3. Configure the threshold to fire when the count is above 50 in the last 5 minutes. +4. Set it to check every 1 minute. +5. Configure an email or Slack connector to notify your team. +6. Save the rule. + +Now your team will be notified whenever the checkout service logs more than 50 errors in a 5-minute window. + +## Learn more + +* [Alerting](../alerts-cases/alerts.md) - Complete guide to {{alert-features}} +* [{{es}} query rule](../alerts-cases/alerts/rule-type-es-query.md) - Detailed configuration options +* [{{connectors-ui}}](../../deploy-manage/manage-connectors.md) - Available notification methods + diff --git a/explore-analyze/discover/run-pattern-analysis-discover.md b/explore-analyze/discover/run-pattern-analysis-discover.md index 127922d84c..4e946b7f56 100644 --- a/explore-analyze/discover/run-pattern-analysis-discover.md +++ b/explore-analyze/discover/run-pattern-analysis-discover.md @@ -6,18 +6,19 @@ applies_to: serverless: ga products: - id: kibana +description: Run pattern analysis on log data in Discover to find patterns in unstructured messages. Categorize logs, view distribution charts, and filter by pattern. --- -# Run a pattern analysis on your log data [run-pattern-analysis-discover] +# Run pattern analysis on log data in Discover [run-pattern-analysis-discover] -Log pattern analysis helps you to find patterns in unstructured log messages and makes it easier to examine your data. It performs categorization analysis on a selected field of a {{data-source}}, creates categories based on the data and displays them together with a chart that shows the distribution of each category and an example document that matches the category. +Use log pattern analysis in **Discover** to find patterns in unstructured log messages automatically. Pattern analysis categorizes your log data, shows the distribution of each category, and provides example documents for each pattern, making it easier to identify trends and troubleshoot issues. Log pattern analysis works on every text field. This example uses the [sample web logs data](../index.md#gs-get-data-into-kibana), or you can use your own data. 1. Go to **Discover**. -2. Expand the {{data-source}} dropdown, and select **Kibana Sample Data Logs**. +2. Expand the {{data-source}} dropdown, and select **{{kib}} Sample Data Logs**. 3. If you don’t see any results, expand the time range, for example, to **Last 15 days**. 4. Click the **Patterns** tab next to **Documents** and **Field statistics**. The pattern analysis starts. The results are displayed under the chart. You can change the analyzed field by using the field selector. In the **Pattern analysis menu**, you can change the **Minimum time range**. This option enables you to widen the time range for calculating patterns which improves accuracy. The patterns, however, are still displayed by the time range you selected in step 3. diff --git a/explore-analyze/discover/save-open-search.md b/explore-analyze/discover/save-open-search.md index dc485a0f03..22e63052e4 100644 --- a/explore-analyze/discover/save-open-search.md +++ b/explore-analyze/discover/save-open-search.md @@ -7,19 +7,20 @@ applies_to: serverless: ga products: - id: kibana +description: Save Discover sessions to reuse searches, share with your team, add to dashboards, or use as a foundation for building visualizations in Kibana. --- -# Discover sessions: Save a search for reuse [save-open-search] +# Save and reuse Discover sessions [save-open-search] -A saved Discover session is a convenient way to reuse a search that you’ve created in **Discover**. Discover sessions are good for saving a configured view of Discover to use later or adding search results to a dashboard, and can also serve as a foundation for building visualizations. +Save your **Discover** sessions to preserve your queries, filters, and view configurations for later use. Saved sessions let you quickly return to important searches, share explorations with your team, add search results to dashboards, or use them as a starting point for creating visualizations. ## Read-only access [discover-read-only-access] -If you don’t have sufficient privileges to save Discover sessions, the following indicator is displayed and the **Save** button is not visible. For more information, refer to [Granting access to {{kib}}](elasticsearch://reference/elasticsearch/roles.md). +If you don't have sufficient privileges to save Discover sessions, the following indicator is displayed and the **Save** button is not visible. For more information, refer to [Granting access to {{kib}}](elasticsearch://reference/elasticsearch/roles.md). :::{image} /explore-analyze/images/kibana-read-only-badge.png -:alt: Example of Discover's read only access indicator in Kibana's header +:alt: Example of Discover's read only access indicator in {{kib}}'s header :screenshot: ::: diff --git a/explore-analyze/discover/search-and-filter.md b/explore-analyze/discover/search-and-filter.md new file mode 100644 index 0000000000..cd7c589485 --- /dev/null +++ b/explore-analyze/discover/search-and-filter.md @@ -0,0 +1,139 @@ +--- +navigation_title: Search and filter data +mapped_pages: + - https://www.elastic.co/guide/en/kibana/current/discover.html#search-in-discover +applies_to: + stack: ga + serverless: ga +products: + - id: kibana +description: Search and filter your Elasticsearch data in Discover using KQL, Lucene, or ES|QL queries. Apply filters to narrow results and find the data you need. +--- + +# Search and filter data in Discover [search-in-discover] + +**Discover** combines powerful text search with structured filtering to help you find specific data quickly. Use KQL for user-friendly querying, Lucene for advanced patterns, or {{esql}} for piped queries. Apply visual filters to narrow results based on field values, ranges, or complex conditions. + +This guide shows you how to query your data effectively, build filters, and use the different query languages available in **Discover**. + +## Search with KQL or Lucene + +In the default mode of **Discover**, you can search your data using the [{{kib}} Query Language (KQL)](../query-filter/languages/kql.md) or [Lucene query syntax](../query-filter/languages/lucene-query-syntax.md). + +### Simple text search + +To search all fields, enter a simple string in the query bar: + +![Search field in Discover](/explore-analyze/images/kibana-discover-search-field.png "") + +:::{note} +Free text searches that don't specify a field may not return expected results depending on how the `index.query.default_field` index setting is configured for the indices matching the current {{data-source}}. +::: + +### Structured search with KQL + +To search particular fields and build more complex queries, use KQL. As you type, KQL prompts you with the fields you can search and the operators you can use to build a structured query. + +For example, to search the ecommerce sample data for documents where the country matches US: + +1. Enter `g`, and then select **geoip.country_iso_code**. +2. Select **:** for equals, and **US** for the value. +3. Press Enter or click the refresh button to run the query. + +For a more complex search, try: + +```ts +geoip.country_iso_code : US and products.taxless_price >= 75 +``` + +Learn more about [KQL syntax](../query-filter/languages/kql.md). + +## Apply filters + +In addition to the query bar, you can use filters to narrow down your results. Filters provide a visual way to build conditions and are particularly useful when you want to: + +* Exclude specific values +* Filter on multiple values for a field +* Build complex filter combinations +* Share filters across other applications + +### Add a filter + +To add a filter: + +1. Click the **Add filter** button (![Add icon](/explore-analyze/images/kibana-add-icon.png "")) next to the query bar. +2. In the **Add filter** pop-up: + * Select the **field** you want to filter on + * Choose an **operator** (is, is not, is one of, exists, and so on) + * Enter or select the **value** to filter by +3. Optionally, you can switch to **Edit as Query DSL** to write the filter as JSON. +4. Click **Add filter**. + +![Add filter dialog in Discover](/explore-analyze/images/kibana-discover-add-filter.png "") + +For example, to exclude results where the day of week is Wednesday: + +1. Set **Field** to `day_of_week` +2. Set **Operator** to `is not` +3. Set **Value** to `Wednesday` +4. Click **Add filter** + +### Quick filters from field values + +You can also create filters directly from the fields sidebar: + +1. Find a field in the sidebar and click it to see its top values. +2. Click the **+** icon next to a value to filter for that value, or the **-** icon to filter it out. + +### Filter pill actions + +Once added, filter pills appear below the query bar. You can interact with them in several ways: + +:::{include} ../_snippets/global-filters.md +::: + +## Search with {{esql}} + +You can use **Discover** with the {{es}} Query Language, {{esql}}. When using {{esql}}, you don't have to select a {{data-source}} - your query determines the data to explore. + +To switch to {{esql}} mode: + +1. Select **Try {{esql}}** from the **Discover** application menu bar. +2. If you've entered a KQL or Lucene query in the default mode, it automatically converts to {{esql}}. + +In {{esql}} mode, the **Documents** tab is renamed to **Results**. + +:::{important} +{applies_to}`stack: ga 9.1` When an {{esql}} query times out, partial results that are available are shown. The timeout is defined by the `search:timeout` advanced setting, which is set to 10 minutes (600000 ms) by default. In serverless projects, this advanced setting is not customizable and the timeout is set to 10 minutes. +::: + +Learn more about using {{esql}} in [Using {{esql}}](try-esql.md). + +## Find and highlight values in the table + +Use the in-table search to find and highlight specific values beyond what's currently visible on your screen. + +The in-table search looks for all matching values in all results and pages currently loaded in the table. The number of results loaded depends on the [Sample size](document-explorer.md#document-explorer-sample-size). If you load more results, the search automatically updates and reflects the new number of matching values, if any are found. + +```{tip} +You can navigate between results with your keyboard by pressing "Enter" to go to the next result, and "Shift + Enter" to go to the previous result. +``` + +![Using the in-table search and navigating through the matches](https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt30bf5f8b9a45ab74/67c234a787966d9fbc994ce0/in-table-search-demo.gif) + +## Inspect your queries + +To see the exact {{es}} query that **Discover** sent: + +:::{include} ../_snippets/inspect-request.md +::: + +This is useful for debugging queries, understanding how filters are translated to {{es}} queries, or copying queries to use in other tools. + +## Learn more + +* [{{kib}} Query Language (KQL)](../query-filter/languages/kql.md) +* [Lucene query syntax](../query-filter/languages/lucene-query-syntax.md) +* [Using {{esql}}](try-esql.md) +* [Filtering data](../query-filter/filtering.md) + diff --git a/explore-analyze/discover/show-field-statistics.md b/explore-analyze/discover/show-field-statistics.md index a26e5b0141..fe6de22892 100644 --- a/explore-analyze/discover/show-field-statistics.md +++ b/explore-analyze/discover/show-field-statistics.md @@ -6,14 +6,12 @@ applies_to: serverless: ga products: - id: kibana +description: View field statistics in Discover to analyze field distributions, top values, and data patterns. Explore numeric, text, date, and geographic fields with charts and summaries. --- -# View field statistics [show-field-statistics] +# View field statistics in Discover [show-field-statistics] -Explore the fields in your data with the **Field statistics** view in **Discover** and answer such questions as: - -* What does the latency look like when one of the containers is down on a Sunday? -* Is the field type and format in the data view appropriate for the data and its cardinality? +Use the **Field statistics** view in **Discover** to analyze your data fields with statistical summaries and visualizations. View distributions, top values, minimum and maximum values, and patterns across your data to understand field characteristics and identify trends. :::{note} Field statistics aren't available when **Discover** is in {{esql}} mode. @@ -22,7 +20,7 @@ Field statistics aren't available when **Discover** is in {{esql}} mode. This example explores the fields in the [sample web logs data](../index.md#gs-get-data-into-kibana), or you can use your own data. 1. Go to **Discover**. -2. Expand the {{data-source}} dropdown, and select **Kibana Sample Data Logs**. +2. Expand the {{data-source}} dropdown, and select **{{kib}} Sample Data Logs**. 3. If you don’t see any results, expand the time range, for example, to **Last 7 days**. 4. Click **Field statistics**. The table summarizes how many documents in the sample contain each field for the selected time period the number of distinct values, and the distribution. diff --git a/explore-analyze/discover/try-esql.md b/explore-analyze/discover/try-esql.md index 053350650f..c5c21675b5 100644 --- a/explore-analyze/discover/try-esql.md +++ b/explore-analyze/discover/try-esql.md @@ -6,16 +6,17 @@ applies_to: serverless: ga products: - id: kibana +description: Learn how to query and explore your Elasticsearch data using ES|QL in Discover. Build queries with piped commands to filter, modify, and visualize data. --- -# Using ES|QL [try-esql] +# Query and explore data using {{esql}} in Discover [try-esql] -The Elasticsearch Query Language, {{esql}}, makes it easier to explore your data without leaving Discover. +The {{es}} Query Language ({{esql}}) provides a powerful piped syntax for querying and modifying data directly in **Discover**. With {{esql}}, you can query your data without selecting a {{data-source}} first, build complex queries by chaining commands, and create visualizations from your query results. -The examples on this page use the {{kib}} sample web logs in Discover and Lens to explore the data and create visualizations. You can also install it by following [Add sample data](../index.md#gs-get-data-into-kibana). +This tutorial walks you through the fundamentals of using {{esql}} in **Discover**. You'll learn how to write queries with piped commands, filter and modify data, and create visualizations. The examples use {{kib}} sample web logs, but you can follow along with your own data. ::::{tip} -For the complete {{esql}} documentation, including all supported commands, functions, and operators, refer to the [{{esql}} reference](elasticsearch://reference/query-languages/esql/esql-syntax-reference.md). For a more detailed overview of {{esql}} in {{kib}}, refer to [Use {{esql}} in Kibana](../query-filter/languages/esql-kibana.md). +For the complete {{esql}} documentation, including all supported commands, functions, and operators, refer to the [{{esql}} reference](elasticsearch://reference/query-languages/esql/esql-syntax-reference.md). For a more detailed overview of {{esql}} in {{kib}}, refer to [Use {{esql}} in {{kib}}](../query-filter/languages/esql-kibana.md). :::: @@ -23,7 +24,7 @@ For the complete {{esql}} documentation, including all supported commands, funct ## Prerequisite [prerequisite] -To view the {{esql}} option in **Discover**, the `enableESQL` setting must be enabled from Kibana’s **Advanced Settings**. It is enabled by default. +To view the {{esql}} option in **Discover**, the `enableESQL` setting must be enabled from {{kib}}'s **Advanced Settings**. It is enabled by default. ## Use {{esql}} [tutorial-try-esql] @@ -34,7 +35,7 @@ To load the sample data: 2. Select **Try {{esql}}** from the application menu bar. :::{tip} - If you've entered a KQL or Lucene query in the default mode of Discover, it automatically converts to ES|QL. + If you've entered a KQL or Lucene query in the default mode of Discover, it automatically converts to {{esql}}. ::: Let’s say we want to find out what operating system users have and how much RAM is on their machine. @@ -93,16 +94,16 @@ We will now take it a step further to sort the data by machine ram and filter ou 3. Click **Save** to save the query and visualization to a dashboard. -### Edit the ES|QL visualization [_edit_the_esql_visualization] +### Edit the {{esql}} visualization [_edit_the_esql_visualization] You can make changes to the visualization by clicking the pencil icon. This opens additional settings that let you adjust the chart type, axes, breakdown, colors, and information displayed to your liking. If you’re not sure which route to go, check one of the suggestions available in the visualization editor. If you’d like to keep the visualization and add it to a dashboard, you can save it using the floppy disk icon. -### ES|QL and time series data [_esql_and_time_series_data] +### {{esql}} and time series data [_esql_and_time_series_data] -By default, ES|QL identifies time series data when an index contains a `@timestamp` field. This enables the time range selector and visualization options for your query. +By default, {{esql}} identifies time series data when an index contains a `@timestamp` field. This enables the time range selector and visualization options for your query. If your index doesn’t have an explicit `@timestamp` field, but has a different time field, you can still enable the time range selector and visualization options by calling the `?_tstart` and `?_tend` parameters in your query. @@ -136,11 +137,11 @@ stack: preview 9.2 serverless: preview ``` -In **Discover**, LOOKUP JOIN commands include interactive options that let you create or edit lookup indices directly from the editor. +In **Discover**, LOOKUP JOIN commands include interactive options that let you create or edit lookup indices directly from the {{esql}} editor. #### Create a lookup index from the editor -You can create a lookup index directly from the ES|QL editor. To populate this index, you can type in data manually or upload a CSV file up to 500 MB. +You can create a lookup index directly from the {{esql}} editor. To populate this index, you can type in data manually or upload a CSV file up to 500 MB. To create lookup indices, you need the [`create_index`](elasticsearch://reference/elasticsearch/security-privileges.md#privileges-list-indices) {{es}} privilege on the corresponding pattern. diff --git a/explore-analyze/discover/work-with-tabs.md b/explore-analyze/discover/work-with-tabs.md new file mode 100644 index 0000000000..6ccc11dd23 --- /dev/null +++ b/explore-analyze/discover/work-with-tabs.md @@ -0,0 +1,86 @@ +--- +navigation_title: Work with tabs +mapped_pages: + - https://www.elastic.co/guide/en/kibana/current/discover.html#discover-tabs +applies_to: + stack: preview 9.2 + serverless: preview +products: + - id: kibana +description: Work with multiple tabs in Discover to explore different aspects of your data simultaneously. Each tab maintains independent queries, filters, and views. +--- + +# Work with multiple tabs in Discover [discover-tabs] + +Use multiple tabs in **Discover** to run different explorations simultaneously. Each tab maintains its own independent query, filters, time range, and view settings, letting you compare different queries, test variations, or monitor multiple data streams without losing your work. + +## What's preserved in each tab + +Each tab maintains: + +* Query mode ({{esql}} or classic mode) +* Query text and filters +* Time range +* Selected data source +* Columns and sort order in the document table +* Active [context-aware experience](context-aware-discover.md) + +This independence means you can have completely different explorations running side by side, each tailored to a specific question or investigation. + +## Common use cases + +Multiple tabs are particularly useful for: + +* **Compare time periods**: Open multiple tabs with the same query but different time ranges to see how patterns change over time +* **Test query variations**: Duplicate a tab to experiment with different {{esql}} queries or filters without losing your original work +* **Switch contexts**: Keep separate tabs for logs, metrics, and traces explorations, each with its specialized interface +* **Test hypotheses**: Switch between different data sources or field combinations to compare approaches +* **Monitor multiple data streams**: Track different aspects of your system simultaneously + +## Create and open tabs + +**Start a new exploration** +: Select the {icon}`plus` icon next to the existing tabs to open a fresh tab with default settings. + +**Duplicate an existing tab** +: Hover over a tab and select the {icon}`boxes_vertical` **Actions** icon, then select **Duplicate**. This creates a new tab with all the settings from the current tab, perfect for testing variations. + +## Manage tabs + +**Rename a tab** +: Double-click the tab label, or hover over a tab and select the {icon}`boxes_vertical` **Actions** icon, then select **Rename**. Give your tabs meaningful names to track different investigations. + +**Reorder tabs** +: Drag and drop a tab to move it to a new position in the tab bar. Organize your tabs in the order that makes sense for your workflow. + +**Close a tab** +: Hover over a tab and select the {icon}`cross` icon. + +**Close multiple tabs at once** +: Hover over a tab and select the {icon}`boxes_vertical` **Actions** icon for options to: + * **Close other tabs** - Keep only the active tab open + * **Close tabs to the right** - Keep your first tabs and discard subsequent tabs + +**Start fresh** +: Select {icon}`plus` **New session** from the toolbar to discard all open tabs and start with a clean slate. Warning: Any unsaved changes to your current session are lost. + +**Reopen recently closed tabs** +: If you close a tab by mistake, select the {icon}`boxes_vertical` **Tabs bar menu** icon located at the end of the tab bar to retrieve recently closed tabs. + +## Save and restore tabs + +When you [save your Discover session](save-open-search.md), all currently open tabs are saved within the session. This means: + +* Your entire multi-tab workspace is preserved +* Each tab retains its individual settings +* When you reopen the saved session, all tabs are restored exactly as you left them + +This makes it easy to create saved workspaces for different investigation patterns you use regularly. + +## Tips for working with tabs + +* **Name your tabs meaningfully**: Use descriptive names like "Last 24h errors", "Prod versus Dev comparison", or "CPU metrics" instead of the default names +* **Keep related explorations together**: Group tabs by investigation topic and use the reorder feature to keep them adjacent +* **Save often**: Don't lose your work - save your multi-tab session when you've set up a useful workspace +* **Start with one good tab**: When you have a solid query, duplicate it to test variations rather than building from scratch + diff --git a/explore-analyze/find-and-organize/data-views.md b/explore-analyze/find-and-organize/data-views.md index 3bbf784817..3d1fd9b7ca 100644 --- a/explore-analyze/find-and-organize/data-views.md +++ b/explore-analyze/find-and-organize/data-views.md @@ -208,7 +208,7 @@ For detailed information on how to use runtime fields with {{es}}, refer to [Run #### Add runtime fields [create-runtime-fields] -To add runtime fields to your data views, open the data view you want to change, then define the field values by emitting a single value using the [Painless scripting language](../../explore-analyze/scripting/modules-scripting-painless.md). You can also add runtime fields in [**Discover**](../../explore-analyze/discover/discover-get-started.md#add-field-in-discover) and [**Lens**](../../explore-analyze/visualize/lens.md#change-the-fields). +To add runtime fields to your data views, open the data view you want to change, then define the field values by emitting a single value using the [Painless scripting language](../../explore-analyze/scripting/modules-scripting-painless.md). You can also add runtime fields in [**Discover**](../../explore-analyze/discover/add-fields-to-data-views.md) and [**Lens**](../../explore-analyze/visualize/lens.md#change-the-fields). 1. Go to the **Data Views** management page using the navigation menu or the [global search field](../../explore-analyze/find-and-organize/find-apps-and-objects.md). 2. Select the data view that you want to add the runtime field to, then click **Add field**. diff --git a/explore-analyze/toc.yml b/explore-analyze/toc.yml index 3cd8fbb37d..21ac7fff41 100644 --- a/explore-analyze/toc.yml +++ b/explore-analyze/toc.yml @@ -171,13 +171,21 @@ toc: - file: discover.md children: - file: discover/discover-get-started.md + - file: discover/try-esql.md + - file: discover/search-and-filter.md - file: discover/document-explorer.md - - file: discover/discover-search-for-relevance.md - file: discover/save-open-search.md - - file: discover/show-field-statistics.md - - file: discover/run-pattern-analysis-discover.md - - file: discover/background-search.md - - file: discover/try-esql.md + - file: discover/discover-advanced-guides.md + children: + - file: discover/compare-documents.md + - file: discover/add-fields-to-data-views.md + - file: discover/work-with-tabs.md + - file: discover/context-aware-discover.md + - file: discover/background-search.md + - file: discover/generate-alerts-from-discover.md + - file: discover/show-field-statistics.md + - file: discover/run-pattern-analysis-discover.md + - file: discover/discover-search-for-relevance.md - file: dashboards.md children: - file: dashboards/using.md diff --git a/explore-analyze/visualize/manage-panels.md b/explore-analyze/visualize/manage-panels.md index 19e6b161be..92450b8750 100644 --- a/explore-analyze/visualize/manage-panels.md +++ b/explore-analyze/visualize/manage-panels.md @@ -84,7 +84,7 @@ There are three types of **Discover** interactions you can add to dashboard pane To use series data interactions, click a data series in the panel. -* **Discover session interactions** — Opens [saved Discover session](../discover.md#save-your-search) data in **Discover**. +* **Discover session interactions** — Opens [saved Discover session](../discover/save-open-search.md) data in **Discover**. To use saved Discover session interactions, open the panel menu and click **View Discover session**. diff --git a/redirects.yml b/redirects.yml index c06d373213..06abe38f20 100644 --- a/redirects.yml +++ b/redirects.yml @@ -536,6 +536,13 @@ redirects: # Search sessions becoming background search 'explore-analyze/discover/search-sessions.md': 'explore-analyze/discover/background-search.md' + # Sections extracted from discover-get-started.md into separate pages + 'explore-analyze/discover/discover-get-started.md#context-aware-discover': 'explore-analyze/discover/context-aware-discover.md' + 'explore-analyze/discover/discover-get-started.md#add-field-in-discover': 'explore-analyze/discover/add-fields-to-data-views.md' + 'explore-analyze/discover/discover-get-started.md#compare-documents-in-discover': 'explore-analyze/discover/compare-documents.md' + 'explore-analyze/discover/discover-get-started.md#copy-row-content': 'explore-analyze/discover/compare-documents.md' + 'explore-analyze/discover/discover-get-started.md#search-in-discover': 'explore-analyze/discover/search-and-filter.md' + 'explore-analyze/discover/discover-get-started.md#alert-from-Discover': 'explore-analyze/discover/generate-alerts-from-discover.md' # Related to https://github.com/elastic/docs-content/pull/3493 'solutions/security/cloud/ingest-third-party-cloud-security-data.md': 'solutions/security/cloud/integrations/ingest-third-party-cloud-security-data.md'