Skip to content
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
bce2755
update explore logs page
mdbirnstiehl Mar 5, 2025
b5067bd
fix links
mdbirnstiehl Mar 5, 2025
850d762
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Mar 6, 2025
5c3b64b
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Mar 6, 2025
7dde8fd
update references to Logs Explorer
mdbirnstiehl Mar 6, 2025
564dba5
remove logs explorer mention
mdbirnstiehl Mar 6, 2025
3913e3b
update nginx tutorial
mdbirnstiehl Mar 6, 2025
3bda0fa
remove logs explorer references
mdbirnstiehl Mar 6, 2025
073bdde
fix error
mdbirnstiehl Mar 6, 2025
f7d9d08
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Mar 7, 2025
9a9b479
fix error
mdbirnstiehl Mar 7, 2025
27a7115
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Mar 7, 2025
9088aca
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Mar 27, 2025
76f9cff
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Mar 27, 2025
3e28699
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Apr 1, 2025
0e84956
fix links
mdbirnstiehl Apr 1, 2025
3df6235
delete raw files
mdbirnstiehl Apr 1, 2025
8dd138c
remove logs app files
mdbirnstiehl Apr 1, 2025
a84aef7
change references to log sources
mdbirnstiehl Apr 1, 2025
22dc27d
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Apr 2, 2025
9ad3dc0
remove logs explorer mentions
mdbirnstiehl Apr 2, 2025
c4202bc
review updates
mdbirnstiehl Apr 2, 2025
74fb40a
review updates
mdbirnstiehl Apr 4, 2025
514bc5e
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Apr 4, 2025
7c55509
remove log explorer fields docs
mdbirnstiehl Apr 7, 2025
9b0b40e
add redirect
mdbirnstiehl Apr 7, 2025
0b8813f
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Apr 7, 2025
f1ec33d
Merge branch 'main' into logs-explorer-removal
mdbirnstiehl Apr 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion reference/fleet/monitor-elastic-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,6 @@ On the **Logs** tab you can filter, search, and explore the agent logs:

* Change the log level to filter the view by log levels. Want to see debugging logs? Refer to [Change the logging level](#change-logging-level).
* Change the time range to view historical logs.
* Click **Open in Logs** to tail agent log files in real time. For more information about logging, refer to [Tail log files](/solutions/observability/logs/logs-stream.md).


## Change the logging level [change-logging-level]
Expand Down
2 changes: 1 addition & 1 deletion reference/observability/fields-and-object-schemas.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Beat modules (for example, [{{filebeat}} modules](beats://reference/filebeat/fil

This reference covers:

* [Logs Explorer fields](/reference/observability/fields-and-object-schemas/logs-app-fields.md)
* [Logs Discover fields](/reference/observability/fields-and-object-schemas/logs-app-fields.md)
* [{{infrastructure-app}} fields](/reference/observability/fields-and-object-schemas/metrics-app-fields.md)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ mapped_pages:
- https://www.elastic.co/guide/en/observability/current/logs-app-fields.html
---

# Logs Explorer fields [logs-app-fields]
# Discover logs fields [logs-app-fields]

This section lists the required fields the **Logs Explorer** uses to display data. Please note that some of the fields listed are not [ECS fields](ecs://reference/index.md#_what_is_ecs).

This section lists the required fields Discover uses to display log data. Please note that some of the fields listed are not [ECS fields](ecs://reference/index.md#_what_is_ecs).

`@timestamp`
: Date/time when the event originated.
Expand Down
2 changes: 1 addition & 1 deletion reference/observability/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ mapped_pages:
This section contains reference information for Elastic Observability features, including:

* Fields reference
* Logs Explorer fields
* Logs Discover fields
* Infrastructure app fields
* Elastic Entity Model

Expand Down
2 changes: 1 addition & 1 deletion solutions/observability/apps/collect-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ See the [Open Telemetry Metrics API](https://github.com/open-telemetry/opentelem
Use **Discover** to validate that metrics are successfully reported to {{kib}}.

1. Open your Observability instance.
2. Find **Discover** in the main menu or use the [global search field](../../../get-started/the-stack.md#kibana-navigation-search), and select the **Logs Explorer** tab.
2. Find **Discover** in the main menu or use the [global search field](../../../get-started/the-stack.md#kibana-navigation-search).
3. Click **All logs** → **Data Views** then select **APM**.
4. Filter the data to only show documents with metrics: `processor.name :"metric"`
5. Narrow your search with a known OpenTelemetry field. For example, if you have an `order_value` field, add `order_value: *` to your search to return only OpenTelemetry metrics documents.
Expand Down
7 changes: 4 additions & 3 deletions solutions/observability/cloud/monitor-cloudtrail-logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,10 +209,11 @@ Navigate to {{kib}} and choose among the following monitoring options:
:alt: Visualize CloudTrail logs with Disocver
:::

* **Visualize your logs with Logs explorer**
* **Visualize your logs with Discover**

:::{image} ../../images/observability-firehose-cloudtrail-logsexplorer.png
:alt: Visualize CloudTrail logs with Discover

:::{image} /solutions/images/observability-firehose-cloudtrail-logsexplorer.png
:alt: Visualize CloudTrail logs with Logs explorer
:::

* **Visualize your logs with the CloudTrail Dashboard**
Expand Down
23 changes: 0 additions & 23 deletions solutions/observability/cloud/monitor-microsoft-azure-openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,6 @@ Now that your log and metric data is streaming to {{es}}, you can view them in {

* [View logs and metrics with the overview dashboard](#azure-openai-overview-dashboard): Use the built-in overview dashboard for insight into your Azure OpenAI service like total requests and token usage.
* [View logs and metrics with Discover](#azure-openai-discover): Use Discover to find and filter your log and metric data based on specific fields.
* [View logs with Logs Explorer](#azure-openai-logs-explorer): Use Logs Explorer for an in-depth view into your logs.


### View logs and metrics with the overview dashboard [azure-openai-overview-dashboard]
Expand Down Expand Up @@ -279,28 +278,6 @@ From here, filter your data and dive deeper into individual logs to find informa

For more on using Discover and creating data views, refer to the [Discover](../../../explore-analyze/discover.md) documentation.


### View logs with Logs Explorer [azure-openai-logs-explorer]

To view Azure OpenAI logs, open {{kib}} and go to **Logs Explorer** (find `Logs Explorer` in the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md)). With **Logs Explorer**, you can quickly search and filter your log data, get information about the structure of log fields, and display your findings in a visualization.

:::{image} /solutions/images/observability-log-explorer.png
:alt: screenshot of the logs explorer main page
:screenshot:
:::

From **Logs Explorer**, you can select the Azure OpenAI integration from the data selector to view your Kubernetes data.

![screenshot of the logs explorer data selector](/solutions/images/observability-azure-open-ai-data-selector.png "")

From here, filter your log data and dive deeper into individual logs to find information and troubleshoot issues. For a list of Azure OpenAI fields you may want to filter by, refer to the [Azure OpenAI integration](https://docs.elastic.co/en/integrations/azure_openai#settings) documentation.

For more on Logs Explorer, refer to:

* [Logs Explorer](../logs/logs-explorer.md) for an overview of Logs Explorer.
* [Filter logs in Logs Explorer](../logs/filter-aggregate-logs.md#logs-filter-logs-explorer) for more on filtering logs in Logs Explorer.


## Step 6: Monitor Microsoft Azure OpenAI APM with OpenTelemetry [azure-openai-apm]

The Azure OpenAI API provides useful data to help monitor and understand your code. Using OpenTelemetry, you can ingest this data into Elastic {{observability}}. From there, you can view and analyze your data to monitor the cost and performance of your applications.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ To ingest Azure subscription and resource logs into Elastic, you use the Azure N
::::

3. In {{kib}}, under **{{observability}}**, find **Overview** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). Refresh the page until you see some data. This may take a few minutes.
4. To analyze your subscription and resource logs, click **Show Logs Explorer**.
4. To analyze your subscription and resource logs, click **Show Logs**.


## Step 3: Ingest logs and metrics from your virtual machines (VMs) [azure-ingest-VM-logs-metrics]
Expand All @@ -112,7 +112,7 @@ To ingest Azure subscription and resource logs into Elastic, you use the Azure N
:::

3. Wait until the extension is installed and sending data (if the list does not update, click **Refresh** ).
4. Back in {{kib}}, view the **Logs Explorer** again. Notice that you can filter the view to show logs for a specific instance, for example `cloud.instance.name : "ingest-tutorial-linux"`.
4. Back in {{kib}}, view the **Discover** again. Notice that you can filter the view to show logs for a specific instance, for example `cloud.instance.name : "ingest-tutorial-linux"`.
5. To view VM metrics, go to **Infrastructure inventory** and then select a VM. (To open **Infrastructure inventory**, find **Infrastructure** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).)

To explore the data further, click **Open as page**.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ To ingest Azure subscription and resource logs into Elastic using the Microsoft
::::

3. In {{kib}}, find the {{observability}} **Overview** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). Refresh the page until you see some data. This may take a few minutes.
4. To analyze your subscription and resource logs, click **Show Logs Explorer**.
4. To analyze your subscription and resource logs, click **Show Logs**.


## Step 3: Ingest logs and metrics from your virtual machines. [azure-step-three]
Expand All @@ -104,7 +104,7 @@ To ingest Azure subscription and resource logs into Elastic using the Microsoft

![Select VMs to collect logs and metrics from](/solutions/images/observability-monitor-azure-elastic-vms.png "")

3. Wait until it is installed and sending data (if the list does not update, click **Refresh** ). To see the logs from the VM, open **Logs Explorer** (find `Logs Explorer` in the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md)).
3. Wait until it is installed and sending data (if the list does not update, click **Refresh** ). To see the logs from the VM, open **Discover** (find `Discover` in the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md)).

To view VM metrics, go to **Infrastructure inventory** and then select a VM. (To open **Infrastructure inventory**, find **Infrastructure** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).)

Expand Down
10 changes: 5 additions & 5 deletions solutions/observability/data-set-quality-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
mapped_pages:
- https://www.elastic.co/guide/en/observability/current/monitor-datasets.html
- https://www.elastic.co/guide/en/serverless/current/observability-monitor-datasets.html
navigation_title: "Data set quality"
applies_to:
stack: beta
serverless: beta
navigation_title: "Data set quality"
---

# Data set quality monitoring [observability-monitor-datasets]
Expand Down Expand Up @@ -35,7 +35,7 @@ Opening the details of a specific data set shows the degraded documents history,

## Investigate issues [observability-monitor-datasets-investigate-issues]

The Data Set Quality page has a couple of different ways to help you find ignored fields and investigate issues. From the data set table, you can open the data set’s details page, and view commonly ignored fields and information about those fields. Open a logs data set in Logs Explorer or other data set types in Discover to find ignored fields in individual documents.
The Data Set Quality page has a couple of different ways to help you find ignored fields and investigate issues. From the data set table, you can open the data set’s details page, and view commonly ignored fields and information about those fields. Open a logs data set in Discover or other data set types in Discover to find ignored fields in individual documents.


### Find ignored fields in data sets [observability-monitor-datasets-find-ignored-fields-in-data-sets]
Expand All @@ -50,12 +50,12 @@ The **Quality issues** section shows fields that have been ignored, the number o

### Find ignored fields in individual logs [observability-monitor-datasets-find-ignored-fields-in-individual-logs]

To use Logs Explorer or Discover to find ignored fields in individual logs:
To use Discover to find ignored fields in individual logs:

1. Find data sets with degraded documents using the **Degraded Docs** column of the data sets table.
2. Click the percentage in the **Degraded Docs** column to open the data set in Logs Explorer or Discover.
2. Click the percentage in the **Degraded Docs** column to open the data set in Discover.

The **Documents** table in Logs Explorer or Discover is automatically filtered to show documents that were not parsed correctly. Under the **actions** column, you’ll find the degraded document icon (![degraded document icon](/solutions/images/serverless-indexClose.svg "")).
The **Documents** table in Discover is automatically filtered to show documents that were not parsed correctly. Under the **actions** column, you’ll find the degraded document icon (![degraded document icon](../images/serverless-indexClose.svg "")).

Now that you know which documents contain ignored fields, examine them more closely to find the origin of the issue:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ After installation is complete and all relevant data is flowing into Elastic, th
| Integration asset | Description |
| --- | --- |
| **Apache** | Prebuilt dashboard for monitoring Apache HTTP server health using error and access log data. |
| **Custom .log files** | Logs Explorer for analyzing custom logs. |
| **Custom .log files** | Discover for analyzing custom logs. |
| **Docker** | Prebuilt dashboard for monitoring the status and health of Docker containers. |
| **MySQL** | Prebuilt dashboard for monitoring MySQl server health using error and access log data. |
| **Nginx** | Prebuilt dashboard for monitoring Nginx server health using error and access log data. |
Expand All @@ -160,7 +160,7 @@ For host monitoring, the following capabilities and features are recommended:
* [Detect anomalies](../../../solutions/observability/infra-and-hosts/detect-metric-anomalies.md) for memory usage and network traffic on hosts.
* [Create alerts](../../../solutions/observability/incident-management/alerting.md) that notify you when an anomaly is detected or a metric exceeds a given value.

* In the [Logs Explorer](../../../solutions/observability/logs/logs-explorer.md), search and filter your log data, get information about the structure of log fields, and display your findings in a visualization. You can also:
* In [Discover](../../../solutions/observability/logs/logs-explorer.md), search and filter your log data, get information about the structure of log fields, and display your findings in a visualization. You can also:

* [Monitor log data set quality](../../../solutions/observability/data-set-quality-monitoring.md) to find degraded documents.
* [Run a pattern analysis](../../../explore-analyze/machine-learning/machine-learning-in-kibana/xpack-ml-aiops.md#log-pattern-analysis) to find patterns in unstructured log messages.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ Logs are collected from setup onward, so you won’t see logs that occurred befo
:::::


Under **Visualize your data**, you’ll see links to **Logs Explorer** to view your logs and **Hosts** to view your host metrics.
Under **Visualize your data**, you’ll see links to **Discover** to view your logs and **Hosts** to view your host metrics.


## Gain deeper insight into your host data [_get_value_out_of_your_data]
Expand All @@ -130,7 +130,7 @@ After using the Hosts page and Discover to confirm you’ve ingested all the hos
* [Detect anomalies](../../../solutions/observability/infra-and-hosts/detect-metric-anomalies.md) for memory usage and network traffic on hosts.
* [Create alerts](../../../solutions/observability/incident-management/create-manage-rules.md) that notify you when an anomaly is detected or a metric exceeds a given value.

* In the [Logs Explorer](../../../solutions/observability/logs/logs-explorer.md), search and filter your log data, get information about the structure of log fields, and display your findings in a visualization. You can also:
* In [Discover](../../../solutions/observability/logs/logs-explorer.md), search and filter your log data, get information about the structure of log fields, and display your findings in a visualization. You can also:

* [Monitor log data set quality](../../../solutions/observability/data-set-quality-monitoring.md) to find degraded documents.
* [Run a pattern analysis](../../../explore-analyze/machine-learning/machine-learning-in-kibana/xpack-ml-aiops.md#log-pattern-analysis) to find patterns in unstructured log messages.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ mapped_pages:

Analyze log data from your hosts, services, Kubernetes, Apache, and many more.

In **Logs Explorer** (powered by Discover), you can quickly search and filter your log data, get information about the structure of the fields, and display your findings in a visualization.
In **Discover**, you can quickly search and filter your log data, get information about the structure of the fields, and display your findings in a visualization.

:::{image} /solutions/images/serverless-log-explorer-overview.png
:alt: Logs Explorer showing log events
:screenshot:
:::{image} ../../images/serverless-log-explorer-overview.png
:alt: Discover showing log events
:class: screenshot
:::

[Learn more about log monitoring →](../../../solutions/observability/logs.md)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -332,7 +332,7 @@ When a rule check is performed, a query is built based on the configuration of t

## Settings [settings]

With log threshold rules, it’s not possible to set an explicit index pattern as part of the configuration. The index pattern is instead inferred from **Log indices** on the [Settings](../logs/configure-data-sources.md) page of the {{logs-app}}.
With log threshold rules, it’s not possible to set an explicit index pattern as part of the configuration. The index pattern is instead inferred from **Log sources** at **Stack Management** → **Advanced settings** under **Observability**.

With each execution of the rule check, the **Log indices** setting is checked, but it is not stored when the rule is created.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ From the main menu, go to **Infrastructure** → **Infrastructure inventory** or
Click **Apply** to save your changes.

::::{note}
The patterns used to match log sources are configured in the Logs app. The default setting is `logs-*,filebeat-*,kibana_sample_data_logs*`. To change the default, refer to [Configure data sources](../../../solutions/observability/logs/configure-data-sources.md).
The patterns used to match log sources are configured in the Logs app. The default setting is `logs-*,filebeat-*,kibana_sample_data_logs*`. To change the default, go to **Log sources** at **Stack Management** → **Advanced settings** under **Observability**.
::::


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -519,23 +519,12 @@ For more on using the **Metrics Explorer** page, refer to [Explore infrastructur

### View Kubernetes logs [monitor-k8s-explore-logs]

Find `Logs Explorer` in the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).
Find `Discover` in the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).

With **Logs Explorer**, you can quickly search and filter your log data, get information about the structure of log fields, and display your findings in a visualization.
From the **Data view** menu, select `All logs`. From here, you can quickly search and filter your log data, get information about the structure of log fields, and display your findings in a visualization. Then, you can filter your log data and dive deeper into individual logs to find and troubleshoot issues. For more information, refer to:

:::{image} /solutions/images/observability-log-explorer.png
:alt: screenshot of the logs explorer main page
:screenshot:
:::

From **Logs Explorer**, you can select the Kubernetes integration from the data selector to view your Kubernetes data.

![screenshot of the logs explorer main page](/solutions/images/observability-logs-explorer-applications.png "")

From here, you can filter your log data and dive deeper into individual logs to find and troubleshoot issues. For more information, refer to:

* [Logs Explorer](../logs/logs-explorer.md) for an over view of Logs Explorer.
* [Filter logs in Logs Explorer](../logs/filter-aggregate-logs.md#logs-filter-logs-explorer) for more on filtering logs in Logs Explorer.
* [Explore logs in Discover](../logs/logs-explorer.md) for an overview of viewing your logs in Discover.
* [Filter logs in Discover](../logs/filter-aggregate-logs.md#logs-filter-discover) for more on filtering logs in Discover.


## Part 4: Monitor application performance [monitor-kubernetes-application-performance]
Expand Down
Loading
Loading