diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-act-on-data.md b/raw-migrated-files/docs-content/serverless/observability-apm-act-on-data.md deleted file mode 100644 index 9fc1e9d3e5..0000000000 --- a/raw-migrated-files/docs-content/serverless/observability-apm-act-on-data.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -navigation_title: "Act on data" ---- - -# Act on application data [observability-apm-act-on-data] - - -In addition to exploring visualizations in the Applications UI, you can make your application data more actionable with: - -| | | -| --- | --- | -| [Rules and alerts](../../../solutions/observability/apps/create-apm-rules-alerts.md) | The Applications UI allows you to define rules to detect complex conditions within your APM data and trigger built-in actions when those conditions are met. | -| [Custom links](../../../solutions/observability/apps/create-custom-links.md) | Build URLs that contain relevant metadata from a specific trace. For example, you can create a link that will take you to a page where you can open a new GitHub issue with context already auto-populated in the issue body. These links will be shown in the *Actions* context menu in selected areas of the Applications UI (for example, by transaction details). | - - - diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-agents-elastic-apm-agents.md b/raw-migrated-files/docs-content/serverless/observability-apm-agents-elastic-apm-agents.md deleted file mode 100644 index 77827b01ac..0000000000 --- a/raw-migrated-files/docs-content/serverless/observability-apm-agents-elastic-apm-agents.md +++ /dev/null @@ -1,226 +0,0 @@ ---- -navigation_title: "APM agents" ---- - -# Elastic APM agents [observability-apm-agents-elastic-apm-agents] - - -::::{admonition} Required role -:class: note - -The **Admin** role or higher is required to use APM agents. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). - -:::: - - -Elastic APM agents automatically measure application performance and track errors. They offer built-in support for popular frameworks and technologies, and provide easy-to-use APIs that allow you to instrument any application. - -Elastic APM agents are built and maintained by Elastic. While they are similar, different programming languages have different nuances and requirements. Select your preferred language below to learn more about how each agent works. - -:::::::{tab-set} - -::::::{tab-item} Java -**Elastic APM Java agent** - -The Elastic APM Java agent auto-instruments supported technologies and records interesting events, like spans for database queries and transactions for incoming HTTP requests. To do this, it leverages the capability of the JVM to instrument the bytecode of classes. This means that for the supported technologies, there are no code changes required. - -Spans are grouped in transactions—by default, one for each incoming HTTP request. But it’s possible to create custom transactions not associated with an HTTP request. Transactions and Spans are sent to Elastic, where they’re transformed, stored, and ready to be visualized. - -**Learn more** - -If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). - -See the [Java agent reference](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/index.md) for full documentation, including: - -* [Supported technologies](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/supported-technologies.md) -* [Set up](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/set-up-apm-java-agent.md) -* [Configuration reference](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/configuration.md) -* [API reference](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/tracing-apis.md) - -::::{important} -Not all APM agent configuration options are compatible with Elastic Cloud serverless. - -:::: -:::::: - -::::::{tab-item} Node.js -**Elastic APM Node.js agent** - -The Elastic APM Node.js agent auto-instruments supported frameworks and records interesting events, like HTTP requests and database queries. To do this, it patches modules as they are loaded to capture when module functions and callbacks are called. Additionally, there are some cases where a module will be patched to allow tracing context to be propagated through the asynchronous continuation. This means that for the supported technologies, there are no code changes required. - -The Agent automatically links module function calls to callback calls to measure their duration and metadata (like the DB statement), as well as HTTP-related information (like the URL, parameters, and headers). - -These events, called Transactions and Spans, are sent to Elastic, where they’re transformed, stored, and ready to be visualized. - -**Learn more** - -If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). - -See the [Node.js agent reference](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/index.md) for full documentation, including: - -* [Supported technologies](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/supported-technologies.md) -* [Set up](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/set-up.md) -* [Configuration reference](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/advanced-setup.md) -* [API reference](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/api.md) - -::::{important} -Not all APM agent configuration options are compatible with Elastic Cloud serverless. - -:::: -:::::: - -::::::{tab-item} Python -**Elastic APM Python agent** - -The Elastic APM Python agent has built-in support for Django and Flask performance metrics and error logging, as well as generic support of other WSGI frameworks for error logging. - -It instruments your application to collect APM events in a few different ways: - -To collect data about incoming requests and background tasks, the Agent integrates with supported technologies to make use of hooks and signals provided by the framework. These framework integrations require limited code changes in your application. - -To collect data from database drivers, HTTP libraries, and so on, Elastic APM agents instrument certain functions and methods in these libraries. Instrumentations are set up automatically and do not require any code changes. - -In addition to APM and error data, the Python agent also collects system and application metrics in regular intervals. This collection happens in a background thread that is started by the agent. - -**Learn more** - -If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). - -See the [Python agent reference](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/index.md) for full documentation, including: - -* [Supported technologies](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/supported-technologies.md) -* [Set up](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/set-up-apm-python-agent.md) -* [Configuration reference](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/configuration.md) -* [API reference](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/api-reference.md) - -::::{important} -Not all APM agent configuration options are compatible with Elastic Cloud serverless. - -:::: -:::::: - -::::::{tab-item} Ruby -**Elastic APM Ruby agent** - -The Elastic APM Ruby agent auto-instruments supported technologies and records interesting events, like HTTP requests and database queries. To do this, it uses relevant public APIs when they are provided by the libraries. Otherwise, it carefully wraps the necessary internal methods. This means that for the supported technologies, there are no code changes required. - -The APM agent automatically keeps track of queries to your data stores to measure their duration and metadata (like the DB statement), as well as HTTP-related information (like the URL, parameters, and headers). - -These events, called Transactions and Spans, are sent to Elastic, where they’re transformed, stored, and ready to be visualized. - -**Learn more** - -If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). - -See the [Ruby agent reference](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/index.md) for full documentation, including: - -* [Supported technologies](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/supported-technologies.md) -* [Set up](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/set-up-apm-ruby-agent.md) -* [Configuration reference](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/configuration.md) -* [API reference](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/api-reference.md) - -::::{important} -Not all APM agent configuration options are compatible with Elastic Cloud serverless. - -:::: -:::::: - -::::::{tab-item} Go -**Elastic APM Go agent** - -The Elastic APM Go agent enables you to trace the execution of operations in your [Go](https://golang.org/) applications. It has built-in support for popular frameworks and toolkits, like [Gorilla](http://www.gorillatoolkit.org/) and [Gin](https://gin-gonic.com/), as well as support for instrumenting Go’s built-in [net/http](https://golang.org/pkg/net/http/), [database/sql](https://golang.org/pkg/database/sql/) drivers. - -The Agent includes instrumentation modules for supported technologies, each providing middleware or wrappers for recording interesting events, such as incoming HTTP requests, outgoing HTTP requests, and database queries. - -To collect data about incoming HTTP requests, install router middleware for one of the supported web frameworks. Incoming requests will be recorded as transactions, along with any related panics or errors. - -To collect data for outgoing HTTP requests, instrument an `http.Client` or `http.Transport` using `module/apmhttp`. To collect data about database queries, use `module/apmsql`, which provides instrumentation for well-known database drivers. - -In order to connect transactions with related spans and errors, and propagate traces between services (distributed tracing), the agent relies on Go’s built-in [context](https://golang.org/pkg/context/) package: transactions and spans are stored in context objects. For example, for incoming HTTP requests, in-flight trace data will be recorded in the `context` object accessible through [net/http.Context](https://golang.org/pkg/net/http/#Request.Context). - -In addition to capturing events like those mentioned here, the agent also collects system and application metrics at regular intervals. This collection happens in a background goroutine that is automatically started when the agent is initialized. - -**Learn more** - -If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). - -See the [Go agent reference](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/index.md) for full documentation, including: - -* [Supported technologies](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/supported-technologies.md) -* [Set up](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/set-up-apm-go-agent.md) -* [Configuration reference](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/configuration.md) -* [API reference](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/api-documentation.md) - -::::{important} -Not all APM agent configuration options are compatible with Elastic Cloud serverless. - -:::: -:::::: - -::::::{tab-item} .NET -**Elastic APM .NET agent** - -The Elastic APM .NET agent auto-instruments supported technologies and records interesting events, like HTTP requests and database queries. To do this, it uses built-in capabilities of the instrumented frameworks like [Diagnostic Source](https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.diagnosticsource?view=netcore-3.0), an HTTP module for IIS, or [IDbCommandInterceptor](https://docs.microsoft.com/en-us/dotnet/api/system.data.entity.infrastructure.interception.idbcommandinterceptor?view=entity-framework-6.2.0) for Entity Framework. This means that for the supported technologies, there are no code changes required beyond enabling auto-instrumentation. - -The Agent automatically registers callback methods for built-in Diagnostic Source events. With this, the supported frameworks trigger Agent code for relevant events to measure their duration and collect metadata, like DB statements, as well as HTTP related information, like the URL, parameters, and headers. These events, called Transactions and Spans, are sent to Elastic, where they’re transformed, stored, and ready to be visualized. - -**Learn more** - -If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). - -See the [.NET agent reference](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/index.md) for full documentation, including: - -* [Supported technologies](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/supported-technologies.md) -* [Set up](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/set-up-apm-net-agent.md) -* [Configuration reference](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/configuration.md) -* [API reference](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/public-api.md) - -::::{important} -Not all APM agent configuration options are compatible with Elastic Cloud serverless. - -:::: -:::::: - -::::::{tab-item} PHP -**Elastic APM PHP agent** - -The Elastic APM PHP agent measures application performance and tracks errors. This extension must be installed in your PHP environment. - -**Learn more** - -If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). - -See the [PHP agent reference](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/index.md) for full documentation, including: - -* [Supported technologies](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/supported-technologies.md) -* [Set up](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/set-up-apm-php-agent.md) -* [Configuration reference](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/configuration.md) -* [API reference](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/public-api.md) - -::::{important} -Not all APM agent configuration options are compatible with Elastic Cloud serverless. - -:::: -:::::: - -::::::: - -## Minimum supported versions [observability-apm-agents-elastic-apm-agents-minimum-supported-versions] - -The following versions of Elastic APM agents are supported: - -| Agent name | Agent version | -| --- | --- | -| **APM AWS Lambda extension** | ≥`1.x` | -| **Go agent** | ≥`1.x` | -| **Java agent** | ≥`1.x` | -| **.NET agent** | ≥`1.x` | -| **Node.js agent** | ≥`4.x` | -| **PHP agent** | ≥`1.x` | -| **Python agent** | ≥`6.x` | -| **Ruby agent** | ≥`3.x` | - -::::{note} -Some recently added features may require newer agent versions than those listed above. In these instances, the required APM agent versions will be documented with the feature. - -:::: diff --git a/raw-migrated-files/observability-docs/observability/apm-act-on-data.md b/raw-migrated-files/observability-docs/observability/apm-act-on-data.md deleted file mode 100644 index cca7738e97..0000000000 --- a/raw-migrated-files/observability-docs/observability/apm-act-on-data.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -navigation_title: "Act on data" ---- - -# Act on application data [apm-act-on-data] - - -In addition to exploring visualizations in the Applications UI in {{kib}}, you can make your application data more actionable with: - -| | | -| --- | --- | -| [Rules and alerts](../../../solutions/observability/apps/create-apm-rules-alerts.md) | The Applications UI allows you to define rules to detect complex conditions within your APM data and trigger built-in actions when those conditions are met. | -| [Custom links](../../../solutions/observability/apps/create-custom-links.md) | Build URLs that contain relevant metadata from a specific trace. For example, you can create a link that will take you to a page where you can open a new GitHub issue with context already auto-populated in the issue body. These links will be shown in the *Actions* context menu in selected areas of the Applications UI (for example, by transaction details). | - - - diff --git a/raw-migrated-files/observability-docs/observability/apm-agents.md b/raw-migrated-files/observability-docs/observability/apm-agents.md deleted file mode 100644 index fa4a54fed2..0000000000 --- a/raw-migrated-files/observability-docs/observability/apm-agents.md +++ /dev/null @@ -1,31 +0,0 @@ -# APM agents [apm-agents] - -Elastic APM agents automatically measure application performance and track errors. They offer built-in support for popular frameworks and technologies, and provide easy-to-use APIs that allow you to instrument any application. - - -## Available APM agents [_available_apm_agents] - -Elastic APM agents are built and maintained by Elastic. While they are similar, different programming languages have different nuances and requirements. Refer to each APM agent’s documentation to learn how to install the agent, configure it, and more: - -* [APM Android agent](asciidocalypse://docs/apm-agent-android/docs/reference/ingestion-tools/apm-agent-android/index.md) -* [APM Go agent](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/index.md) -* [APM iOS agent](asciidocalypse://docs/apm-agent-ios/docs/reference/ingestion-tools/apm-agent-swift/index.md) -* [APM Java agent](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/index.md) -* [APM .NET agent](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/index.md) -* [APM Node.js agent](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/index.md) -* [APM PHP agent](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/index.md) -* [APM Python agent](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/index.md) -* [APM Ruby agent](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/index.md) -* [APM Real User Monitoring JavaScript agent](asciidocalypse://docs/apm-agent-rum-js/docs/reference/ingestion-tools/apm-agent-rum-js/index.md) - - -## More APM agent resources [_more_apm_agent_resources] - -Though you should refer to each APM agent’s documentation for information on using the agent, there are some tasks you can complete in the Applications UI: - -* [Centrally configure APM agents in Kibana](../../../solutions/observability/apps/apm-agent-central-configuration.md) -* [Upload a RUM source map](../../../solutions/observability/apps/create-upload-source-maps-rum.md) - - - - diff --git a/raw-migrated-files/observability-docs/observability/apm-getting-started-apm-server.md b/raw-migrated-files/observability-docs/observability/apm-getting-started-apm-server.md deleted file mode 100644 index 440ed2eeb1..0000000000 --- a/raw-migrated-files/observability-docs/observability/apm-getting-started-apm-server.md +++ /dev/null @@ -1,149 +0,0 @@ ---- -navigation_title: "Get started" ---- - -# Get started with APM [apm-getting-started-apm-server] - - -::::{note} -Starting in version 8.15.0, the {{es}} apm-data plugin manages APM index templates, lifecycle policies, and ingest pipelines. -:::: - - -The APM Server receives performance data from your APM agents, validates and processes it, and then transforms the data into {{es}} documents. If you’re on this page, then you’ve chosen to self-manage the Elastic Stack, and you now must decide how to run and configure the APM Server. There are two options, and the components required are different for each: - -* **[Fleet-managed APM Server](../../../solutions/observability/apps/get-started-with-apm.md#apm-setup-fleet-managed-apm)** -* **[APM Server binary](../../../solutions/observability/apps/get-started-with-apm.md#apm-setup-apm-server-binary)** - - -## Fleet-managed APM Server [apm-setup-fleet-managed-apm] - -Fleet is a web-based UI in {{kib}} that is used to centrally manage {{agent}}s. In this deployment model, use {{agent}} to spin up APM Server instances that can be centrally-managed in a custom-curated user interface. - -:::{image} ../../../images/observability-fm-ov.png -:alt: APM Server fleet overview -::: - -**Pros**: - -* Conveniently manage one, some, or many different integrations from one central {{fleet}} UI. -* Centrally manage multiple APM Servers running on edge machines. - -**Supported outputs**: - -* {{es}} -* {{ess}} - -::::{note} -Fleet-managed APM Server does *not* support all the outputs that are supported by the APM Server binary method of running Elastic APM. -:::: - - -**Required components**: - -* APM agents -* {{agent}} (which runs multiple subprocesses including APM Server, Fleet Server, and {{stack}}) - -**Configuration method**: {{kib}} UI - - -## APM Server binary [apm-setup-apm-server-binary] - -Install, configure, and run the APM Server binary wherever you need it. - -:::{image} ../../../images/observability-bin-ov.png -:alt: APM Server binary overview -::: - -**Pros**: - -* Simplest self-managed option -* No addition component knowledge required -* YAML configuration simplifies automation - -**Supported outputs**: - -* {{es}} -* {{ess}} -* {{ls}} -* Kafka -* Redis -* File -* Console - -**Required components**: - -* APM agents -* APM Server -* {{stack}} - -**Configuration method**: YAML - - -## Help me decide [_help_me_decide] - -This decision tree highlights key factors to help you make an informed decision about implementing Elastic APM. It provides practical guidance and is not intended to serve as a comprehensive reference of all possible implementations and capabilities. - -
- - - -APM Server binary - -Fleet-managed APM Server -YAML - - - - -UI - -Curated UI or YAML-based? -What is your preferred method of configuration? - -no - - - - -yes - -Integrations collect observability data from specific services and protect endpoints. -Do you want to use other integrations with APM Server? - -Logstash or -Kafka - - -Elasticsearch - - - -Most users need Elasticsearch -Which output are you using? - -APM agents live on edge machines a single APM Server is hosted centrally -but -Centrally hosted - - -APM agents APM Server live on each edge machine -and -Edge machines - - -Note that central is required for RUM. -Where will you run APM Server? - -no - - -yes - - - -Elastic Cloud simplifies the management and deployment of Elastic APM. -Are you using Elastic Cloud? - - -
diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index 64611163b1..4d5d501cb3 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -211,8 +211,6 @@ toc: - file: docs-content/serverless/ingest-wiz-data.md - file: docs-content/serverless/intro.md - file: docs-content/serverless/observability-ai-assistant.md - - file: docs-content/serverless/observability-apm-act-on-data.md - - file: docs-content/serverless/observability-apm-agents-elastic-apm-agents.md - file: docs-content/serverless/observability-apm-get-started.md - file: docs-content/serverless/observability-ecs-application-logs.md - file: docs-content/serverless/observability-plaintext-application-logs.md @@ -438,9 +436,6 @@ toc: - file: logstash/logstash/ts-logstash.md - file: observability-docs/observability/index.md children: - - file: observability-docs/observability/apm-act-on-data.md - - file: observability-docs/observability/apm-agents.md - - file: observability-docs/observability/apm-getting-started-apm-server.md - file: observability-docs/observability/index.md - file: observability-docs/observability/obs-ai-assistant.md - file: security-docs/security/index.md diff --git a/solutions/observability/apps/common-problems.md b/solutions/observability/apps/common-problems.md deleted file mode 100644 index 044cc150b9..0000000000 --- a/solutions/observability/apps/common-problems.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -mapped_urls: - - https://www.elastic.co/guide/en/observability/current/apm-common-problems.html - - https://www.elastic.co/guide/en/serverless/current/observability-apm-troubleshooting.html ---- - -% This page exists in the Troubleshoot section (troubleshoot/observability/apm/common-problems.md), so probably OK to delete from Solutions? - - -# Common problems - -% What needs to be done: Align serverless/stateful - -% Use migrated content from existing pages that map to this page: - -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-common-problems.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-troubleshooting.md - -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): - -$$$apm-ssl-client-fails$$$ diff --git a/solutions/observability/apps/configure-logstash-output.md b/solutions/observability/apps/configure-logstash-output.md index c82a594434..468255c407 100644 --- a/solutions/observability/apps/configure-logstash-output.md +++ b/solutions/observability/apps/configure-logstash-output.md @@ -397,8 +397,7 @@ Validation for this test fails because the certificate is not valid for the spec curl: (51) SSL: certificate verification failed (result: 5) ``` -See the [troubleshooting docs](common-problems.md#apm-ssl-client-fails) for info about resolving this issue. - +See the [troubleshooting docs](../../../troubleshoot/observability/apm/common-problems.md#apm-ssl-client-fails) for info about resolving this issue. ### Test the APM Server to {{ls}} connection [_test_the_apm_server_to_ls_connection] @@ -408,5 +407,5 @@ If you have APM Server running as a service, first stop the service. Then test y apm-server -c apm-server.yml -e -v ``` -Any errors will be printed to the console. See the [troubleshooting docs](common-problems.md#apm-ssl-client-fails) for info about resolving common errors. +Any errors will be printed to the console. See the [troubleshooting docs](../../../troubleshoot/observability/apm/common-problems.md#apm-ssl-client-fails) for info about resolving common errors. diff --git a/solutions/observability/apps/configure-synthetics-projects.md b/solutions/observability/apps/configure-synthetics-projects.md index 0d86ee32c1..6bb6cd0906 100644 --- a/solutions/observability/apps/configure-synthetics-projects.md +++ b/solutions/observability/apps/configure-synthetics-projects.md @@ -13,7 +13,7 @@ Create a `synthetics.config.js` or `synthetics.config.ts` file in the root of th ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack ```ts import type { SyntheticsConfig } from '@elastic/synthetics'; diff --git a/solutions/observability/apps/configure-synthetics-settings.md b/solutions/observability/apps/configure-synthetics-settings.md index 322e189fcf..2863da9688 100644 --- a/solutions/observability/apps/configure-synthetics-settings.md +++ b/solutions/observability/apps/configure-synthetics-settings.md @@ -91,7 +91,7 @@ Project API keys are used to push {{project-monitors}} remotely from a CLI or CD In the **Project API keys** tab, you can generate project API keys to use with your projects. Learn more about using API keys in [Use {{project-monitors-cap}}](../../../solutions/observability/apps/create-monitors-with-project-monitors.md). ::::{important} -In an Elastic Stack v9 deployment, to create a Project API key you must be logged into {{kib}} as a user with the privileges described in [Writer role](../../../solutions/observability/apps/writer-role.md). +**In an Elastic Stack deployment**, to create a Project API key you must be logged into {{kib}} as a user with the privileges described in [Writer role](../../../solutions/observability/apps/writer-role.md). In a serverless project, to create a Project API key you must be logged in as a user with [Editor](../../../solutions/observability/apps/grant-users-access-to-secured-resources.md) access. diff --git a/solutions/observability/apps/create-monitors-with-project-monitors.md b/solutions/observability/apps/create-monitors-with-project-monitors.md index 0492676f5b..d83d574c06 100644 --- a/solutions/observability/apps/create-monitors-with-project-monitors.md +++ b/solutions/observability/apps/create-monitors-with-project-monitors.md @@ -66,7 +66,7 @@ Then, follow the prompts on screen to set up the correct default variables for y ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. To generate an API key: @@ -201,7 +201,7 @@ For more details on writing journeys and configuring browser monitors, refer to ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack While inside the project directory you can do two things with the `npx @elastic/synthetics` command: diff --git a/solutions/observability/apps/elastic-apm-agents.md b/solutions/observability/apps/elastic-apm-agents.md index 84160056bd..1e70975f59 100644 --- a/solutions/observability/apps/elastic-apm-agents.md +++ b/solutions/observability/apps/elastic-apm-agents.md @@ -2,17 +2,229 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-agents.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-agents-elastic-apm-agents.html + +navigation_title: "APM agents" --- -# Elastic APM agents +# Elastic APM agents [observability-apm-agents-elastic-apm-agents] + + +::::{admonition} Required role +:class: note + +**For Observability serverless projects**, the **Admin** role or higher is required to use APM agents. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). + +:::: + + +Elastic APM agents automatically measure application performance and track errors. They offer built-in support for popular frameworks and technologies, and provide easy-to-use APIs that allow you to instrument any application. + +Elastic APM agents are built and maintained by Elastic. While they are similar, different programming languages have different nuances and requirements. Select your preferred language below to learn more about how each agent works. + +:::::::{tab-set} + +::::::{tab-item} Java +**Elastic APM Java agent** + +The Elastic APM Java agent auto-instruments supported technologies and records interesting events, like spans for database queries and transactions for incoming HTTP requests. To do this, it leverages the capability of the JVM to instrument the bytecode of classes. This means that for the supported technologies, there are no code changes required. + +Spans are grouped in transactions—by default, one for each incoming HTTP request. But it’s possible to create custom transactions not associated with an HTTP request. Transactions and Spans are sent to Elastic, where they’re transformed, stored, and ready to be visualized. + +**Learn more** + +If you're ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). + +See the [Java agent reference](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/index.md) for full documentation, including: + +* [Supported technologies](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/supported-technologies.md) +* [Set up](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/set-up-apm-java-agent.md) +* [Configuration reference](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/configuration.md) +* [API reference](asciidocalypse://docs/apm-agent-java/docs/reference/ingestion-tools/apm-agent-java/tracing-apis.md) + +::::{important} +Not all APM agent configuration options are compatible with Elastic Cloud serverless. + +:::: +:::::: + +::::::{tab-item} Node.js +**Elastic APM Node.js agent** + +The Elastic APM Node.js agent auto-instruments supported frameworks and records interesting events, like HTTP requests and database queries. To do this, it patches modules as they are loaded to capture when module functions and callbacks are called. Additionally, there are some cases where a module will be patched to allow tracing context to be propagated through the asynchronous continuation. This means that for the supported technologies, there are no code changes required. + +The Agent automatically links module function calls to callback calls to measure their duration and metadata (like the DB statement), as well as HTTP-related information (like the URL, parameters, and headers). + +These events, called Transactions and Spans, are sent to Elastic, where they're transformed, stored, and ready to be visualized. + +**Learn more** + +If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). + +See the [Node.js agent reference](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/index.md) for full documentation, including: + +* [Supported technologies](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/supported-technologies.md) +* [Set up](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/set-up.md) +* [Configuration reference](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/advanced-setup.md) +* [API reference](asciidocalypse://docs/apm-agent-nodejs/docs/reference/ingestion-tools/apm-agent-nodejs/api.md) + +::::{important} +Not all APM agent configuration options are compatible with Elastic Cloud serverless. + +:::: +:::::: + +::::::{tab-item} Python +**Elastic APM Python agent** + +The Elastic APM Python agent has built-in support for Django and Flask performance metrics and error logging, as well as generic support of other WSGI frameworks for error logging. + +It instruments your application to collect APM events in a few different ways: + +To collect data about incoming requests and background tasks, the Agent integrates with supported technologies to make use of hooks and signals provided by the framework. These framework integrations require limited code changes in your application. + +To collect data from database drivers, HTTP libraries, and so on, Elastic APM agents instrument certain functions and methods in these libraries. Instrumentations are set up automatically and do not require any code changes. + +In addition to APM and error data, the Python agent also collects system and application metrics in regular intervals. This collection happens in a background thread that is started by the agent. + +**Learn more** + +If you’re ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). + +See the [Python agent reference](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/index.md) for full documentation, including: + +* [Supported technologies](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/supported-technologies.md) +* [Set up](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/set-up-apm-python-agent.md) +* [Configuration reference](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/configuration.md) +* [API reference](asciidocalypse://docs/apm-agent-python/docs/reference/ingestion-tools/apm-agent-python/api-reference.md) + +::::{important} +Not all APM agent configuration options are compatible with Elastic Cloud serverless. + +:::: +:::::: + +::::::{tab-item} Ruby +**Elastic APM Ruby agent** + +The Elastic APM Ruby agent auto-instruments supported technologies and records interesting events, like HTTP requests and database queries. To do this, it uses relevant public APIs when they are provided by the libraries. Otherwise, it carefully wraps the necessary internal methods. This means that for the supported technologies, there are no code changes required. + +The APM agent automatically keeps track of queries to your data stores to measure their duration and metadata (like the DB statement), as well as HTTP-related information (like the URL, parameters, and headers). + +These events, called Transactions and Spans, are sent to Elastic, where they're transformed, stored, and ready to be visualized. + +**Learn more** + +If you're ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). + +See the [Ruby agent reference](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/index.md) for full documentation, including: + +* [Supported technologies](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/supported-technologies.md) +* [Set up](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/set-up-apm-ruby-agent.md) +* [Configuration reference](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/configuration.md) +* [API reference](asciidocalypse://docs/apm-agent-ruby/docs/reference/ingestion-tools/apm-agent-ruby/api-reference.md) + +::::{important} +Not all APM agent configuration options are compatible with Elastic Cloud serverless. + +:::: +:::::: + +::::::{tab-item} Go +**Elastic APM Go agent** + +The Elastic APM Go agent enables you to trace the execution of operations in your [Go](https://golang.org/) applications. It has built-in support for popular frameworks and toolkits, like [Gorilla](http://www.gorillatoolkit.org/) and [Gin](https://gin-gonic.com/), as well as support for instrumenting Go’s built-in [net/http](https://golang.org/pkg/net/http/), [database/sql](https://golang.org/pkg/database/sql/) drivers. + +The Agent includes instrumentation modules for supported technologies, each providing middleware or wrappers for recording interesting events, such as incoming HTTP requests, outgoing HTTP requests, and database queries. + +To collect data about incoming HTTP requests, install router middleware for one of the supported web frameworks. Incoming requests will be recorded as transactions, along with any related panics or errors. + +To collect data for outgoing HTTP requests, instrument an `http.Client` or `http.Transport` using `module/apmhttp`. To collect data about database queries, use `module/apmsql`, which provides instrumentation for well-known database drivers. + +In order to connect transactions with related spans and errors, and propagate traces between services (distributed tracing), the agent relies on Go’s built-in [context](https://golang.org/pkg/context/) package: transactions and spans are stored in context objects. For example, for incoming HTTP requests, in-flight trace data will be recorded in the `context` object accessible through [net/http.Context](https://golang.org/pkg/net/http/#Request.Context). + +In addition to capturing events like those mentioned here, the agent also collects system and application metrics at regular intervals. This collection happens in a background goroutine that is automatically started when the agent is initialized. + +**Learn more** + +If you're ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). + +See the [Go agent reference](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/index.md) for full documentation, including: + +* [Supported technologies](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/supported-technologies.md) +* [Set up](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/set-up-apm-go-agent.md) +* [Configuration reference](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/configuration.md) +* [API reference](asciidocalypse://docs/apm-agent-go/docs/reference/ingestion-tools/apm-agent-go/api-documentation.md) + +::::{important} +Not all APM agent configuration options are compatible with Elastic Cloud serverless. + +:::: +:::::: + +::::::{tab-item} .NET +**Elastic APM .NET agent** + +The Elastic APM .NET agent auto-instruments supported technologies and records interesting events, like HTTP requests and database queries. To do this, it uses built-in capabilities of the instrumented frameworks like [Diagnostic Source](https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.diagnosticsource?view=netcore-3.0), an HTTP module for IIS, or [IDbCommandInterceptor](https://docs.microsoft.com/en-us/dotnet/api/system.data.entity.infrastructure.interception.idbcommandinterceptor?view=entity-framework-6.2.0) for Entity Framework. This means that for the supported technologies, there are no code changes required beyond enabling auto-instrumentation. + +The Agent automatically registers callback methods for built-in Diagnostic Source events. With this, the supported frameworks trigger Agent code for relevant events to measure their duration and collect metadata, like DB statements, as well as HTTP related information, like the URL, parameters, and headers. These events, called Transactions and Spans, are sent to Elastic, where they're transformed, stored, and ready to be visualized. + +**Learn more** + +If you're ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). + +See the [.NET agent reference](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/index.md) for full documentation, including: + +* [Supported technologies](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/supported-technologies.md) +* [Set up](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/set-up-apm-net-agent.md) +* [Configuration reference](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/configuration.md) +* [API reference](asciidocalypse://docs/apm-agent-dotnet/docs/reference/ingestion-tools/apm-agent-dotnet/public-api.md) + +::::{important} +Not all APM agent configuration options are compatible with Elastic Cloud serverless. + +:::: +:::::: + +::::::{tab-item} PHP +**Elastic APM PHP agent** + +The Elastic APM PHP agent measures application performance and tracks errors. This extension must be installed in your PHP environment. + +**Learn more** + +If you're ready to give Elastic APM a try, see [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). + +See the [PHP agent reference](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/index.md) for full documentation, including: + +* [Supported technologies](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/supported-technologies.md) +* [Set up](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/set-up-apm-php-agent.md) +* [Configuration reference](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/configuration.md) +* [API reference](asciidocalypse://docs/apm-agent-php/docs/reference/ingestion-tools/apm-agent-php/public-api.md) + +::::{important} +Not all APM agent configuration options are compatible with Elastic Cloud serverless. + +:::: +:::::: + +::::::: -% What needs to be done: Align serverless/stateful +## Minimum supported versions [observability-apm-agents-elastic-apm-agents-minimum-supported-versions] -% Use migrated content from existing pages that map to this page: +The following versions of Elastic APM agents are supported: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-agents.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-agents-elastic-apm-agents.md +| Agent name | Agent version | +| --- | --- | +| **APM AWS Lambda extension** | ≥`1.x` | +| **Go agent** | ≥`1.x` | +| **Java agent** | ≥`1.x` | +| **.NET agent** | ≥`1.x` | +| **Node.js agent** | ≥`4.x` | +| **PHP agent** | ≥`1.x` | +| **Python agent** | ≥`6.x` | +| **Ruby agent** | ≥`3.x` | -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +::::{note} +Some recently added features may require newer agent versions than those listed above. In these instances, the required APM agent versions will be documented with the feature. -$$$observability-apm-agents-elastic-apm-agents-minimum-supported-versions$$$ \ No newline at end of file +:::: \ No newline at end of file diff --git a/solutions/observability/apps/find-transaction-latency-failure-correlations.md b/solutions/observability/apps/find-transaction-latency-failure-correlations.md index 7a814d5be9..d777f55727 100644 --- a/solutions/observability/apps/find-transaction-latency-failure-correlations.md +++ b/solutions/observability/apps/find-transaction-latency-failure-correlations.md @@ -14,7 +14,7 @@ To find correlations: ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack Select a service on the **Services** page in the Applications UI then select a transaction group from the **Transactions** tab. diff --git a/solutions/observability/apps/get-started-with-apm.md b/solutions/observability/apps/get-started-with-apm.md index 37ae0cfacf..a9ccfcb6a8 100644 --- a/solutions/observability/apps/get-started-with-apm.md +++ b/solutions/observability/apps/get-started-with-apm.md @@ -2,19 +2,159 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-getting-started-apm-server.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-get-started.html + +navigation_title: "Get started" --- -# Get started with APM +# Get started with APM [apm-getting-started-apm-server] -% What needs to be done: Align serverless/stateful -% Use migrated content from existing pages that map to this page: +::::{note} +Starting in version 8.15.0, the {{es}} apm-data plugin manages APM index templates, lifecycle policies, and ingest pipelines. +:::: + + +The APM Server receives performance data from your APM agents, validates and processes it, and then transforms the data into {{es}} documents. If you’re on this page, then you’ve chosen to self-manage the Elastic Stack, and you now must decide how to run and configure the APM Server. There are two options, and the components required are different for each: + +* **[Fleet-managed APM Server](../../../solutions/observability/apps/get-started-with-apm.md#apm-setup-fleet-managed-apm)** +* **[APM Server binary](../../../solutions/observability/apps/get-started-with-apm.md#apm-setup-apm-server-binary)** + + +## Fleet-managed APM Server [apm-setup-fleet-managed-apm] + +Fleet is a web-based UI in {{kib}} that is used to centrally manage {{agent}}s. In this deployment model, use {{agent}} to spin up APM Server instances that can be centrally-managed in a custom-curated user interface. + +:::{image} ../../../images/observability-fm-ov.png +:alt: APM Server fleet overview +::: + +**Pros**: + +* Conveniently manage one, some, or many different integrations from one central {{fleet}} UI. +* Centrally manage multiple APM Servers running on edge machines. + +**Supported outputs**: + +* {{es}} +* {{ess}} + +::::{note} +Fleet-managed APM Server does *not* support all the outputs that are supported by the APM Server binary method of running Elastic APM. +:::: + + +**Required components**: + +* APM agents +* {{agent}} (which runs multiple subprocesses including APM Server, Fleet Server, and {{stack}}) -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-getting-started-apm-server.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-get-started.md +**Configuration method**: {{kib}} UI -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): -$$$apm-setup-apm-server-binary$$$ +## APM Server binary [apm-setup-apm-server-binary] + +Install, configure, and run the APM Server binary wherever you need it. + +:::{image} ../../../images/observability-bin-ov.png +:alt: APM Server binary overview +::: + +**Pros**: + +* Simplest self-managed option +* No addition component knowledge required +* YAML configuration simplifies automation + +**Supported outputs**: + +* {{es}} +* {{ess}} +* {{ls}} +* Kafka +* Redis +* File +* Console + +**Required components**: + +* APM agents +* APM Server +* {{stack}} + +**Configuration method**: YAML + + +## Help me decide [_help_me_decide] + +This decision tree highlights key factors to help you make an informed decision about implementing Elastic APM. It provides practical guidance and is not intended to serve as a comprehensive reference of all possible implementations and capabilities. + +
+ + + +APM Server binary + +Fleet-managed APM Server +YAML + + + + +UI + +Curated UI or YAML-based? +What is your preferred method of configuration? + +no + + + + +yes + +Integrations collect observability data from specific services and protect endpoints. +Do you want to use other integrations with APM Server? + +Logstash or +Kafka + + +Elasticsearch + + + +Most users need Elasticsearch +Which output are you using? + +APM agents live on edge machines a single APM Server is hosted centrally +but +Centrally hosted + + +APM agents APM Server live on each edge machine +and +Edge machines + + +Note that central is required for RUM. +Where will you run APM Server? + +no + + +yes + + + +Elastic Cloud simplifies the management and deployment of Elastic APM. +Are you using Elastic Cloud? + + +
+ + +% What needs to be done: Align serverless/stateful + +% Use migrated content from existing pages that map to this page: -$$$apm-setup-fleet-managed-apm$$$ \ No newline at end of file +% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-get-started.md \ No newline at end of file diff --git a/solutions/observability/apps/grant-users-access-to-secured-resources.md b/solutions/observability/apps/grant-users-access-to-secured-resources.md index e7da241a85..e8610c4d0a 100644 --- a/solutions/observability/apps/grant-users-access-to-secured-resources.md +++ b/solutions/observability/apps/grant-users-access-to-secured-resources.md @@ -11,7 +11,7 @@ You can use role-based access control to grant users access to secured resources ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack Typically you need the create the following separate roles: diff --git a/solutions/observability/apps/manage-data-retention.md b/solutions/observability/apps/manage-data-retention.md index f15d0ee73a..83c064b032 100644 --- a/solutions/observability/apps/manage-data-retention.md +++ b/solutions/observability/apps/manage-data-retention.md @@ -36,7 +36,7 @@ To find Synthetics data streams: ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. Navigate to [{{kib}} index management](../../../manage-data/lifecycle/index-lifecycle-management/index-management-in-kibana.md). diff --git a/solutions/observability/apps/services.md b/solutions/observability/apps/services.md index ebe1fbcb35..7aad1f24cb 100644 --- a/solutions/observability/apps/services.md +++ b/solutions/observability/apps/services.md @@ -54,7 +54,7 @@ To create a service group: ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. To open **Service inventory**, find **Applications** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). diff --git a/solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md b/solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md index 5d86dd67e6..99fe4d9f9d 100644 --- a/solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md +++ b/solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md @@ -25,7 +25,7 @@ Connect your OpenTelemetry Collector instances to Elastic {{observability}} or { ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack @@ -159,7 +159,7 @@ Define environment variables to configure the OpenTelemetry agent or SDK and ena ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack ```bash diff --git a/solutions/observability/apps/use-synthetics-cli.md b/solutions/observability/apps/use-synthetics-cli.md index 907daeb417..38135da948 100644 --- a/solutions/observability/apps/use-synthetics-cli.md +++ b/solutions/observability/apps/use-synthetics-cli.md @@ -284,7 +284,7 @@ Generate a Time-based One-Time Password (TOTP) for multifactor authentication (M ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack ```sh diff --git a/solutions/observability/get-started.md b/solutions/observability/get-started.md index 0dce4663a7..b98814113b 100644 --- a/solutions/observability/get-started.md +++ b/solutions/observability/get-started.md @@ -18,7 +18,7 @@ New to Elastic {{observability}}? Discover more about our observability features Learn about key features available to help you get value from your observability data: * [What is Elastic {{observability}}?](../../solutions/observability/get-started/what-is-elastic-observability.md) -* [What’s new in Elastic Stack v9.0](https://www.elastic.co/guide/en/observability/current/whats-new.html) +* [What’s new in Elastic Stack](https://www.elastic.co/guide/en/observability/current/whats-new.html) * [{{obs-serverless}} billing dimensions](../../deploy-manage/cloud-organization/billing/elastic-observability-billing-dimensions.md) @@ -74,7 +74,7 @@ Ready to dig into more features of Elastic Observability? See these guides: * [Create an alert](../../solutions/observability/incident-management/alerting.md) * [Create a service-level objective (SLO)](../../solutions/observability/incident-management/create-an-slo.md) -## Related content for Elastic Stack v9.0 [_related_content] +## Related content for Elastic Stack [_related_content] * [Starting with the {{es}} Platform and its Solutions](/get-started/index.md) for new users * [Adding data to {{es}}](../../manage-data/ingest.md) for other ways to ingest data \ No newline at end of file diff --git a/solutions/observability/get-started/quickstart-collect-data-with-aws-firehose.md b/solutions/observability/get-started/quickstart-collect-data-with-aws-firehose.md index 8390984bb2..2517f70917 100644 --- a/solutions/observability/get-started/quickstart-collect-data-with-aws-firehose.md +++ b/solutions/observability/get-started/quickstart-collect-data-with-aws-firehose.md @@ -67,7 +67,7 @@ Data collection with AWS Firehose is supported on ESS deployments in AWS, Azure ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack * A deployment using our hosted {{ess}} on [{{ecloud}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body). The deployment includes an {{es}} cluster for storing and searching your data, and {{kib}} for visualizing and managing your data. @@ -144,7 +144,7 @@ The following table shows the type of data ingested by the supported AWS service ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. In {{kib}}, go to the **Observability** UI and click **Add Data**. diff --git a/solutions/observability/get-started/quickstart-monitor-hosts-with-elastic-agent.md b/solutions/observability/get-started/quickstart-monitor-hosts-with-elastic-agent.md index 8a37edd80f..db826b37fe 100644 --- a/solutions/observability/get-started/quickstart-monitor-hosts-with-elastic-agent.md +++ b/solutions/observability/get-started/quickstart-monitor-hosts-with-elastic-agent.md @@ -18,7 +18,7 @@ The script also generates an {{agent}} configuration file that you can use with ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack * An {{es}} cluster for storing and searching your data, and {{kib}} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. To get started quickly, try out our hosted {{ess}} on [{{ecloud}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body). @@ -59,7 +59,7 @@ The script also generates an {{agent}} configuration file that you can use with ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. In {{kib}}, go to the **Observability** UI and click **Add Data**. diff --git a/solutions/observability/get-started/quickstart-monitor-hosts-with-opentelemetry.md b/solutions/observability/get-started/quickstart-monitor-hosts-with-opentelemetry.md index 61c133bc71..5c7e32f6d1 100644 --- a/solutions/observability/get-started/quickstart-monitor-hosts-with-opentelemetry.md +++ b/solutions/observability/get-started/quickstart-monitor-hosts-with-opentelemetry.md @@ -19,7 +19,7 @@ In this quickstart guide, you’ll learn how to monitor your hosts using the Ela ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack * An {{es}} cluster for storing and searching your data, and {{kib}} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. The quickest way to get started with this quickstart is using a trial project on [Elastic serverless](https://docs.elastic.co/serverless/quickstart-monitor-hosts-with-otel.html). @@ -60,7 +60,7 @@ Follow these steps to collect logs and metrics using the EDOT Collector: ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. In {{kib}}, go to the **Observability** UI and click **Add Data**. diff --git a/solutions/observability/get-started/quickstart-monitor-kubernetes-cluster-with-elastic-agent.md b/solutions/observability/get-started/quickstart-monitor-kubernetes-cluster-with-elastic-agent.md index e118dfb8d1..c2c201edeb 100644 --- a/solutions/observability/get-started/quickstart-monitor-kubernetes-cluster-with-elastic-agent.md +++ b/solutions/observability/get-started/quickstart-monitor-kubernetes-cluster-with-elastic-agent.md @@ -18,7 +18,7 @@ The kubectl command installs the standalone Elastic Agent in your Kubernetes clu ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack * An {{es}} cluster for storing and searching your data, and {{kib}} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. To get started quickly, try out our hosted {{ess}} on [{{ecloud}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body). @@ -54,7 +54,7 @@ The kubectl command installs the standalone Elastic Agent in your Kubernetes clu ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. In {{kib}}, go to the **Observability** UI and click **Add Data**. diff --git a/solutions/observability/get-started/quickstart-unified-kubernetes-observability-with-elastic-distributions-of-opentelemetry-edot.md b/solutions/observability/get-started/quickstart-unified-kubernetes-observability-with-elastic-distributions-of-opentelemetry-edot.md index 0fb7abb6b7..8868dd37af 100644 --- a/solutions/observability/get-started/quickstart-unified-kubernetes-observability-with-elastic-distributions-of-opentelemetry-edot.md +++ b/solutions/observability/get-started/quickstart-unified-kubernetes-observability-with-elastic-distributions-of-opentelemetry-edot.md @@ -28,7 +28,7 @@ For a more detailed description of the components and advanced configuration, re ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack * An {{es}} cluster for storing and searching your data, and {{kib}} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. To get started quickly, try out our hosted {{ess}} on [{{ecloud}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body). @@ -57,7 +57,7 @@ For a more detailed description of the components and advanced configuration, re ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. In {{kib}}, go to the **Observability** UI and click **Add Data**. diff --git a/solutions/observability/incident-management/create-an-slo-burn-rate-rule.md b/solutions/observability/incident-management/create-an-slo-burn-rate-rule.md index 043a0c32e2..f36d44dcc3 100644 --- a/solutions/observability/incident-management/create-an-slo-burn-rate-rule.md +++ b/solutions/observability/incident-management/create-an-slo-burn-rate-rule.md @@ -14,7 +14,7 @@ navigation_title: "SLO burn rate" **For Observability serverless projects**, The **Editor** role or higher is required to create SLOs. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). -**For Elastic Stack v9**, to create and manage SLOs, you need an [appropriate license](https://www.elastic.co/subscriptions), an {{es}} cluster with both `transform` and `ingest` [node roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#node-roles) present, and [SLO access](../../../solutions/observability/incident-management/configure-service-level-objective-slo-access.md) must be configured. +**For Elastic Stack**, to create and manage SLOs, you need an [appropriate license](https://www.elastic.co/subscriptions), an {{es}} cluster with both `transform` and `ingest` [node roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#node-roles) present, and [SLO access](../../../solutions/observability/incident-management/configure-service-level-objective-slo-access.md) must be configured. :::: diff --git a/solutions/observability/incident-management/create-an-slo.md b/solutions/observability/incident-management/create-an-slo.md index 84998f6f02..b69df795cd 100644 --- a/solutions/observability/incident-management/create-an-slo.md +++ b/solutions/observability/incident-management/create-an-slo.md @@ -11,7 +11,7 @@ mapped_urls: **For Observability serverless projects**, The **Editor** role or higher is required to create SLOs. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). -**For Elastic Stack v9**, to create and manage SLOs, you need an [appropriate license](https://www.elastic.co/subscriptions), an {{es}} cluster with both `transform` and `ingest` [node roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#node-roles) present, and [SLO access](../../../solutions/observability/incident-management/configure-service-level-objective-slo-access.md) must be configured. +**For Elastic Stack**, to create and manage SLOs, you need an [appropriate license](https://www.elastic.co/subscriptions), an {{es}} cluster with both `transform` and `ingest` [node roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#node-roles) present, and [SLO access](../../../solutions/observability/incident-management/configure-service-level-objective-slo-access.md) must be configured. :::: @@ -28,7 +28,7 @@ From here, complete the following steps: 3. [Describe your SLO](../../../solutions/observability/incident-management/create-an-slo.md#slo-describe). ::::{note} -**For Elastic Stack v9**, the cluster must include one or more nodes with both `ingest` and `transform` [roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#node-roles). The roles can exist on the same node or be distributed across separate nodes. On ESS deployments (Elastic Cloud), this is handled by the hot nodes, which serve as both `ingest` and `transform` nodes. +**For Elastic Stack**, the cluster must include one or more nodes with both `ingest` and `transform` [roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#node-roles). The roles can exist on the same node or be distributed across separate nodes. On ESS deployments (Elastic Cloud), this is handled by the hot nodes, which serve as both `ingest` and `transform` nodes. :::: diff --git a/solutions/observability/incident-management/create-manage-cases.md b/solutions/observability/incident-management/create-manage-cases.md index 499e5d6856..4ea9f92f78 100644 --- a/solutions/observability/incident-management/create-manage-cases.md +++ b/solutions/observability/incident-management/create-manage-cases.md @@ -30,7 +30,7 @@ Open a new case to keep track of issues and share the details with colleagues. T **For Observability serverless projects**, you can add users who are assigned the Editor user role (or a more permissive role) for the project. - **For Elastic Stack v9**, You can add users only if they meet the necessary [prerequisites](../../../solutions/observability/incident-management/configure-access-to-cases.md). + **For Elastic Stack**, You can add users only if they meet the necessary [prerequisites](../../../solutions/observability/incident-management/configure-access-to-cases.md). 6. If you defined [custom fields](../../../solutions/observability/incident-management/configure-case-settings.md#case-custom-fields), they appear in the **Additional fields** section. 7. (Optional) Under External incident management system, you can select a connector to send cases to an external system. If you’ve created any connectors previously, they will be listed here. If there are no connectors listed, you can [create one](../../../solutions/observability/incident-management/configure-case-settings.md). diff --git a/solutions/observability/infra-and-hosts/detect-metric-anomalies.md b/solutions/observability/infra-and-hosts/detect-metric-anomalies.md index 2c1c724a16..e63ce0f9ab 100644 --- a/solutions/observability/infra-and-hosts/detect-metric-anomalies.md +++ b/solutions/observability/infra-and-hosts/detect-metric-anomalies.md @@ -28,7 +28,7 @@ After creating {{ml}} jobs, you cannot change the settings. You can recreate the ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack 1. To open **Infrastructure inventory**, find **Infrastructure** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). diff --git a/solutions/observability/infra-and-hosts/get-started-with-system-metrics.md b/solutions/observability/infra-and-hosts/get-started-with-system-metrics.md index 94ee7b521e..fb893674bb 100644 --- a/solutions/observability/infra-and-hosts/get-started-with-system-metrics.md +++ b/solutions/observability/infra-and-hosts/get-started-with-system-metrics.md @@ -16,7 +16,7 @@ In this guide you’ll learn how to onboard system metrics data from a machine o ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack To follow the steps in this guide, you need an {{stack}} deployment that includes: @@ -47,7 +47,7 @@ The **Admin** role or higher is required to onboard system metrics data. To lear ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack {{agent}} is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more. A single agent makes it easier and faster to deploy monitoring across your infrastructure. Each agent has a single policy you can update to add integrations for new data sources, security protections, and more. diff --git a/solutions/observability/logs/stream-any-log-file.md b/solutions/observability/logs/stream-any-log-file.md index 07a0eb465a..d9fd2740ba 100644 --- a/solutions/observability/logs/stream-any-log-file.md +++ b/solutions/observability/logs/stream-any-log-file.md @@ -18,7 +18,7 @@ Continue with this guide for instructions on manual configuration. ::::{tab-set} :group: stack-serverless -:::{tab-item} Elastic Stack v9 +:::{tab-item} Elastic Stack :sync: stack To follow the steps in this guide, you need an {{stack}} deployment that includes: diff --git a/solutions/observability/unknown-bucket.md b/solutions/observability/unknown-bucket.md deleted file mode 100644 index 0756de5b3a..0000000000 --- a/solutions/observability/unknown-bucket.md +++ /dev/null @@ -1,5 +0,0 @@ -# Unknown Bucket - -% What needs to be done: Placeholder - -% Scope notes: Content in this bucket needs to have a home found for it \ No newline at end of file diff --git a/solutions/observability/unknown-bucket/aws-metrics.md b/solutions/observability/unknown-bucket/aws-metrics.md deleted file mode 100644 index c4cec06584..0000000000 --- a/solutions/observability/unknown-bucket/aws-metrics.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/observability/current/aws-metrics.html ---- - -# AWS metrics [aws-metrics] - -::::{important} -Additional AWS charges for GetMetricData API requests are generated using this module. - -:::: - - - -## Monitor EC2 instances [monitor-ec2-instances] - -To help you analyze the EC2 instance metrics listed on the **Infrastructure inventory** page, you can select view filters based on the following predefined metrics or you can add [custom metrics](../infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics). - -| | | -| --- | --- | -| **CPU Usage** | Average of `aws.ec2.cpu.total.pct`. | -| **Inbound Traffic** | Average of `aws.ec2.network.in.bytes_per_sec`. | -| **Outbound Traffic** | Average of `aws.ec2.network.out.bytes_per_sec`. | -| **Disk Reads (Bytes)** | Average of `aws.ec2.diskio.read.bytes_per_sec`. | -| **Disk Writes (Bytes)** | Average of `aws.ec2.diskio.write.bytes_per_sec`. | - - -## Monitor S3 buckets [monitor-s3-buckets] - -To help you analyze the S3 bucket metrics listed on the **Infrastructure inventory** page, you can select view filters based on the following predefined metrics or you can add [custom metrics](../infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics). - -| | | -| --- | --- | -| **Bucket Size** | Average of `aws.s3_daily_storage.bucket.size.bytes`. | -| **Total Requests** | Average of `aws.s3_request.requests.total`. | -| **Number of Objects** | Average of `aws.s3_daily_storage.number_of_objects`. | -| **Downloads (Bytes)** | Average of `aws.s3_request.downloaded.bytes`. | -| **Uploads (Bytes)** | Average of `aws.s3_request.uploaded.bytes`. | - - -## Monitor SQS queues [monitor-sqs-queues] - -To help you analyze the SQS queue metrics listed on the **Infrastructure inventory** page, you can select view filters based on the following predefined metrics or you can add [custom metrics](../infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics). - -| | | -| --- | --- | -| **Messages Available** | Max of `aws.sqs.messages.visible`. | -| **Messages Delayed** | Max of `aws.sqs.messages.delayed`. | -| **Messages Added** | Max of `aws.sqs.messages.sent`. | -| **Messages Returned Empty** | Max of `aws.sqs.messages.not_visible`. | -| **Oldest Message** | Max of `aws.sqs.oldest_message_age.sec`. | - - -## Monitor RDS databases [monitor-rds-databases] - -To help you analyze the RDS database metrics listed on the **Infrastructure inventory** page, you can select view filters based on the following predefined metrics or you can add [custom metrics](../infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics). - -| | | -| --- | --- | -| **CPU Usage** | Average of `aws.rds.cpu.total.pct`. | -| **Connections** | Average of `aws.rds.database_connections`. | -| **Queries Executed** | Average of `aws.rds.queries`. | -| **Active Transactions** | Average of `aws.rds.transactions.active`. | -| **Latency** | Average of `aws.rds.latency.dml`. | - -For information about which required fields the {{infrastructure-app}} uses to display AWS services metrics, see the [{{infrastructure-app}} field reference](asciidocalypse://docs/docs-content/docs/reference/observability/fields-and-object-schemas/metrics-app-fields.md). - diff --git a/solutions/observability/unknown-bucket/container-metrics.md b/solutions/observability/unknown-bucket/container-metrics.md deleted file mode 100644 index 4002c74d3f..0000000000 --- a/solutions/observability/unknown-bucket/container-metrics.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/observability/current/container-metrics.html ---- - -# Container metrics [container-metrics] - -Learn about key container metrics displayed in the {{infrastructure-app}}: - -* [Docker](#key-metrics-docker) -* [Kubernetes](#key-metrics-kubernetes) - - -## Docker container metrics [key-metrics-docker] - -These are the key metrics displayed for Docker containers. - - -### CPU usage metrics [key-metrics-docker-cpu] - -| Metric | Description | -| --- | --- | -| **CPU Usage (%)** | Average CPU for the container.
**Field Calculation:** `average(docker.cpu.total.pct)` | - - -### Memory metrics [key-metrics-docker-memory] - -| Metric | Description | -| --- | --- | -| **Memory Usage (%)** | Average memory usage for the container.
**Field Calculation:** `average(docker.memory.usage.pct)` | - - -### Network metrics [key-metrics-docker-network] - -| Metric | Description | -| --- | --- | -| **Inbound Traffic (RX)** | Derivative of the maximum of `docker.network.in.bytes` scaled to a 1 second rate.
**Field Calculation:** `average(docker.network.inbound.bytes) * 8 / (max(metricset.period, kql='docker.network.inbound.bytes: *') / 1000)` | -| **Outbound Traffic (TX)** | Derivative of the maximum of `docker.network.out.bytes` scaled to a 1 second rate.
**Field Calculation:** `average(docker.network.outbound.bytes) * 8 / (max(metricset.period, kql='docker.network.outbound.bytes: *') / 1000)` | - - -### Disk metrics [key-metrics-disk-network] - -| Metric | Description | -| --- | --- | -| **Disk Read IOPS** | Average count of read operations from the device per second.
**Field Calculation:** `counter_rate(max(docker.diskio.read.ops), kql='docker.diskio.read.ops: *')` | -| **Disk Write IOPS** | Average count of write operations from the device per second.
**Field Calculation:** `counter_rate(max(docker.diskio.write.ops), kql='docker.diskio.write.ops: *')` | - - -## Kubernetes container metrics [key-metrics-kubernetes] - -These are the key metrics displayed for Kubernetes (containerd) containers. - - -### CPU usage metrics [key-metrics-kubernetes-cpu] - -| Metric | Description | -| --- | --- | -| **CPU Usage (%)** | Average CPU for the container.
**Field Calculation:** `average(kubernetes.container.cpu.usage.limit.pct)` | - - -### Memory metrics [key-metrics-kubernetes-memory] - -| Metric | Description | -| --- | --- | -| **Memory Usage (%)** | Average memory usage for the container.
**Field Calculation:** `average(kubernetes.container.memory.usage.limit.pct)` | diff --git a/solutions/observability/unknown-bucket/host-metrics.md b/solutions/observability/unknown-bucket/host-metrics.md deleted file mode 100644 index 301ea9b1a1..0000000000 --- a/solutions/observability/unknown-bucket/host-metrics.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/observability/current/host-metrics.html ---- - -# Host metrics [host-metrics] - -Learn about key host metrics displayed in the {{infrastructure-app}}: - -* [Hosts](#key-metrics-hosts) -* [CPU usage](#key-metrics-cpu) -* [Memory](#key-metrics-memory) -* [Log](#key-metrics-log) -* [Network](#key-metrics-network) -* [Disk](#key-metrics-network) -* [Legacy metrics](#legacy-metrics) - - -## Hosts metrics [key-metrics-hosts] - -| Metric | Description | -| --- | --- | -| **Hosts** | Number of hosts returned by your search criteria.
**Field Calculation:** `count(system.cpu.cores)` | - - -## CPU usage metrics [key-metrics-cpu] - -| Metric | Description | -| --- | --- | -| **CPU Usage (%)** | Average of percentage of CPU time spent in states other than Idle and IOWait, normalized by the number of CPU cores. Includes both time spent on user space and kernel space. 100% means all CPUs of the host are busy.
**Field Calculation**: `average(system.cpu.total.norm.pct)`
For legacy metric calculations, refer to [Legacy metrics](#legacy-metrics). | -| **CPU Usage - iowait (%)** | The percentage of CPU time spent in wait (on disk).
**Field Calculation:** `average(system.cpu.iowait.pct) / max(system.cpu.cores)` | -| **CPU Usage - irq (%)** | The percentage of CPU time spent servicing and handling hardware interrupts.
**Field Calculation:** `average(system.cpu.irq.pct) / max(system.cpu.cores)` | -| **CPU Usage - nice (%)** | The percentage of CPU time spent on low-priority processes.
**Field Calculation:** `average(system.cpu.nice.pct) / max(system.cpu.cores)` | -| **CPU Usage - softirq (%)** | The percentage of CPU time spent servicing and handling software interrupts.
**Field Calculation:** `average(system.cpu.softirq.pct) / max(system.cpu.cores)` | -| **CPU Usage - steal (%)** | The percentage of CPU time spent in involuntary wait by the virtual CPU while the hypervisor was servicing another processor. Available only on Unix.
**Field Calculation:** `average(system.cpu.steal.pct) / max(system.cpu.cores)` | -| **CPU Usage - system (%)** | The percentage of CPU time spent in kernel space.
**Field Calculation:** `average(system.cpu.system.pct) / max(system.cpu.cores)` | -| **CPU Usage - user (%)** | The percentage of CPU time spent in user space. On multi-core systems, you can have percentages that are greater than 100%. For example, if 3 cores are at 60% use, then the system.cpu.user.pct will be 180%.
**Field Calculation:** `average(system.cpu.user.pct) / max(system.cpu.cores)` | -| **Load (1m)** | 1 minute load average.
Load average gives an indication of the number of threads that are runnable (either busy running on CPU, waiting to run, or waiting for a blocking IO operation to complete).
**Field Calculation:** average(system.load.1) | -| **Load (5m)** | 5 minute load average.
Load average gives an indication of the number of threads that are runnable (either busy running on CPU, waiting to run, or waiting for a blocking IO operation to complete).
**Field Calculation:** `average(system.load.5)` | -| **Load (15m)** | 15 minute load average.
Load average gives an indication of the number of threads that are runnable (either busy running on CPU, waiting to run, or waiting for a blocking IO operation to complete).
**Field Calculation:** `average(system.load.15)` | -| **Normalized Load** | 1 minute load average normalized by the number of CPU cores.
Load average gives an indication of the number of threads that are runnable (either busy running on CPU, waiting to run, or waiting for a blocking IO operation to complete).
100% means the 1 minute load average is equal to the number of CPU cores of the host.
Taking the example of a 32 CPU cores host, if the 1 minute load average is 32, the value reported here is 100%. If the 1 minute load average is 48, the value reported here is 150%.
**Field Calculation:** `average(system.load.1) / max(system.load.cores)` | - - -## Memory metrics [key-metrics-memory] - -| Metric | Description | -| --- | --- | -| **Memory Cache** | Memory (page) cache.
**Field Calculation:** `average(system.memory.used.bytes ) - average(system.memory.actual.used.bytes)` | -| **Memory Free** | Total available memory.
**Field Calculation:** `max(system.memory.total) - average(system.memory.actual.used.bytes)` | -| **Memory Free (excluding cache)** | Total available memory excluding the page cache.
**Field Calculation:** `system.memory.free` | -| **Memory Total** | Total memory capacity.
**Field Calculation:** `avg(system.memory.total)` | -| **Memory Usage (%)** | Percentage of main memory usage excluding page cache.
This includes resident memory for all processes plus memory used by the kernel structures and code apart from the page cache.
A high level indicates a situation of memory saturation for the host. For example, 100% means the main memory is entirely filled with memory that can’t be reclaimed, except by swapping out.
**Field Calculation:** `average(system.memory.actual.used.pct)` | -| **Memory Used** | Main memory usage excluding page cache.
**Field Calculation:** `average(system.memory.actual.used.bytes)` | - - -## Log metrics [key-metrics-log] - -| Metric | Description | -| --- | --- | -| **Log Rate** | Derivative of the cumulative sum of the document count scaled to a 1 second rate. This metric relies on the same indices as the logs.
**Field Calculation:** `cumulative_sum(doc_count)` | - - -## Network metrics [key-metrics-network] - -| Metric | Description | -| --- | --- | -| **Network Inbound (RX)** | Number of bytes that have been received per second on the public interfaces of the hosts.
**Field Calculation**: `sum(host.network.ingress.bytes) * 8 / 1000`
For legacy metric calculations, refer to [Legacy metrics](#legacy-metrics). | -| **Network Outbound (TX)** | Number of bytes that have been sent per second on the public interfaces of the hosts.
**Field Calculation**: `sum(host.network.egress.bytes) * 8 / 1000`
For legacy metric calculations, refer to [Legacy metrics](#legacy-metrics). | - - -## Disk metrics [key-metrics-disk] - -| Metric | Description | -| --- | --- | -| **Disk Latency** | Time spent to service disk requests.
**Field Calculation:** `average(system.diskio.read.time + system.diskio.write.time) / (system.diskio.read.count + system.diskio.write.count)` | -| **Disk Read IOPS** | Average count of read operations from the device per second.
**Field Calculation:** `counter_rate(max(system.diskio.read.count), kql='system.diskio.read.count: *')` | -| **Disk Read Throughput** | Average number of bytes read from the device per second.
**Field Calculation:** `counter_rate(max(system.diskio.read.bytes), kql='system.diskio.read.bytes: *')` | -| **Disk Usage - Available (%)** | Percentage of disk space available.
**Field Calculation:** `1-average(system.filesystem.used.pct)` | -| **Disk Usage - Max (%)** | Percentage of disk space used. A high percentage indicates that a partition on a disk is running out of space.
**Field Calculation:** `max(system.filesystem.used.pct)` | -| **Disk Write IOPS** | Average count of write operations from the device per second.
**Field Calculation:** `counter_rate(max(system.diskio.write.count), kql='system.diskio.write.count: *')` | -| **Disk Write Throughput** | Average number of bytes written from the device per second.
**Field Calculation:** `counter_rate(max(system.diskio.write.bytes), kql='system.diskio.write.bytes: *')` | - - -## Legacy metrics [legacy-metrics] - -Over time, we may change the formula used to calculate a specific metric. To avoid affecting your existing rules, instead of changing the actual metric definition, we create a new metric and refer to the old one as "legacy." - -The UI and any new rules you create will use the new metric definition. However, any alerts that use the old definition will refer to the metric as "legacy." - -| Metric | Description | -| --- | --- | -| **CPU Usage (legacy)** | Percentage of CPU time spent in states other than Idle and IOWait, normalized by the number of CPU cores. This includes both time spent on user space and kernel space. 100% means all CPUs of the host are busy.
**Field Calculation:** `(average(system.cpu.user.pct) + average(system.cpu.system.pct)) / max(system.cpu.cores)` | -| **Network Inbound (RX) (legacy)** | Number of bytes that have been received per second on the public interfaces of the hosts.
**Field Calculation:** `average(host.network.ingress.bytes) * 8 / (max(metricset.period, kql='host.network.ingress.bytes: *') / 1000)` | -| **Network Outbound (TX) (legacy)** | Number of bytes that have been sent per second on the public interfaces of the hosts.
**Field Calculation:** `average(host.network.egress.bytes) * 8 / (max(metricset.period, kql='host.network.egress.bytes: *') / 1000)` | diff --git a/solutions/observability/unknown-bucket/kubernetes-pod-metrics.md b/solutions/observability/unknown-bucket/kubernetes-pod-metrics.md deleted file mode 100644 index ad7b1c3581..0000000000 --- a/solutions/observability/unknown-bucket/kubernetes-pod-metrics.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/observability/current/kubernetes-pod-metrics.html ---- - -# Kubernetes pod metrics [kubernetes-pod-metrics] - -Learn about Kubernetes pod metrics displayed in the {{infrastructure-app}}. - -| | | -| --- | --- | -| **CPU Usage** | Average of `kubernetes.pod.cpu.usage.node.pct`. | -| **Memory Usage** | Average of `kubernetes.pod.memory.usage.node.pct`. | -| **Inbound Traffic** | Derivative of the maximum of `kubernetes.pod.network.rx.bytes` scaled to a 1 second rate. | -| **Outbound Traffic** | Derivative of the maximum of `kubernetes.pod.network.tx.bytes` scaled to a 1 second rate. | - -For information about which required fields the {{infrastructure-app}} uses to display Kubernetes pods metrics, see the [{{infrastructure-app}} field reference](asciidocalypse://docs/docs-content/docs/reference/observability/fields-and-object-schemas/metrics-app-fields.md). - diff --git a/solutions/toc.yml b/solutions/toc.yml index 431ef91c81..b377e5dba8 100644 --- a/solutions/toc.yml +++ b/solutions/toc.yml @@ -177,7 +177,6 @@ toc: - file: observability/apps/elastic-apm-agent-configuration-api.md - file: observability/apps/opentelemetry-intake-api.md - file: observability/apps/jaeger-event-intake.md - - file: observability/apps/common-problems.md - file: observability/apps/what-happens-when-apm-server-elasticsearch-is-down.md - file: observability/apps/apm-server-response-codes.md - file: observability/apps/processing-performance.md @@ -369,12 +368,6 @@ toc: - file: observability/data-set-quality-monitoring.md - file: observability/observability-ai-assistant.md - file: observability/tools-and-apis.md - - file: observability/unknown-bucket.md - children: - - file: observability/unknown-bucket/host-metrics.md - - file: observability/unknown-bucket/container-metrics.md - - file: observability/unknown-bucket/kubernetes-pod-metrics.md - - file: observability/unknown-bucket/aws-metrics.md - file: security.md children: - file: security/elastic-security-serverless.md diff --git a/troubleshoot/observability/apm.md b/troubleshoot/observability/apm.md index 16c30ed3f6..c14e40a5ca 100644 --- a/troubleshoot/observability/apm.md +++ b/troubleshoot/observability/apm.md @@ -11,7 +11,7 @@ mapped_pages: This section provides solutions to common questions and problems, and processing and performance guidance. -* [Common problems](../../solutions/observability/apps/common-problems.md) +* [Common problems](/troubleshoot/observability/apm/common-problems.md) * [What happens when APM Server or {{es}} is down?](../../solutions/observability/apps/what-happens-when-apm-server-elasticsearch-is-down.md) * [APM Server response codes](../../solutions/observability/apps/apm-server-response-codes.md) * [Processing and performance](../../solutions/observability/apps/processing-performance.md)