diff --git a/manage-data/ingest/tools.md b/manage-data/ingest/tools.md index e7110685d4..69a3d8118c 100644 --- a/manage-data/ingest/tools.md +++ b/manage-data/ingest/tools.md @@ -41,7 +41,7 @@ Depending on the type of data you want to ingest, you have a number of methods a | Tools | Usage | Links to more information | | ------- | --------------- | ------------------------- | -| Integrations | Ingest data using a variety of Elastic integrations. | [Elastic Integrations](asciidocalypse://docs/integration-docs/docs/reference/index.md) | +| Integrations | Ingest data using a variety of Elastic integrations. | [Elastic Integrations](integration-docs://reference/index.md) | | File upload | Upload data from a file and inspect it before importing it into {{es}}. | [Upload data files](/manage-data/ingest/upload-data-files.md) | | APIs | Ingest data through code by using the APIs of one of the language clients or the {{es}} HTTP APIs. | [Document APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-document) | | OpenTelemetry | Collect and send your telemetry data to Elastic Observability | [Elastic Distributions of OpenTelemetry](https://github.com/elastic/opentelemetry?tab=readme-ov-file#elastic-distributions-of-opentelemetry) | diff --git a/raw-migrated-files/docs-content/serverless/project-setting-data.md b/raw-migrated-files/docs-content/serverless/project-setting-data.md index 8345640f0b..ec07cabde9 100644 --- a/raw-migrated-files/docs-content/serverless/project-setting-data.md +++ b/raw-migrated-files/docs-content/serverless/project-setting-data.md @@ -11,7 +11,7 @@ Access to individual features is governed by Elastic user roles. Consult your ad | Feature | Description | Available in | | --- | --- | --- | -| [Integrations](asciidocalypse://docs/integration-docs/docs/reference/index.md) | Connect your data to your project. | [![Observability](../../../images/serverless-obs-badge.svg "")](../../../solutions/observability.md)[![Security](../../../images/serverless-sec-badge.svg "")](../../../solutions/security/elastic-security-serverless.md) | +| [Integrations](integration-docs://reference/index.md) | Connect your data to your project. | [![Observability](../../../images/serverless-obs-badge.svg "")](../../../solutions/observability.md)[![Security](../../../images/serverless-sec-badge.svg "")](../../../solutions/security/elastic-security-serverless.md) | | [Fleet and Elastic Agent](/reference/ingestion-tools/fleet/index.md) | Add monitoring for logs, metrics, and other types of data to a host. | [![Observability](../../../images/serverless-obs-badge.svg "")](../../../solutions/observability.md)[![Security](../../../images/serverless-sec-badge.svg "")](../../../solutions/security/elastic-security-serverless.md) | | [{{data-sources-cap}}](../../../explore-analyze/find-and-organize/data-views.md) | Manage the fields in the data views that retrieve your data from {{es-serverless}}. | [![Elasticsearch](../../../images/serverless-es-badge.svg "")](../../../solutions/search.md)[![Observability](../../../images/serverless-obs-badge.svg "")](../../../solutions/observability.md)[![Security](../../../images/serverless-sec-badge.svg "")](../../../solutions/security/elastic-security-serverless.md) | | [Index management](../../../manage-data/data-store/index-basics.md) | View index settings, mappings, and statistics and perform operations on indices. | [![Elasticsearch](../../../images/serverless-es-badge.svg "")](../../../solutions/search.md)[![Observability](../../../images/serverless-obs-badge.svg "")](../../../solutions/observability.md)[![Security](../../../images/serverless-sec-badge.svg "")](../../../solutions/security/elastic-security-serverless.md) | diff --git a/reference/ingestion-tools/fleet/config-file-example-apache.md b/reference/ingestion-tools/fleet/config-file-example-apache.md index 2b790d9223..0056d080b0 100644 --- a/reference/ingestion-tools/fleet/config-file-example-apache.md +++ b/reference/ingestion-tools/fleet/config-file-example-apache.md @@ -76,7 +76,7 @@ inputs: <7> 9. For available input types, refer to [{{agent}} inputs](/reference/ingestion-tools/fleet/elastic-agent-inputs-list.md). 10. Learn about [Data streams](/reference/ingestion-tools/fleet/data-streams.md) for time series data. 11. Specify a unique ID for each individual input stream. Naming the ID by appending the associated `data_stream` dataset (for example `{{user-defined-unique-id}}-apache.access` or `{{user-defined-unique-id}}-apache.error`) is a recommended practice, but any unique ID will work. -12. Refer to [Logs](asciidocalypse://docs/integration-docs/docs/reference/apache.md#apache-logs) in the Apache HTTP Server integration documentation for the logs available to ingest and exported fields. +12. Refer to [Logs](integration-docs://reference/apache/index.md) in the Apache HTTP Server integration documentation for the logs available to ingest and exported fields. 13. Path to the log files to be monitored. @@ -128,7 +128,7 @@ inputs: <7> 9. Learn about [Data streams](/reference/ingestion-tools/fleet/data-streams.md) for time series data. 10. Specify a unique ID for each individual input stream. Naming the ID by appending the associated `data_stream` dataset (for example `{{user-defined-unique-id}}-apache.status`) is a recommended practice, but any unique ID will work. 11. A user-defined dataset. You can specify anything that makes sense to signify the source of the data. -12. Refer to [Metrics](asciidocalypse://docs/integration-docs/docs/reference/apache.md#apache-metrics) in the Apache HTTP Server integration documentation for the type of metrics collected and exported fields. +12. Refer to [Metrics](integration-docs://reference/apache/index.md) in the Apache HTTP Server integration documentation for the type of metrics collected and exported fields. diff --git a/reference/ingestion-tools/fleet/config-file-example-nginx.md b/reference/ingestion-tools/fleet/config-file-example-nginx.md index 31aceba62c..43d3011b7a 100644 --- a/reference/ingestion-tools/fleet/config-file-example-nginx.md +++ b/reference/ingestion-tools/fleet/config-file-example-nginx.md @@ -81,7 +81,7 @@ inputs: <7> 9. For available input types, refer to [{{agent}} inputs](/reference/ingestion-tools/fleet/elastic-agent-inputs-list.md). 10. Learn about [Data streams](/reference/ingestion-tools/fleet/data-streams.md) for time series data. 11. Specify a unique ID for each individual input stream. Naming the ID by appending the associated `data_stream` dataset (for example `{{user-defined-unique-id}}-nginx.access` or `{{user-defined-unique-id}}-nginx.error`) is a recommended practice, but any unique ID will work. -12. Refer to [Logs reference](asciidocalypse://docs/integration-docs/docs/reference/nginx.md#nginx-logs-reference) in the Nginx HTTP integration documentation for the logs available to ingest and exported fields. +12. Refer to [Logs reference](integration-docs://reference/nginx/index.md#logs-reference) in the Nginx HTTP integration documentation for the logs available to ingest and exported fields. 13. Path to the log files to be monitored. @@ -135,7 +135,7 @@ inputs: <7> 10. Learn about [Data streams](/reference/ingestion-tools/fleet/data-streams.md) for time series data. 11. Specify a unique ID for each individual input stream. Naming the ID by appending the associated `data_stream` dataset (for example `{{user-defined-unique-id}}-nginx.stubstatus`) is a recommended practice, but any unique ID will work. 12. A user-defined dataset. You can specify anything that makes sense to signify the source of the data. -13. Refer to [Metrics reference](asciidocalypse://docs/integration-docs/docs/reference/nginx.md#nginx-metrics-reference) in the Nginx integration documentation for the type of metrics collected and exported fields. +13. Refer to [Metrics reference](integration-docs://reference/nginx/index.md#metrics-reference) in the Nginx integration documentation for the type of metrics collected and exported fields. diff --git a/reference/ingestion-tools/fleet/configuring-kubernetes-metadata.md b/reference/ingestion-tools/fleet/configuring-kubernetes-metadata.md index 23773d6b3a..de759bafe7 100644 --- a/reference/ingestion-tools/fleet/configuring-kubernetes-metadata.md +++ b/reference/ingestion-tools/fleet/configuring-kubernetes-metadata.md @@ -7,7 +7,7 @@ mapped_pages: Kubernetes [metadata](/solutions/observability/infra-and-hosts/tutorial-observe-kubernetes-deployments.md#beats-metadata) refer to contextual information extracted from Kubernetes resources. Metadata information enrich metrics and logs collected from a Kubernetes cluster, enabling deeper insights into Kubernetes environments. -When the {{agent}}'s policy includes the [{{k8s}} Integration](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md) which configures the collection of Kubernetes related metrics and container logs, the mechanisms used for the metadata enrichment are: +When the {{agent}}'s policy includes the [{{k8s}} Integration](integration-docs://reference/kubernetes.md) which configures the collection of Kubernetes related metrics and container logs, the mechanisms used for the metadata enrichment are: * [Kubernetes Provider](/reference/ingestion-tools/fleet/kubernetes-provider.md) for log collection * Kubernetes metadata enrichers for metrics diff --git a/reference/ingestion-tools/fleet/dissect-processor.md b/reference/ingestion-tools/fleet/dissect-processor.md index 82450dfad2..98a060a7b0 100644 --- a/reference/ingestion-tools/fleet/dissect-processor.md +++ b/reference/ingestion-tools/fleet/dissect-processor.md @@ -83,5 +83,5 @@ This configuration produces fields like: `service.name` is an ECS [keyword field](elasticsearch://reference/elasticsearch/mapping-reference/keyword.md), which means that you can use it in {{es}} for filtering, sorting, and aggregations. -When possible, use ECS-compatible field names. For more information, see the [Elastic Common Schema](asciidocalypse://docs/integration-docs/docs/reference/index.md) documentation. +When possible, use ECS-compatible field names. For more information, see the [Elastic Common Schema](integration-docs://reference/index.md) documentation. diff --git a/reference/ingestion-tools/fleet/elastic-agent-unprivileged.md b/reference/ingestion-tools/fleet/elastic-agent-unprivileged.md index 8f50804f8f..3a467ef97a 100644 --- a/reference/ingestion-tools/fleet/elastic-agent-unprivileged.md +++ b/reference/ingestion-tools/fleet/elastic-agent-unprivileged.md @@ -100,12 +100,12 @@ As well, a warning is displayed in {{kib}} if you try to add an integration that Examples of integrations that require {{agent}} to have administrative privileges are: -* [{{elastic-defend}}](asciidocalypse://docs/integration-docs/docs/reference/endpoint.md) -* [Auditd Manager](asciidocalypse://docs/integration-docs/docs/reference/auditd_manager.md) -* [File Integrity Monitoring](asciidocalypse://docs/integration-docs/docs/reference/fim.md) -* [Network Packet Capture](asciidocalypse://docs/integration-docs/docs/reference/network_traffic.md) -* [System Audit](asciidocalypse://docs/integration-docs/docs/reference/system_audit.md) -* [Universal Profiling Agent](asciidocalypse://docs/integration-docs/docs/reference/profiler_agent.md) +* [{{elastic-defend}}](integration-docs://reference/endpoint/index.md) +* [Auditd Manager](integration-docs://reference/auditd_manager/index.md) +* [File Integrity Monitoring](integration-docs://reference/fim/index.md) +* [Network Packet Capture](integration-docs://reference/network_traffic/index.md) +* [System Audit](integration-docs://reference/system_audit/index.md) +* [Universal Profiling Agent](integration-docs://reference/profiler_agent/index.md) ## Viewing an {{agent}} privilege mode [unprivileged-view-mode] diff --git a/reference/ingestion-tools/fleet/example-standalone-monitor-nginx-serverless.md b/reference/ingestion-tools/fleet/example-standalone-monitor-nginx-serverless.md index ef2e3ac904..ddab9ab2c7 100644 --- a/reference/ingestion-tools/fleet/example-standalone-monitor-nginx-serverless.md +++ b/reference/ingestion-tools/fleet/example-standalone-monitor-nginx-serverless.md @@ -311,4 +311,4 @@ Congratulations! You have successfully set up monitoring for nginx using standal ## What’s next? [_whats_next] * Learn more about [{{fleet}} and {{agent}}](/reference/ingestion-tools/fleet/index.md). -* Learn more about [{{integrations}}](asciidocalypse://docs/integration-docs/docs/reference/index.md). +* Learn more about [{{integrations}}](integration-docs://reference/index.md). diff --git a/reference/ingestion-tools/fleet/example-standalone-monitor-nginx.md b/reference/ingestion-tools/fleet/example-standalone-monitor-nginx.md index c1e3cfc2ef..6722e91959 100644 --- a/reference/ingestion-tools/fleet/example-standalone-monitor-nginx.md +++ b/reference/ingestion-tools/fleet/example-standalone-monitor-nginx.md @@ -310,4 +310,4 @@ Congratulations! You have successfully set up monitoring for nginx using standal ## What’s next? [_whats_next_2] * Learn more about [{{fleet}} and {{agent}}](/reference/ingestion-tools/fleet/index.md). -* Learn more about [{{integrations}}](asciidocalypse://docs/integration-docs/docs/reference/index.md). +* Learn more about [{{integrations}}](integration-docs://reference/index.md). diff --git a/reference/ingestion-tools/fleet/hints-annotations-autodiscovery.md b/reference/ingestion-tools/fleet/hints-annotations-autodiscovery.md index 2e3dd1999e..f4d2a6d7cf 100644 --- a/reference/ingestion-tools/fleet/hints-annotations-autodiscovery.md +++ b/reference/ingestion-tools/fleet/hints-annotations-autodiscovery.md @@ -41,7 +41,7 @@ The host to use for metrics retrieval. If not defined, the host will be set as t ### `co.elastic.hints/data_stream` [_co_elastic_hintsdata_stream] -The list of data streams to enable. If not specified, the integration’s default data streams are used. To find the defaults, refer to the [Elastic integrations documentation](asciidocalypse://docs/integration-docs/docs/reference/index.md). +The list of data streams to enable. If not specified, the integration’s default data streams are used. To find the defaults, refer to the [Elastic integrations documentation](integration-docs://reference/index.md). If data streams are specified, additional hints can be defined per data stream. For example, `co.elastic.hints/info.period: 5m` if the data stream specified is `info` for the [Redis module](beats://reference/metricbeat/metricbeat-module-redis.md). diff --git a/reference/ingestion-tools/fleet/index.md b/reference/ingestion-tools/fleet/index.md index a36e976d5d..e1b750f136 100644 --- a/reference/ingestion-tools/fleet/index.md +++ b/reference/ingestion-tools/fleet/index.md @@ -34,7 +34,7 @@ Looking for a general guide that explores all of your options for ingesting data ## {{integrations}} -[{{integrations}}](asciidocalypse://docs/integration-docs/docs/reference/index.md) provide an easy way to connect Elastic to external services and systems, and quickly get insights or take action. They can collect new sources of data, and they often ship with out-of-the-box assets like dashboards, visualizations, and pipelines to extract structured fields out of logs and events. This makes it easier to get insights within seconds. Integrations are available for popular services and platforms like Nginx or AWS, as well as many generic input types like log files. +[{{integrations}}](integration-docs://reference/index.md) provide an easy way to connect Elastic to external services and systems, and quickly get insights or take action. They can collect new sources of data, and they often ship with out-of-the-box assets like dashboards, visualizations, and pipelines to extract structured fields out of logs and events. This makes it easier to get insights within seconds. Integrations are available for popular services and platforms like Nginx or AWS, as well as many generic input types like log files. {{kib}} provides a web-based UI to add and manage integrations. You can browse a unified view of available integrations that shows both {{agent}} and {{beats}} integrations. diff --git a/reference/ingestion-tools/fleet/ingest-pipeline-kubernetes.md b/reference/ingestion-tools/fleet/ingest-pipeline-kubernetes.md index a2acbb02fa..692aac13b0 100644 --- a/reference/ingestion-tools/fleet/ingest-pipeline-kubernetes.md +++ b/reference/ingestion-tools/fleet/ingest-pipeline-kubernetes.md @@ -11,7 +11,7 @@ Custom pipelines can be used to add custom data processing, like adding fields, ## Metadata enrichment for Kubernetes [_metadata_enrichment_for_kubernetes] -The [{{k8s}} Integration](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md) is used to collect logs and metrics from Kubernetes clusters with {{agent}}. During the collection, the integration enhances the collected information with extra useful information that users can correlate with different Kubernetes assets. This additional information added on top of collected data, such as labels, annotations, ancestor names of Kubernetes assets, and others, are called metadata. +The [{{k8s}} Integration](integration-docs://reference/kubernetes.md) is used to collect logs and metrics from Kubernetes clusters with {{agent}}. During the collection, the integration enhances the collected information with extra useful information that users can correlate with different Kubernetes assets. This additional information added on top of collected data, such as labels, annotations, ancestor names of Kubernetes assets, and others, are called metadata. The [{{k8s}} Provider](/reference/ingestion-tools/fleet/kubernetes-provider.md) offers the `add_resource_metadata` option to configure the metadata enrichment options. diff --git a/reference/ingestion-tools/fleet/install-elastic-agents.md b/reference/ingestion-tools/fleet/install-elastic-agents.md index 12842cd155..50ca62e327 100644 --- a/reference/ingestion-tools/fleet/install-elastic-agents.md +++ b/reference/ingestion-tools/fleet/install-elastic-agents.md @@ -77,7 +77,7 @@ Using our lab environment as an example, we can observe the following resource c ### CPU and RSS memory size [_cpu_and_rss_memory_size] -We tested using an AWS `m7i.large` instance type with 2 vCPUs, 8.0 GB of memory, and up to 12.5 Gbps of bandwidth. The tests ingested a single log file using both the [throughput and scale preset](/reference/ingestion-tools/fleet/elasticsearch-output.md#output-elasticsearch-performance-tuning-settings) with self monitoring enabled. These tests are representative of use cases that attempt to ingest data as fast as possible. This does not represent the resource overhead when using [{{elastic-defend}}](asciidocalypse://docs/integration-docs/docs/reference/endpoint.md). +We tested using an AWS `m7i.large` instance type with 2 vCPUs, 8.0 GB of memory, and up to 12.5 Gbps of bandwidth. The tests ingested a single log file using both the [throughput and scale preset](/reference/ingestion-tools/fleet/elasticsearch-output.md#output-elasticsearch-performance-tuning-settings) with self monitoring enabled. These tests are representative of use cases that attempt to ingest data as fast as possible. This does not represent the resource overhead when using [{{elastic-defend}}](integration-docs://reference/endpoint/index.md). | | | | | --- | --- | --- | diff --git a/reference/ingestion-tools/fleet/integrations-assets-best-practices.md b/reference/ingestion-tools/fleet/integrations-assets-best-practices.md index 04da3c0a3d..c91c10c27b 100644 --- a/reference/ingestion-tools/fleet/integrations-assets-best-practices.md +++ b/reference/ingestion-tools/fleet/integrations-assets-best-practices.md @@ -36,7 +36,7 @@ The {{fleet}} integration assets are not supposed to work when sending arbitrary While it’s possible to include {{fleet}} and {{agent}} integration assets in a custom integration, this is not recommended nor supported. Assets from another integration should not be referenced directly from a custom integration. -As an example scenario, one may want to ingest Redis logs from Kafka. This can be done using the [Redis integration](asciidocalypse://docs/integration-docs/docs/reference/redis-intro.md), but only certain files and paths are allowed. It’s technically possible to use the [Custom Kafka Logs integration](asciidocalypse://docs/integration-docs/docs/reference/kafka_log.md) with a custom ingest pipeline, referencing the ingest pipeline of the Redis integration to ingest logs into the index templates of the Custom Kafka Logs integration data streams. +As an example scenario, one may want to ingest Redis logs from Kafka. This can be done using the [Redis integration](integration-docs://reference/redis-intro.md), but only certain files and paths are allowed. It’s technically possible to use the [Custom Kafka Logs integration](integration-docs://reference/kafka_log.md) with a custom ingest pipeline, referencing the ingest pipeline of the Redis integration to ingest logs into the index templates of the Custom Kafka Logs integration data streams. However, referencing assets of an integration from another custom integration is not recommended nor supported. A configuration as described above can break when the integration is upgraded, as can happen automatically. diff --git a/reference/ingestion-tools/fleet/manage-integrations.md b/reference/ingestion-tools/fleet/manage-integrations.md index 065b367c2f..f077822b62 100644 --- a/reference/ingestion-tools/fleet/manage-integrations.md +++ b/reference/ingestion-tools/fleet/manage-integrations.md @@ -8,7 +8,7 @@ mapped_pages: ::::{admonition} -Integrations are available for a wide array of popular services and platforms. To see the full list of available integrations, go to the **Integrations** page in {{kib}}, or visit [Elastic Integrations](asciidocalypse://docs/integration-docs/docs/reference/index.md). +Integrations are available for a wide array of popular services and platforms. To see the full list of available integrations, go to the **Integrations** page in {{kib}}, or visit [Elastic Integrations](integration-docs://reference/index.md). {{agent}} integrations provide a simple, unified way to collect data from popular apps and services, and protect systems from security threats. diff --git a/reference/ingestion-tools/fleet/migrate-auditbeat-to-agent.md b/reference/ingestion-tools/fleet/migrate-auditbeat-to-agent.md index 72033c4ad5..74f5d99622 100644 --- a/reference/ingestion-tools/fleet/migrate-auditbeat-to-agent.md +++ b/reference/ingestion-tools/fleet/migrate-auditbeat-to-agent.md @@ -21,20 +21,20 @@ The following table describes the integrations you can use instead of {{auditbea | If you use…​ | You can use this instead…​ | Notes | | --- | --- | --- | -| [Auditd](beats://reference/auditbeat/auditbeat-module-auditd.md) module | [Auditd Manager](asciidocalypse://docs/integration-docs/docs/reference/auditd_manager.md) integration | This integration is a direct replacement of the module. You can port rules andconfiguration to this integration. Starting in {{stack}} 8.4, you can also set the`immutable` flag in the audit configuration. | -| [Auditd Logs](asciidocalypse://docs/integration-docs/docs/reference/auditd.md) integration | Use this integration if you don’t need to manage rules. It only parses logs fromthe audit daemon `auditd`. Please note that the events created by this integrationare different than the ones created by[Auditd Manager](asciidocalypse://docs/integration-docs/docs/reference/auditd_manager.md), since the latter merges allrelated messages in a single event while [Auditd Logs](asciidocalypse://docs/integration-docs/docs/reference/auditd.md)creates one event per message. | -| [File Integrity](beats://reference/auditbeat/auditbeat-module-file_integrity.md) module | [File Integrity Monitoring](asciidocalypse://docs/integration-docs/docs/reference/fim.md) integration | This integration is a direct replacement of the module. It reports real-timeevents, but cannot report who made the changes. If you need to track thisinformation, use [{{elastic-defend}}](/solutions/security/configure-elastic-defend/install-elastic-defend.md) instead. | +| [Auditd](beats://reference/auditbeat/auditbeat-module-auditd.md) module | [Auditd Manager](integration-docs://reference/auditd_manager/index.md) integration | This integration is a direct replacement of the module. You can port rules andconfiguration to this integration. Starting in {{stack}} 8.4, you can also set the`immutable` flag in the audit configuration. | +| [Auditd Logs](integration-docs://reference/auditd.md) integration | Use this integration if you don’t need to manage rules. It only parses logs fromthe audit daemon `auditd`. Please note that the events created by this integrationare different than the ones created by[Auditd Manager](integration-docs://reference/auditd_manager/index.md), since the latter merges allrelated messages in a single event while [Auditd Logs](integration-docs://reference/auditd.md)creates one event per message. | +| [File Integrity](beats://reference/auditbeat/auditbeat-module-file_integrity.md) module | [File Integrity Monitoring](integration-docs://reference/fim/index.md) integration | This integration is a direct replacement of the module. It reports real-timeevents, but cannot report who made the changes. If you need to track thisinformation, use [{{elastic-defend}}](/solutions/security/configure-elastic-defend/install-elastic-defend.md) instead. | | [System](beats://reference/auditbeat/auditbeat-module-system.md) module | It depends…​ | There is not a single integration that collects all this information. | -| [System.host](beats://reference/auditbeat/auditbeat-dataset-system-host.md) dataset | [Osquery](asciidocalypse://docs/integration-docs/docs/reference/osquery.md) or [Osquery Manager](asciidocalypse://docs/integration-docs/docs/reference/osquery_manager.md) integration | Schedule collection of information like:

* [system_info](https://www.osquery.io/schema/5.1.0/#system_info) for hostname, unique ID, and architecture
* [os_version](https://www.osquery.io/schema/5.1.0/#os_version)
* [interface_addresses](https://www.osquery.io/schema/5.1.0/#interface_addresses) for IPs and MACs
| +| [System.host](beats://reference/auditbeat/auditbeat-dataset-system-host.md) dataset | [Osquery](integration-docs://reference/osquery.md) or [Osquery Manager](integration-docs://reference/osquery_manager/index.md) integration | Schedule collection of information like:

* [system_info](https://www.osquery.io/schema/5.1.0/#system_info) for hostname, unique ID, and architecture
* [os_version](https://www.osquery.io/schema/5.1.0/#os_version)
* [interface_addresses](https://www.osquery.io/schema/5.1.0/#interface_addresses) for IPs and MACs
| | [System.login](beats://reference/auditbeat/auditbeat-dataset-system-login.md) dataset | [Endpoint](/solutions/security/configure-elastic-defend/install-elastic-defend.md) | Report login events. | -| [Osquery](asciidocalypse://docs/integration-docs/docs/reference/osquery.md) or [Osquery Manager](asciidocalypse://docs/integration-docs/docs/reference/osquery_manager.md) integration | Use the [last](https://www.osquery.io/schema/5.1.0/#last) table for Linux and macOS. | -| {{fleet}} [system](asciidocalypse://docs/integration-docs/docs/reference/system.md) integration | Collect login events for Windows through the [Security event log](asciidocalypse://docs/integration-docs/docs/reference/system.md#system-security). | -| [System.package](beats://reference/auditbeat/auditbeat-dataset-system-package.md) dataset | [System Audit](asciidocalypse://docs/integration-docs/docs/reference/system_audit.md) integration | This integration is a direct replacement of the System Package dataset. Starting in {{stack}} 8.7, you can port rules and configuration settings to this integration. This integration currently schedules collection of information such as:

* [rpm_packages](https://www.osquery.io/schema/5.1.0/#rpm_packages)
* [deb_packages](https://www.osquery.io/schema/5.1.0/#deb_packages)
* [homebrew_packages](https://www.osquery.io/schema/5.1.0/#homebrew_packages)
| -| [Osquery](asciidocalypse://docs/integration-docs/docs/reference/osquery.md) or [Osquery Manager](asciidocalypse://docs/integration-docs/docs/reference/osquery_manager.md) integration | Schedule collection of information like:

* [rpm_packages](https://www.osquery.io/schema/5.1.0/#rpm_packages)
* [deb_packages](https://www.osquery.io/schema/5.1.0/#deb_packages)
* [homebrew_packages](https://www.osquery.io/schema/5.1.0/#homebrew_packages)
* [apps](https://www.osquery.io/schema/5.1.0/#apps) (MacOS)
* [programs](https://www.osquery.io/schema/5.1.0/#programs) (Windows)
* [npm_packages](https://www.osquery.io/schema/5.1.0/#npm_packages)
* [atom_packages](https://www.osquery.io/schema/5.1.0/#atom_packages)
* [chocolatey_packages](https://www.osquery.io/schema/5.1.0/#chocolatey_packages)
* [portage_packages](https://www.osquery.io/schema/5.1.0/#portage_packages)
* [python_packages](https://www.osquery.io/schema/5.1.0/#python_packages)
| -| [System.process](beats://reference/auditbeat/auditbeat-dataset-system-process.md) dataset | [Endpoint](/solutions/security/configure-elastic-defend/install-elastic-defend.md) | Best replacement because out of the box it reports events forevery process in [ECS](asciidocalypse://docs/integration-docs/docs/reference/index.md) format and has excellentintegration in [Kibana](/get-started/the-stack.md). | -| [Custom Windows event log](asciidocalypse://docs/integration-docs/docs/reference/winlog.md) and [Sysmon](asciidocalypse://docs/integration-docs/docs/reference/sysmon_linux.md) integrations | Provide process data. | -| [Osquery](asciidocalypse://docs/integration-docs/docs/reference/osquery.md) or[Osquery Manager](asciidocalypse://docs/integration-docs/docs/reference/osquery_manager.md) integration | Collect data from the [process](https://www.osquery.io/schema/5.1.0/#process) table on some OSeswithout polling. | +| [Osquery](integration-docs://reference/osquery.md) or [Osquery Manager](integration-docs://reference/osquery_manager/index.md) integration | Use the [last](https://www.osquery.io/schema/5.1.0/#last) table for Linux and macOS. | +| {{fleet}} [system](integration-docs://reference/system.md) integration | Collect login events for Windows through the [Security event log](integration-docs://reference/system/index.md#security). | +| [System.package](beats://reference/auditbeat/auditbeat-dataset-system-package.md) dataset | [System Audit](integration-docs://reference/system_audit/index.md) integration | This integration is a direct replacement of the System Package dataset. Starting in {{stack}} 8.7, you can port rules and configuration settings to this integration. This integration currently schedules collection of information such as:

* [rpm_packages](https://www.osquery.io/schema/5.1.0/#rpm_packages)
* [deb_packages](https://www.osquery.io/schema/5.1.0/#deb_packages)
* [homebrew_packages](https://www.osquery.io/schema/5.1.0/#homebrew_packages)
| +| [Osquery](integration-docs://reference/osquery.md) or [Osquery Manager](integration-docs://reference/osquery_manager/index.md) integration | Schedule collection of information like:

* [rpm_packages](https://www.osquery.io/schema/5.1.0/#rpm_packages)
* [deb_packages](https://www.osquery.io/schema/5.1.0/#deb_packages)
* [homebrew_packages](https://www.osquery.io/schema/5.1.0/#homebrew_packages)
* [apps](https://www.osquery.io/schema/5.1.0/#apps) (MacOS)
* [programs](https://www.osquery.io/schema/5.1.0/#programs) (Windows)
* [npm_packages](https://www.osquery.io/schema/5.1.0/#npm_packages)
* [atom_packages](https://www.osquery.io/schema/5.1.0/#atom_packages)
* [chocolatey_packages](https://www.osquery.io/schema/5.1.0/#chocolatey_packages)
* [portage_packages](https://www.osquery.io/schema/5.1.0/#portage_packages)
* [python_packages](https://www.osquery.io/schema/5.1.0/#python_packages)
| +| [System.process](beats://reference/auditbeat/auditbeat-dataset-system-process.md) dataset | [Endpoint](/solutions/security/configure-elastic-defend/install-elastic-defend.md) | Best replacement because out of the box it reports events forevery process in [ECS](integration-docs://reference/index.md) format and has excellentintegration in [Kibana](/get-started/the-stack.md). | +| [Custom Windows event log](integration-docs://reference/winlog.md) and [Sysmon](integration-docs://reference/sysmon_linux/index.md) integrations | Provide process data. | +| [Osquery](integration-docs://reference/osquery.md) or[Osquery Manager](integration-docs://reference/osquery_manager/index.md) integration | Collect data from the [process](https://www.osquery.io/schema/5.1.0/#process) table on some OSeswithout polling. | | [System.socket](beats://reference/auditbeat/auditbeat-dataset-system-socket.md) dataset | [Endpoint](/solutions/security/configure-elastic-defend/install-elastic-defend.md) | Best replacement because it supports monitoring network connections on Linux,Windows, and MacOS. Includes process and user metadata. Currently does notdo flow accounting (byte and packet counts) or domain name enrichment (but doescollect DNS queries separately). | -| [Osquery](asciidocalypse://docs/integration-docs/docs/reference/osquery.md) or [Osquery Manager](asciidocalypse://docs/integration-docs/docs/reference/osquery_manager.md) integration | Monitor socket events via the [socket_events](https://www.osquery.io/schema/5.1.0/#socket_events) tablefor Linux and MacOS. | -| [System.user](beats://reference/auditbeat/auditbeat-dataset-system-user.md) dataset | [Osquery](asciidocalypse://docs/integration-docs/docs/reference/osquery.md) or [Osquery Manager](asciidocalypse://docs/integration-docs/docs/reference/osquery_manager.md) integration | Monitor local users via the [user](https://www.osquery.io/schema/5.1.0/#user) table for Linux, Windows, and MacOS. | +| [Osquery](integration-docs://reference/osquery.md) or [Osquery Manager](integration-docs://reference/osquery_manager/index.md) integration | Monitor socket events via the [socket_events](https://www.osquery.io/schema/5.1.0/#socket_events) tablefor Linux and MacOS. | +| [System.user](beats://reference/auditbeat/auditbeat-dataset-system-user.md) dataset | [Osquery](integration-docs://reference/osquery.md) or [Osquery Manager](integration-docs://reference/osquery_manager/index.md) integration | Monitor local users via the [user](https://www.osquery.io/schema/5.1.0/#user) table for Linux, Windows, and MacOS. | diff --git a/reference/ingestion-tools/fleet/processor-syntax.md b/reference/ingestion-tools/fleet/processor-syntax.md index 7f596ca123..ef17189c2c 100644 --- a/reference/ingestion-tools/fleet/processor-syntax.md +++ b/reference/ingestion-tools/fleet/processor-syntax.md @@ -55,7 +55,7 @@ Each condition receives a field to compare. You can specify multiple fields unde For each field, you can specify a simple field name or a nested map, for example `dns.question.name`. -Refer to the [integrations documentation](asciidocalypse://docs/integration-docs/docs/reference/index.md) for a list of all fields created by a specific integration. +Refer to the [integrations documentation](integration-docs://reference/index.md) for a list of all fields created by a specific integration. The supported conditions are: diff --git a/reference/ingestion-tools/fleet/running-on-aks-managed-by-fleet.md b/reference/ingestion-tools/fleet/running-on-aks-managed-by-fleet.md index b1405fc658..cc91f4d4e4 100644 --- a/reference/ingestion-tools/fleet/running-on-aks-managed-by-fleet.md +++ b/reference/ingestion-tools/fleet/running-on-aks-managed-by-fleet.md @@ -14,9 +14,9 @@ On managed Kubernetes solutions like AKS, {{agent}} has no access to several dat 1. Metrics from [Kubernetes control plane](https://kubernetes.io/docs/concepts/overview/components/#control-plane-components) components are not available. Consequently metrics are not available for `kube-scheduler` and `kube-controller-manager` components. In this regard, the respective **dashboards** will not be populated with data. 2. **Audit logs** are available only on Kubernetes master nodes as well, hence cannot be collected by {{agent}}. -3. Fields `orchestrator.cluster.name` and `orchestrator.cluster.url` are not populated. `orchestrator.cluster.name` field is used as a cluster selector for default Kubernetes dashboards, shipped with [Kubernetes integration](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md). +3. Fields `orchestrator.cluster.name` and `orchestrator.cluster.url` are not populated. `orchestrator.cluster.name` field is used as a cluster selector for default Kubernetes dashboards, shipped with [Kubernetes integration](integration-docs://reference/kubernetes.md). - In this regard, you can use [`add_fields` processor](beats://reference/filebeat/add-fields.md) to add `orchestrator.cluster.name` and `orchestrator.cluster.url` fields for each [Kubernetes integration](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md)'s component: + In this regard, you can use [`add_fields` processor](beats://reference/filebeat/add-fields.md) to add `orchestrator.cluster.name` and `orchestrator.cluster.url` fields for each [Kubernetes integration](integration-docs://reference/kubernetes.md)'s component: ```yaml - add_fields: diff --git a/reference/ingestion-tools/fleet/running-on-eks-managed-by-fleet.md b/reference/ingestion-tools/fleet/running-on-eks-managed-by-fleet.md index 4db5c6eb57..31e16dfae7 100644 --- a/reference/ingestion-tools/fleet/running-on-eks-managed-by-fleet.md +++ b/reference/ingestion-tools/fleet/running-on-eks-managed-by-fleet.md @@ -14,9 +14,9 @@ On managed Kubernetes solutions like EKS, {{agent}} has no access to several dat 1. Metrics from [Kubernetes control plane](https://kubernetes.io/docs/concepts/overview/components/#control-plane-components) components are not available. Consequently metrics are not available for `kube-scheduler` and `kube-controller-manager` components. In this regard, the respective **dashboards** will not be populated with data. 2. **Audit logs** are available only on Kubernetes master nodes as well, hence cannot be collected by {{agent}}. -3. Fields `orchestrator.cluster.name` and `orchestrator.cluster.url` are not populated. `orchestrator.cluster.name` field is used as a cluster selector for default Kubernetes dashboards, shipped with [Kubernetes integration](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md). +3. Fields `orchestrator.cluster.name` and `orchestrator.cluster.url` are not populated. `orchestrator.cluster.name` field is used as a cluster selector for default Kubernetes dashboards, shipped with [Kubernetes integration](integration-docs://reference/kubernetes.md). - In this regard, you can use [`add_fields` processor](beats://reference/filebeat/add-fields.md) to add `orchestrator.cluster.name` and `orchestrator.cluster.url` fields for each [Kubernetes integration](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md)'s component: + In this regard, you can use [`add_fields` processor](beats://reference/filebeat/add-fields.md) to add `orchestrator.cluster.name` and `orchestrator.cluster.url` fields for each [Kubernetes integration](integration-docs://reference/kubernetes.md)'s component: ```yaml - add_fields: diff --git a/reference/ingestion-tools/fleet/running-on-kubernetes-managed-by-fleet.md b/reference/ingestion-tools/fleet/running-on-kubernetes-managed-by-fleet.md index ecc679f688..37634e844a 100644 --- a/reference/ingestion-tools/fleet/running-on-kubernetes-managed-by-fleet.md +++ b/reference/ingestion-tools/fleet/running-on-kubernetes-managed-by-fleet.md @@ -39,7 +39,7 @@ mapped_pages: ``` ::::{warning} - On managed Kubernetes solutions, such as AKS, GKE or EKS, {{agent}} does not have the required permissions to collect metrics from [Kubernetes control plane](https://kubernetes.io/docs/concepts/overview/components/#control-plane-components) components, like `kube-scheduler` and `kube-controller-manager`. Audit logs are only available on Kubernetes control plane nodes as well, and hence cannot be collected by {{agent}}. Refer [here](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md#kubernetes-scheduler-and-controllermanager) to find more information. For more information about specific cloud providers, refer to [Run {{agent}} on Azure AKS managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-aks-managed-by-fleet.md), [Run {{agent}} on GKE managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-gke-managed-by-fleet.md) and [Run {{agent}} on Amazon EKS managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-eks-managed-by-fleet.md) + On managed Kubernetes solutions, such as AKS, GKE or EKS, {{agent}} does not have the required permissions to collect metrics from [Kubernetes control plane](https://kubernetes.io/docs/concepts/overview/components/#control-plane-components) components, like `kube-scheduler` and `kube-controller-manager`. Audit logs are only available on Kubernetes control plane nodes as well, and hence cannot be collected by {{agent}}. Refer [here](integration-docs://reference/kubernetes/kube-scheduler.md) and [here](integration-docs://reference/kubernetes/kube-controller-manager.md) to find more information. For more information about specific cloud providers, refer to [Run {{agent}} on Azure AKS managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-aks-managed-by-fleet.md), [Run {{agent}} on GKE managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-gke-managed-by-fleet.md) and [Run {{agent}} on Amazon EKS managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-eks-managed-by-fleet.md) :::: @@ -78,7 +78,7 @@ The size and the number of nodes in a Kubernetes cluster can be large at times, ### Step 2: Configure {{agent}} policy [_step_2_configure_agent_policy] -The {{agent}} needs to be assigned to a policy to enable the proper inputs. To achieve Kubernetes observability, the policy needs to include the Kubernetes integration. Refer to [Create a policy](/reference/ingestion-tools/fleet/agent-policy.md#create-a-policy) and [Add an integration to a policy](/reference/ingestion-tools/fleet/agent-policy.md#add-integration) to learn how to configure the [Kubernetes integration](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md). +The {{agent}} needs to be assigned to a policy to enable the proper inputs. To achieve Kubernetes observability, the policy needs to include the Kubernetes integration. Refer to [Create a policy](/reference/ingestion-tools/fleet/agent-policy.md#create-a-policy) and [Add an integration to a policy](/reference/ingestion-tools/fleet/agent-policy.md#add-integration) to learn how to configure the [Kubernetes integration](integration-docs://reference/kubernetes.md). ### Step 3: Enroll {{agent}} to the policy [_step_3_enroll_agent_to_the_policy] diff --git a/reference/ingestion-tools/fleet/running-on-kubernetes-standalone.md b/reference/ingestion-tools/fleet/running-on-kubernetes-standalone.md index fc31230260..c9834fd4b8 100644 --- a/reference/ingestion-tools/fleet/running-on-kubernetes-standalone.md +++ b/reference/ingestion-tools/fleet/running-on-kubernetes-standalone.md @@ -37,7 +37,7 @@ mapped_pages: ``` ::::{warning} - On managed Kubernetes solutions, such as AKS, GKE or EKS, {{agent}} does not have the required permissions to collect metrics from [Kubernetes control plane](https://kubernetes.io/docs/concepts/overview/components/#control-plane-components) components, like `kube-scheduler` and `kube-controller-manager`. Audit logs are only available on Kubernetes control plane nodes as well, and hence cannot be collected by {{agent}}. Refer [here](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md#kubernetes-scheduler-and-controllermanager) to find more information. For more information about specific cloud providers, refer to [Run {{agent}} on Azure AKS managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-aks-managed-by-fleet.md), [Run {{agent}} on GKE managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-gke-managed-by-fleet.md) and [Run {{agent}} on Amazon EKS managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-eks-managed-by-fleet.md) + On managed Kubernetes solutions, such as AKS, GKE or EKS, {{agent}} does not have the required permissions to collect metrics from [Kubernetes control plane](https://kubernetes.io/docs/concepts/overview/components/#control-plane-components) components, like `kube-scheduler` and `kube-controller-manager`. Audit logs are only available on Kubernetes control plane nodes as well, and hence cannot be collected by {{agent}}. Refer [here](integration-docs://reference/kubernetes/kube-scheduler.md) and [here](integration-docs://reference/kubernetes/kube-controller-manager.md) to find more information. For more information about specific cloud providers, refer to [Run {{agent}} on Azure AKS managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-aks-managed-by-fleet.md), [Run {{agent}} on GKE managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-gke-managed-by-fleet.md) and [Run {{agent}} on Amazon EKS managed by {{fleet}}](/reference/ingestion-tools/fleet/running-on-eks-managed-by-fleet.md) :::: diff --git a/reference/ingestion-tools/fleet/scaling-on-kubernetes.md b/reference/ingestion-tools/fleet/scaling-on-kubernetes.md index 864bf27b76..b3adfc67e6 100644 --- a/reference/ingestion-tools/fleet/scaling-on-kubernetes.md +++ b/reference/ingestion-tools/fleet/scaling-on-kubernetes.md @@ -31,7 +31,7 @@ The document is divided in two main sections: #### Configure agent resources [_configure_agent_resources] -The {{k8s}} {{observability}} is based on [Elastic {{k8s}} integration](asciidocalypse://docs/integration-docs/docs/reference/kubernetes.md), which collects metrics from several components: +The {{k8s}} {{observability}} is based on [Elastic {{k8s}} integration](integration-docs://reference/kubernetes.md), which collects metrics from several components: * **Per node:** diff --git a/reference/ingestion-tools/fleet/structure-config-file.md b/reference/ingestion-tools/fleet/structure-config-file.md index 633646c8e5..0ce62b9379 100644 --- a/reference/ingestion-tools/fleet/structure-config-file.md +++ b/reference/ingestion-tools/fleet/structure-config-file.md @@ -7,7 +7,7 @@ mapped_pages: The `elastic-agent.yml` policy file contains all of the settings that determine how {{agent}} runs. The most important and commonly used settings are described here, including input and output options, providers used for variables and conditional output, security settings, logging options, enabling of special features, and specifications for {{agent}} upgrades. -An `elastic-agent.yml` file is modular: You can combine input, output, and all other settings to enable the [{{integrations}}](asciidocalypse://docs/integration-docs/docs/reference/index.md) to use with {{agent}}. Refer to [Create a standalone {{agent}} policy](/reference/ingestion-tools/fleet/create-standalone-agent-policy.md) for the steps to download the settings to use as a starting point, and then refer to the following examples to learn about the available settings: +An `elastic-agent.yml` file is modular: You can combine input, output, and all other settings to enable the [{{integrations}}](integration-docs://reference/index.md) to use with {{agent}}. Refer to [Create a standalone {{agent}} policy](/reference/ingestion-tools/fleet/create-standalone-agent-policy.md) for the steps to download the settings to use as a starting point, and then refer to the following examples to learn about the available settings: * [Config file examples](/reference/ingestion-tools/fleet/config-file-examples.md) * [Use standalone {{agent}} to monitor nginx](/reference/ingestion-tools/fleet/example-standalone-monitor-nginx.md). diff --git a/reference/ingestion-tools/fleet/upgrade-integration.md b/reference/ingestion-tools/fleet/upgrade-integration.md index 37fc5b8ffb..0297464dcc 100644 --- a/reference/ingestion-tools/fleet/upgrade-integration.md +++ b/reference/ingestion-tools/fleet/upgrade-integration.md @@ -53,16 +53,16 @@ Some integration packages, like System, are installed by default during {{fleet} The following integrations are installed automatically when you select certain options in the {{fleet}} UI. All of them have an option to upgrade integration policies automatically, too: -* [Elastic Agent](asciidocalypse://docs/integration-docs/docs/reference/elastic_agent.md) - installed automatically when the default **Collect agent logs** or **Collect agent metrics** option is enabled in an {{agent}} policy). -* [Fleet Server](asciidocalypse://docs/integration-docs/docs/reference/fleet_server.md) - installed automatically when {{fleet-server}} is set up through the {{fleet}} UI. -* [System](asciidocalypse://docs/integration-docs/docs/reference/system.md) - installed automatically when the default **Collect system logs and metrics** option is enabled in an {{agent}} policy). +* [Elastic Agent](integration-docs://reference/elastic_agent/index.md) - installed automatically when the default **Collect agent logs** or **Collect agent metrics** option is enabled in an {{agent}} policy). +* [Fleet Server](integration-docs://reference/fleet_server/index.md) - installed automatically when {{fleet-server}} is set up through the {{fleet}} UI. +* [System](integration-docs://reference/system.md) - installed automatically when the default **Collect system logs and metrics** option is enabled in an {{agent}} policy). -The [Elastic Defend](asciidocalypse://docs/integration-docs/docs/reference/endpoint.md) integration also has an option to upgrade installation policies automatically. +The [Elastic Defend](integration-docs://reference/endpoint/index.md) integration also has an option to upgrade installation policies automatically. Note that for the following integrations, when the integration is updated automatically the integration policy is upgraded automatically as well. This behavior cannot be disabled. -* [Elastic APM](asciidocalypse://docs/integration-docs/docs/reference/apm.md) -* [Cloud Security Posture Management](asciidocalypse://docs/integration-docs/docs/reference/cloud_security_posture.md#cloud_security_posture-cloud-security-posture-management-cspm) +* [Elastic APM](integration-docs://reference/apm/index.md) +* [Cloud Security Posture Management](integration-docs://reference/cloud_security_posture/index.md#cloud-security-posture-management-cspm) * [Elastic Synthetics](/solutions/observability/apps/synthetic-monitoring.md) For integrations that support the option to auto-upgrade the integration policy, when this option is selected (the default), {{fleet}} automatically upgrades your policies when a new version of the integration is available. If there are conflicts during the upgrade, your integration policies will not be upgraded, and you’ll need to [upgrade integration policies manually](#upgrade-integration-policies-manually).