Skip to content
Draft
5 changes: 4 additions & 1 deletion manage-data/ingest/ingest-reference-architectures.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/index.html
- https://www.elastic.co/guide/en/ingest/current/use-case-arch.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
---
Expand All @@ -27,5 +30,5 @@ You can host {{es}} on your own hardware or send your data to {{es}} on {{ecloud
| [*{{agent}} to proxy to Elasticsearch*](./ingest-reference-architectures/agent-proxy.md)<br><br>![Image showing connections between {{agent}} and {{es}} using a proxy](/manage-data/images/ingest-ea-proxy-es.png "") | Agents have [network restrictions](./ingest-reference-architectures/agent-proxy.md) that prevent connecting outside of the {{agent}} network Note that [{{ls}} as proxy](./ingest-reference-architectures/ls-networkbridge.md) is one option.<br> |
| [*{{agent}} to {{es}} with Kafka as middleware message queue*](./ingest-reference-architectures/agent-kafka-es.md)<br><br>![Image showing {{agent}} collecting data and using Kafka as a message queue enroute to {{es}}](/manage-data/images/ingest-ea-kafka.png "") | Kafka is your [middleware message queue](./ingest-reference-architectures/agent-kafka-es.md):<br><br>* [Kafka ES sink connector](./ingest-reference-architectures/agent-kafka-essink.md) to write from Kafka to {{es}}<br>* [{{ls}} to read from Kafka and route to {{es}}](./ingest-reference-architectures/agent-kafka-ls.md)<br> |
| [*{{ls}} to Elasticsearch*](./ingest-reference-architectures/ls-for-input.md)<br><br>![Image showing {{ls}} collecting data and sending to {{es}}](/manage-data/images/ingest-ls-es.png "") | You need to collect data from a source that {{agent}} can’t read (such as databases, AWS Kinesis). Check out the [{{ls}} input plugins](logstash-docs-md://lsr/input-plugins.md).<br> |
| [*Elastic air-gapped architectures*](./ingest-reference-architectures/airgapped-env.md)<br><br>![Image showing {{stack}} in an air-gapped environment](/manage-data/images/ingest-ea-airgapped.png "") | You want to deploy {{agent}} and {{stack}} in an air-gapped environment (no access to outside networks)<br> |
| [*Elastic air-gapped architectures*](./ingest-reference-architectures/airgapped-env.md) {applies_to}`serverless: unavailable`<br><br>![Image showing {{stack}} in an air-gapped environment](/manage-data/images/ingest-ea-airgapped.png "") | You want to deploy {{agent}} and {{stack}} in an air-gapped environment (no access to outside networks)<br> |

Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-apis.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-es-airgapped.html
applies_to:
stack: ga
deployment:
self: ga
products:
- id: elastic-agent
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-installed.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-kafka-es.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-kafka-essink.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
- id: logstash
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-kafka-ls.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
- id: logstash
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-ls-airgapped.html
applies_to:
stack: ga
deployment:
self: ga
products:
- id: elastic-agent
- id: logstash
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-ls.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
- id: logstash
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-proxy.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/agent-to-es.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/airgapped-env.html
applies_to:
stack: ga
deployment:
self: ga
products:
- id: elastic-agent
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/ls-enrich.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/ls-for-input.html
applies_to:
stack: ga
serverless: ga
products:
- id: logstash
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/ls-multi.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
- id: logstash
Expand All @@ -19,7 +22,7 @@ Use when
: Data collected by {{agent}} needs to be routed to different {{es}} clusters or non-{{es}} destinations depending on the content

Example
: Let’s take an example of a Windows workstation, for which we are collecting different types of logs using the System and Windows integrations. These logs need to be sent to different {{es}} clusters and to S3 for backup and a mechanism to send it to other destinations such as different SIEM solutions. In addition, the {{es}} destination is derived based on the type of datastream and an organization identifier.
: Let’s take an example of a Windows workstation, for which we are collecting different types of logs using the System and Windows integrations. These logs need to be sent to different {{es}} clusters and to S3 for backup and a mechanism to send it to other destinations such as different SIEM solutions. In addition, the {{es}} destination is derived based on the type of data stream and an organization identifier.

In such use cases, agents send the data to {{ls}} as a routing mechanism to different destinations. Note that the System and Windows integrations must be installed on all {{es}} clusters to which the data is routed.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/ls-networkbridge.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
- id: logstash
Expand Down
3 changes: 3 additions & 0 deletions manage-data/ingest/ingest-reference-architectures/lspq.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/ingest/current/lspq.html
applies_to:
stack: ga
serverless: ga
products:
- id: elastic-agent
- id: logstash
Expand Down
24 changes: 15 additions & 9 deletions reference/apm-agents/index.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,21 @@
---
applies_to:
stack: ga
serverless:
observability: ga
---

# APM agents

% TO-DO: Add links to "APM basics"%

This section contains reference information for APM Agents features, including:

* .NET
* Go
* Java
* Node.js
* PHP
* Python
* Ruby
* RUM JavaScript
* iOS
* [APM .NET agent](apm-agent-dotnet://reference/index.md)
* [APM Go agent](apm-agent-go://reference/index.md)
* [APM Java agent](apm-agent-java://reference/index.md)
* [APM Node.js agent](apm-agent-nodejs://reference/index.md)
* [APM PHP agent](apm-agent-php://reference/index.md)
* [APM Python agent](apm-agent-python://reference/index.md)
* [APM Ruby agent](apm-agent-ruby://reference/index.md)
* [APM RUM JavaScript agent](apm-agent-rum-js://reference/index.md) {applies_to}`serverless: unavailable`
4 changes: 3 additions & 1 deletion reference/fleet/add-cloud-metadata-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_cloud_metadata
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add-cloud-metadata-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down Expand Up @@ -53,7 +55,7 @@ The metadata that is added to events varies by hosting provider. For examples, r
| --- | --- | --- | --- |
| `timeout` | No | `3s` | Maximum amount of time to wait for a successful response when detecting the hosting provider. If a timeout occurs, no instance metadata is added to the events. This makes it possible to enable this processor for all your deployments (in the cloud or on-premise). |
| `providers` | No | | List of provider names to use. If `providers` is not configured, all providers that do not access a remote endpoint are enabled by default. The list of providers may alternatively be configured with the environment variable `BEATS_ADD_CLOUD_METADATA_PROVIDERS`, by setting it to a comma-separated list of provider names.<br><br>The list of supported provider names includes:<br><br>* `alibaba` or `ecs` for the Alibaba Cloud provider (disabled by default).<br>* `azure` for Azure Virtual Machine (enabled by default).<br>* `digitalocean` for Digital Ocean (enabled by default).<br>* `aws` or `ec2` for Amazon Web Services (enabled by default).<br>* `gcp` for Google Compute Engine (enabled by default).<br>* `openstack` or `nova` for Openstack Nova (enabled by default).<br>* `openstack-ssl` or `nova-ssl` for Openstack Nova when SSL metadata APIs are enabled (enabled by default).<br>* `tencent` or `qcloud` for Tencent Cloud (disabled by default).<br>* `huawei` for Huawei Cloud (enabled by default).<br> |
| `overwrite` | No | `false` | Whether to overwrite existing cloud fields. If `true`, the processoroverwrites existing `cloud.*` fields. |
| `overwrite` | No | `false` | Whether to overwrite existing cloud fields. If `true`, the processor overwrites existing `cloud.*` fields. |

The `add_cloud_metadata` processor supports SSL options to configure the http client used to query cloud metadata.

Expand Down
4 changes: 4 additions & 0 deletions reference/fleet/add-fleet-server-cloud.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add-fleet-server-cloud.html
applies_to:
deployment:
ess: ga
serverless: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
3 changes: 3 additions & 0 deletions reference/fleet/add-fleet-server-kubernetes.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add-fleet-server-kubernetes.html
applies_to:
deployment:
self: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
5 changes: 5 additions & 0 deletions reference/fleet/add-fleet-server-mixed.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add-fleet-server-mixed.html
applies_to:
deployment:
ece: ga
self: ga
ess: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
6 changes: 5 additions & 1 deletion reference/fleet/add-fleet-server-on-prem.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add-fleet-server-on-prem.html
applies_to:
deployment:
ece: ga
self: ga
products:
- id: fleet
- id: elastic-agent
Expand Down Expand Up @@ -55,7 +59,7 @@ You can install only a single {{agent}} per host, which means you cannot run {{f

Before deploying, you need to:

* Obtain or generate a Cerfiticate Authority (CA) certificate.
* Obtain or generate a Certificate Authority (CA) certificate.
* Ensure components have access to the ports needed for communication.


Expand Down
3 changes: 3 additions & 0 deletions reference/fleet/add-integration-to-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
navigation_title: Add an integration to an {{agent}} policy
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add-integration-to-policy.html
applies_to:
stack: ga
serverless: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
4 changes: 3 additions & 1 deletion reference/fleet/add_cloudfoundry_metadata-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_cloudfoundry_metadata
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_cloudfoundry_metadata-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down Expand Up @@ -64,5 +66,5 @@ For efficient annotation, application metadata retrieved by the Cloud Foundry cl
| `client_secret` | Yes | | Client Secret to authenticate with Cloud Foundry. |
| `cache_duration` | No | `120s` | Maximum amount of time to cache an application’s metadata. |
| `cache_retry_delay` | No | `20s` | Time to wait before trying to obtain an application’s metadata again in case of error. |
| `ssl` | No | | SSL configuration to use when connecting to Cloud Foundry. For a list ofavailable settings, refer to [SSL/TLS](/reference/fleet/elastic-agent-ssl-configuration.md), specificallythe settings under [Table 7, Common configuration options](/reference/fleet/elastic-agent-ssl-configuration.md#common-ssl-options) and [Table 8, Client configuration options](/reference/fleet/elastic-agent-ssl-configuration.md#client-ssl-options). |
| `ssl` | No | | SSL configuration to use when connecting to Cloud Foundry. For a list of available settings, refer to [SSL/TLS](/reference/fleet/elastic-agent-ssl-configuration.md), specifically the settings under [Table 7, Common configuration options](/reference/fleet/elastic-agent-ssl-configuration.md#common-ssl-options) and [Table 8, Client configuration options](/reference/fleet/elastic-agent-ssl-configuration.md#client-ssl-options). |

4 changes: 3 additions & 1 deletion reference/fleet/add_docker_metadata-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_docker_metadata
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_docker_metadata-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down Expand Up @@ -72,7 +74,7 @@ If the Docker daemon is restarted, the mounted socket will become invalid, and m
| Name | Required | Default | Description |
| --- | --- | --- | --- |
| `host` | No | `unix:///var/run/docker.sock` | Docker socket (UNIX or TCP socket). |
| `ssl` | No | | SSL configuration to use when connecting to the Docker socket. For a list ofavailable settings, refer to [SSL/TLS](/reference/fleet/elastic-agent-ssl-configuration.md), specificallythe settings under [Table 7, Common configuration options](/reference/fleet/elastic-agent-ssl-configuration.md#common-ssl-options) and [Table 8, Client configuration options](/reference/fleet/elastic-agent-ssl-configuration.md#client-ssl-options). |
| `ssl` | No | | SSL configuration to use when connecting to the Docker socket. For a list of available settings, refer to [SSL/TLS](/reference/fleet/elastic-agent-ssl-configuration.md), specifically the settings under [Table 7, Common configuration options](/reference/fleet/elastic-agent-ssl-configuration.md#common-ssl-options) and [Table 8, Client configuration options](/reference/fleet/elastic-agent-ssl-configuration.md#client-ssl-options). |
| `match_fields` | No | | List of fields to match a container ID. At least one of the fields most hold a container ID to get the event enriched. |
| `match_pids` | No | `["process.pid", "process.parent.pid"]` | List of fields that contain process IDs. If the process is running in Docker, the event will be enriched. |
| `match_source` | No | `true` | Whether to match the container ID from a log path present in the `log.file.path` field. |
Expand Down
2 changes: 2 additions & 0 deletions reference/fleet/add_fields-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_fields
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_fields-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
2 changes: 2 additions & 0 deletions reference/fleet/add_host_metadata-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_host_metadata
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_host_metadata-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
2 changes: 2 additions & 0 deletions reference/fleet/add_id-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_id
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_id-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
2 changes: 2 additions & 0 deletions reference/fleet/add_kubernetes_metadata-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_kubernetes_metadata
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_kubernetes_metadata-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
2 changes: 2 additions & 0 deletions reference/fleet/add_labels-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_labels
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_labels-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
2 changes: 2 additions & 0 deletions reference/fleet/add_locale-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_locale
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_locale-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
2 changes: 2 additions & 0 deletions reference/fleet/add_network_direction-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
navigation_title: add_network_direction
mapped_pages:
- https://www.elastic.co/guide/en/fleet/current/add_network_direction-processor.html
applies_to:
stack: ga
products:
- id: fleet
- id: elastic-agent
Expand Down
Loading
Loading