elastic · mergify · May 29, 2025
@@ -0,0 +1,21 @@
+---
+name: Comment on PR for .asciidoc changes
+
+on:
+  # We need to use pull_request_target to be able to comment on PRs from forks
+  pull_request_target:
+    types:
+      - synchronize
+      - opened
+      - reopened
+    branches:
+      - main
+      - master
+      - "9.0"
+
+jobs:
+  comment-on-asciidoc-change:
+    permissions:
+      contents: read
+      pull-requests: write
+    uses: elastic/docs-builder/.github/workflows/comment-on-asciidoc-changes.yml@main
@@ -0,0 +1,18 @@
+---
+name: github-commands-comment
+
+on:
+  pull_request_target:
+    types:
+      - opened
+
+permissions:
+  contents: read
+
+jobs:
+  comment:
+    runs-on: ubuntu-latest
+    permissions:
+      pull-requests: write
+    steps:
+      - uses: elastic/oblt-actions/elastic/github-commands@v1
@@ -15,6 +15,75 @@ pull_request_rules:
           git merge <remote-repo>/{{base}}
           git push <remote-repo> {{head}}
           ```
+  - name: backport patches to main branch
+    conditions:
+      - merged
+      - label=backport-main
+    actions:
+      backport:
+        assignees:
+          - "{{ author }}"
+        labels:
+          - "backport"
+        branches:
+          - "main"
+        title: "[{{ destination_branch }}](backport #{{ number }}) {{ title }}"
+  - name: backport patches to 8.x branch
+    conditions:
+      - merged
+      - base=main
+      - label=backport-8.x
+    actions:
+      backport:
+        assignees:
+          - "{{ author }}"
+        branches:
+          - "8.x"
+        title: "[{{ destination_branch }}] {{ title }} (backport #{{ number }})"
+        labels:
+          - backport
+  - name: backport patches to 8.18 branch
+    conditions:
+      - merged
+      - base=main
+      - label=backport-8.18
+    actions:
+      backport:
+        assignees:
+          - "{{ author }}"
+        branches:
+          - "8.18"
+        title: "[{{ destination_branch }}] {{ title }} (backport #{{ number }})"
+        labels:
+          - backport
+  - name: backport patches to 8.17 branch
+    conditions:
+      - merged
+      - base=main
+      - label=backport-8.17
+    actions:
+      backport:
+        assignees:
+          - "{{ author }}"
+        branches:
+          - "8.17"
+        title: "[{{ destination_branch }}] {{ title }} (backport #{{ number }})"
+        labels:
+          - backport
+  - name: backport patches to 8.16 branch
+    conditions:
+      - merged
+      - base=main
+      - label=backport-8.16
+    actions:
+      backport:
+        assignees:
+          - "{{ author }}"
+        branches:
+          - "8.16"
+        title: "[{{ destination_branch }}] {{ title }} (backport #{{ number }})"
+        labels:
+          - backport
   - name: backport patches to 8.15 branch
     conditions:
       - merged

@@ -18,6 +18,7 @@ include::8-ls-input.asciidoc[]
 
 include::99-airgapped.asciidoc[]
 
+
 // === Next set of architectures
 // include::3-schemamod.asciidoc[]
 // include::6b-filebeat-es.asciidoc[]

@@ -0,0 +1,12 @@
+include::{docs-root}/shared/versions/stack/{source_branch}.asciidoc[]
+include::{docs-root}/shared/attributes.asciidoc[]
+
+:doctype: book
+
+[[ingest-guide]]
+= Elastic Ingest Overview
+
+include::ingest-intro.asciidoc[]
+include::ingest-tools.asciidoc[]
+include::ingest-additional-proc.asciidoc[]
+include::ingest-solutions.asciidoc[]
@@ -0,0 +1,27 @@
+[[ingest-addl-proc]]
+== Additional ingest processing
+
+You can start with {agent} and Elastic {integrations-docs}[integrations], and still
+take advantage of additional processing options if you need them.
+
+{agent} processors::
+You can use link:{fleet-guide}/elastic-agent-processor-configuration.html[{agent} processors] to sanitize or enrich raw data at the source.
+Use {agent} processors if you need to control what data is sent across the wire, or if you need to enrich the raw data with information available on the host.
+
+{es} ingest pipelines::
+You can use {es} link:{ref}/[ingest pipelines] to enrich incoming data or normalize field data before the data is indexed.
+{es} ingest pipelines enable you to manipulate the data as it comes in. 
+This approach helps you avoid adding processing overhead to the hosts from which you're collecting data.
+
+{es} runtime fields::  
+You can use {es} link:{ref}/runtime.html[runtime fields] to define or alter the schema at query time.
+You can start working with your data without needing to understand how it is
+structured, add fields to existing documents without reindexing your data,
+override the value returned from an indexed field, and/or define fields for a
+specific use without modifying the underlying schema.
+
+{ls} `elastic_integration filter`::
+You can use the {ls} link:{logstash-ref}/[`elastic_integration filter`] and
+other link:{logstash-ref}/filter-plugins.html[{ls} filters] to
+link:{logstash-ref}/ea-integrations.html[extend Elastic integrations] by
+transforming data before it goes to {es}.
@@ -0,0 +1,77 @@
+[[ingest-faq]]
+== Frequently Asked Questions
+
+Q: What Elastic products and tools are available for ingesting data into Elasticsearch. 
+
+Q: What's the best option for ingesting data?
+
+Q: What's the role of Logstash `filter-elastic-integration`? 
+
+
+
+.WORK IN PROGRESS
+****
+Temporary parking lot to capture outstanding questions and notes.
+****
+
+
+
+Also cover (here or in general outline): 
+
+- https://www.elastic.co/guide/en/kibana/master/connect-to-elasticsearch.html#_add_sample_data[Sample data]
+- OTel
+- Beats
+- Use case: GeoIP
+- Airgapped
+- Place for table, also adding use case + products (Exp: Logstash for multi-tenant)
+- Role of LS in general content use cases 
+
+
+
+[discrete]
+=== Questions to answer:
+
+* Messaging for data sources that don't have an integration 
+    - We're deemphasizing beats in preparation for deprecation
+    - We're not quite there with OTel yet
+    * How should we handle this in the near term? 
+        Probably doesn't make sense to either ignore or jump them straight to Logstash
+
+* Should we mention Fleet and Stand-alone agent? 
+** If so, when, where, and how? 
+* How does this relate to Ingest Architectures
+* Enrichment for general content
+
+* How to message current vs. desired state. 
+    Especially Beats and OTel. 
+* HOW TO MESSAGE OTel - Current state. Future state. 
+* Consistent use of terminology vs. matching users' vocabulary (keywords)
+
+[discrete]
+==== Random
+
+* DocsV3 - need for a sheltered space to develop new content
+** Related: https://github.com/elastic/docsmobile/issues/708         
+** Need a place to incubate a new doc (previews, links, etc.) 
+** Refine messaging in private
+
+
+[discrete]
+=== Other resources to use, reference, reconcile
+
+* Timeseries decision tree (needs updates)
+* PM's video
+** Needs an update. (We might relocate content before updating.) 
+* PM's product table
+** Needs an update.(We might relocate content before updating.) 
+** Focuses on Agent over integrations.
+** Same link text resolves to different locations. 
+** Proposal: Harvest the good and possibly repurpose the table format. 
+* Ingest Reference architectures
+* Linkable content such as beats? Solutions ingest resources? 
+
+* https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current/getting-started-guides.html[Starting with the Elastic Platform and Solutions]
+* https://www.elastic.co/guide/en/observability/current/observability-get-started.html[Get started with Elastic Observability]
+* https://www.elastic.co/guide/en/security/current/ingest-data.html[Ingest data into Elastic Security]
+* 
+
@@ -0,0 +1,59 @@
+[[ingest-intro]]
+== Ingesting data into {es}
+
+Bring your data!
+Whether you call it _adding_, _indexing_, or _ingesting_ data, you have to get
+the data into {es} before you can search it, visualize it, and use it for insights. 
+
+Our ingest tools are flexible, and support a wide range of scenarios.
+We can help you with everything from popular and straightforward use cases, all
+the way to advanced use cases that require additional processing in order to modify or
+reshape your data before it goes to {es}.
+
+You can ingest:
+
+* **General content** (data without timestamps), such as HTML pages, catalogs, and files 
+* **Timestamped (time series) data**, such as logs, metrics, and traces for Elastic Security, Observability, Search solutions, or for your own custom solutions 
+
+[discrete]
+[[ingest-general]]
+=== Ingesting general content
+
+Elastic offer tools designed to ingest specific types of general content. 
+The content type determines the best ingest option.
+
+* To index **documents** directly into {es}, use the {es} link:{ref}/docs.html[document APIs].
+* To send **application data** directly to {es}, use an link:https://www.elastic.co/guide/en/elasticsearch/client/index.html[{es}
+language client]. 
+* To index **web page content**, use the Elastic link:https://www.elastic.co/web-crawler[web crawler].
+* To sync **data from third-party sources**, use link:{ref}/es-connectors.html[connectors]. 
+  A connector syncs content from an original data source to an {es} index. 
+  Using connectors you can create _searchable_, read-only replicas of your data sources.
+* To index **single files** for testing in a non-production environment, use the {kib} link:{kibana-ref}/connect-to-elasticsearch.html#upload-data-kibana[file uploader].
+
+If you would like to try things out before you add your own data, try using our {kibana-ref}/connect-to-elasticsearch.html#_add_sample_data[sample data].
+
+[discrete]
+[[ingest-timestamped]]
+=== Ingesting time-stamped data
+
+[ingest-best-timestamped]
+.What's the best approach for ingesting time-stamped data? 
+**** 
+The best approach for ingesting data is the _simplest option_ that _meets your needs_ and _satisfies your use case_.
+
+In most cases, the _simplest option_ for ingesting timestamped data is using {agent} paired with an Elastic integration.
+
+* Install {fleet-guide}[Elastic Agent] on the computer(s) from which you want to collect data.
+* Add the {integrations-docs}[Elastic integration] for the data source to your deployment.
+
+Integrations are available for many popular platforms and services, and are a
+good place to start for ingesting data into Elastic solutions--Observability,
+Security, and Search--or your own search application.
+
+Check out the {integrations-docs}/all_integrations[Integration quick reference]
+to search for available integrations. 
+If you don't find an integration for your data source or if you need
+additional processing to extend the integration, we still have you covered.
+Check out <<ingest-addl-proc,additional processing>> for a sneak peek.
+****