Skip to content

A troubleshooting section added and more doc improvements. #243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 18, 2025
Merged

A troubleshooting section added and more doc improvements. #243

Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8366f4f
A troubleshooting section added, supported processors revised and uns…
mashhurs Jan 22, 2025
592560a
Pipeline execution error cases added.
mashhurs Jan 27, 2025
ceb4c10
Update the ingest pipelines internal error case.
mashhurs Jan 27, 2025
f912d99
Revise the troubleshooting statements, add the Logstash behavior to t…
mashhurs Jan 27, 2025
b79ea1f
Update the content based on the doc guidelines recommendation, remove…
mashhurs Jan 28, 2025
96c0d81
Resource links corrected.
mashhurs Jan 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 74 additions & 8 deletions docs/index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ The plugin starts in an unsafe mode with a runtime error indicating that API per

To avoid these issues, set up user authentication and ensure that security in {es} is enabled (default).
--

[id="plugins-{type}s-{plugin}-supported_ingest_processors"]
==== Supported Ingest Processors

Expand All @@ -165,27 +165,27 @@ It has access to the Painless and Mustache scripting engines where applicable:

| `append` | _none_
| `bytes` | _none_
| `communityid` | _none_
| `community_id` | _none_
| `convert` | _none_
| `csv` | _none_
| `date` | _none_
| `dateindexname` | _none_
| `date_index_name` | _none_
| `dissect` | _none_
| `dotexpander` | _none_
| `dot_expander` | _none_
| `drop` | _none_
| `fail` | _none_
| `fingerprint` | _none_
| `foreach` | _none_
| `grok` | _none_
| `gsub` | _none_
| `htmlstrip` | _none_
| `html_strip` | _none_
| `join` | _none_
| `json` | _none_
| `keyvalue` | _none_
| `kv` | _none_
| `lowercase` | _none_
| `networkdirection` | _none_
| `network_direction` | _none_
| `pipeline` | resolved pipeline _must_ be wholly-composed of supported processors
| `registereddomain` | _none_
| `registered_domain` | _none_
| `remove` | _none_
| `rename` | _none_
| `reroute` | _none_
Expand All @@ -206,6 +206,21 @@ h| GeoIp

|=======================================================================

[id="plugins-{type}s-{plugin}-unsupported_ingest_processors"]
==== Unsupported Ingest Processors

This plugin has limited capabilities to execute all processors, as certain processors require external access or depend on additional resources that are not available in the plugin's environment.
Some processors may not function as expected, potentially leading to errors or silent failures in Logstash.
The following processors are known unsupported processors. Note that this list is not fully comprehensive:

[cols="<1,<5",options="header"]
|=======================================================================
|Processor | Behavior
| `set_security_user` | Logstash will not warn or error
| `inference` | Logstash raises `No processor type exists with name [inference]` error
| `enrich` | Logstash raises `No processor type exists with name [enrich]` error
|=======================================================================


[id="plugins-{type}s-{plugin}-field_mappings"]
===== Field Mappings
Expand Down Expand Up @@ -279,6 +294,57 @@ To achieve this, mappings are cached for a maximum of {cached-entry-ttl}, and ca
* when a reloaded mapping is newly _empty_, the previous non-empty mapping is _replaced_ with a new empty entry so that subsequent events will use the empty value
* when the reload of a mapping _fails_, this plugin emits a log warning but the existing cache entry is unchanged and gets closer to its expiry.

[id="plugins-{type}s-{plugin}-troubleshooting"]
==== Troubleshooting

Troubleshooting ingest pipelines associated with data streams requires a pragmatic approach, involving thorough analysis and debugging techniques.
To identify the root cause of issues with pipeline execution, it is essential to enable debug-level logging. This allows monitoring the plugin's behavior and detect any anomalies or errors that may be causing pipeline execution.
The plugin operates through following phases: pipeline _resolution_, ingest pipeline _creation_, and pipeline _execution_.

* If you encounter `No pipeline resolved for event ...` messages in the debug logs, it indicates that the plugin is unable to resolve the ingest pipeline from the data stream. In such cases, explicitly
defining the pipeline name using the <<plugins-{type}s-{plugin}-pipeline_name>> is a one option to resolve the issue.
To further diagnose and resolve the issue, verify whether the data stream's index settings include a `default_pipeline` or `final_pipeline` configuration.
You can inspect the index settings by running a simple query in the {kib} Dev Tools: `POST _index_template/_simulate_index/{type}-{dataset}-{namespace}`. Make sure to replace `{type}-{dataset}-{namespace}` with values corresponding to your data stream.

For further guidance, we recommend exploring following {es} {ref}/ingest.html#pipelines-for-fleet-elastic-agent[Ingest pipelines for fleet] and https://docs.elastic.co/integrations/all_integrations[Elastic {integrations}] resources.

* If you notice `pipeline not found: ...` debug messages in the logs or `Pipeline {pipeline-name} could not be loaded` warning messages, it indicates that the plugin has successfully resolved the ingest pipeline from `default_pipeline` or `final_pipeline`, but the specified pipeline does not exist.
To confirm whether pipeline exists, run a simple request in the {kib} Dev Tools console: `GET _ingest/pipeline/{ingest-pipeline-name}`.

For further guidance, it is recommended visiting {es} {ref}/ingest.html#pipelines-for-fleet-elastic-agent[Ingest pipelines for fleet] and https://docs.elastic.co/integrations/all_integrations[Elastic {integrations}] guidelines.

* If you encounter `failed to create ingest pipeline {pipeline-name} from pipeline configuration` error messages, it indicates that the plugin is unable to create an ingest pipeline from the resolved pipeline configuration.
This issue typically arises when the pipeline configuration contains unsupported or invalid processor(s) that the plugin cannot execute.
In such situations, the log output will mostly include a stack trace with detailed information about the issue.
For example, the following error message indicating `inference` processor in the pipeline configuration which is not supported processor type.

[source]
----
2025-01-21 12:29:13 [2025-01-21T20:29:13,986][ERROR][co.elastic.logstash.filters.elasticintegration.IngestPipelineFactory][main] failed to create ingest pipeline logs-my.custom-1.0.0 from pipeline configuration
2025-01-21 12:29:13 org.elasticsearch.ElasticsearchParseException: No processor type exists with name [inference]
2025-01-21 12:29:13 at org.elasticsearch.ingest.ConfigurationUtils.newConfigurationException(ConfigurationUtils.java:470) ~[logstash-filter-elastic_integration-0.1.16.jar:?]
2025-01-21 12:29:13 at org.elasticsearch.ingest.ConfigurationUtils.readProcessor(ConfigurationUtils.java:635)
----

Review the pipeline configuration for <<plugins-{type}s-{plugin}-unsupported_ingest_processors>> and refer to relevant integration and ingestion pipeline documentation for guidance.

- **Errors happened during pipeline execution**
These errors typically fall into two main categories, each requiring specific investigation and resolution steps:
1. **Logstash catches issues while running ingest pipelines**
When errors occur during the execution of ingest pipelines, Logstash will attach the `_ingest_pipeline_failure` tag to the event, making it easier to identify and investigate problematic events.
The detailed logs will be also available in the Logstash logs for your investigation.
The root cause may depend on configuration, environment or integration you are running.
Check out {ls} and {integrations} documents for further assistance.

2. **Errors internally occurred in the ingest pipeline**
If an ingest pipeline is configured with failure processors, failures during pipeline execution are internally handled by the ingest pipeline itself and will not be visible to Logstash. This means that errors are captured and processed within the pipeline, rather than being passed to Logstash for logging or tagging.
To identify and analyze such cases, go to the {kib} -> Stack Management -> Ingest pipelines and find the ingest pipeline you are using.
Click on it and navigate to the Failure processors section. If failure processors are configured, they will typically specify which field contains the failure details. For example, the pipeline might store error information in a field like `error.message` or a custom field defined in the failure processor configuration.
Go to the {kib} Dev Tools and search for the data (`GET index-ingest-pipeline-is-writing/_search`) and look for the fields mentioned in the failure processors .
The fields will have error details which will help you to analyze the root cause.

Check out {integrations} and {es} {ref}/ingest.html#handling-pipeline-failures[Handling pipeline failures] documents for further assistance.

[id="plugins-{type}s-{plugin}-options"]
==== {elastic-integration-name} Filter Configuration Options

Expand Down