Skip to content

Fix errors in 9.0 build #1877

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Fix errors in 9.0 build #1877

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions docs/plugins/filters/elastic_integration.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ START - GENERATED VARIABLES, DO NOT EDIT!
:version: v9.0.0
:release_date: 2025-04-28
:changelog_url: https://github.com/elastic/logstash-filter-elastic_integration/blob/v9.0.0/CHANGELOG.md
:include_path: ../../../../logstash/docs/include
:include_path: ../include
///////////////////////////////////////////
END - GENERATED VARIABLES, DO NOT EDIT!
///////////////////////////////////////////
Expand Down Expand Up @@ -53,21 +53,21 @@ Events that _fail_ ingest pipeline processing will be tagged with `_ingest_pipel

- This plugin requires Java 17 minimum with {ls} `8.x` versions and Java 21 minimum with {ls} `9.x` versions.
- When you upgrade the {stack}, upgrade {ls} (or this plugin specifically) _before_ you upgrade {kib}.
(Note that this requirement is a departure from the typical {stack} https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html#install-order-elastic-stack[installation order].)
(Note that this requirement is a departure from the typical {stack} https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html#install-order-elastic-stack[installation order].)
+
The {es}-{ls}-{kib} installation order recommended here ensures the best experience with {agent}-managed pipelines, and embeds functionality from a version of {es} Ingest Node that is compatible with the plugin version (`major`.`minor`).
The {es}-{ls}-{kib} installation order recommended here ensures the best experience with {agent}-managed pipelines, and embeds functionality from a version of {es} Ingest Node that is compatible with the plugin version (`major`.`minor`).

[id="plugins-{type}s-{plugin}-es-tips"]
===== Using `filter-elastic_integration` with `output-elasticsearch`

Elastic {integrations} are designed to work with {logstash-ref}/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data-streams[data streams] and {logstash-ref}/plugins-outputs-elasticsearch.html#_compatibility_with_the_elastic_common_schema_ecs[ECS-compatible] output.
Be sure that these features are enabled in the {logstash-ref}/plugins-outputs-elasticsearch.html[`output-elasticsearch`] plugin.
Be sure that these features are enabled in the {logstash-ref}/plugins-outputs-elasticsearch.html[`output-elasticsearch`] plugin.

* Set {logstash-ref}/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data_stream[`data-stream`] to `true`. +
(Check out {logstash-ref}/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data-streams[Data streams] for additional data streams settings.)
* Set {logstash-ref}/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data_stream[`data-stream`] to `true`. +
(Check out {logstash-ref}/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data-streams[Data streams] for additional data streams settings.)
* Set {logstash-ref}/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-ecs_compatibility[`ecs-compatibility`] to `v1` or `v8`.

Check out the {logstash-ref}/plugins-outputs-elasticsearch.html[`output-elasticsearch` plugin] docs for additional settings.
Check out the {logstash-ref}/plugins-outputs-elasticsearch.html[`output-elasticsearch` plugin] docs for additional settings.

[id="plugins-{type}s-{plugin}-minimum_configuration"]
==== Minimum configuration
Expand Down Expand Up @@ -145,12 +145,12 @@ At the startup phase, this plugin confirms that current user has sufficient priv

|=======================================================================

[NOTE]
[NOTE]
--
This plugin cannot determine if an anonymous user has the required privileges when it connects to an {es} cluster that has security features disabled or when the user does not provide credentials.
The plugin starts in an unsafe mode with a runtime error indicating that API permissions are insufficient, and prevents events from being processed by the ingest pipeline.

To avoid these issues, set up user authentication and ensure that security in {es} is enabled (default).
To avoid these issues, set up user authentication and ensure that security in {es} is enabled (default).
--

[id="plugins-{type}s-{plugin}-supported_ingest_processors"]
Expand Down
4 changes: 4 additions & 0 deletions docs/plugins/index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,7 @@ include::outputs.asciidoc[]
include::filters.asciidoc[]

include::codecs.asciidoc[]

// AsciiDoc "redirects"

include::static/other/dead-letter-queues.asciidoc[]
4 changes: 4 additions & 0 deletions docs/plugins/static/other/dead-letter-queues.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[id="dead-letter-queues",role="exclude"]
== Dead letter queues (DLQ)

Refer to {logstash-ref}/dead-letter-queues.html[Dead Letter Queues].
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about adding yet another way to handle bad links--we fixed many in logstash-docs-md and we have redirects elsewhere. For example, we fixed lined 64 and 66 in logstash-docs-md.

Is this approach temporary or part of a broader strategy?

Copy link
Author

@colleenmcginnis colleenmcginnis Jul 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know. How many PRs would it take to update this link to use {logstash-ref}/dead-letter-queues.html permanently so the change wouldn't be overwritten in future updates? (You've scared me into not wanting to attempt to edit AsciiDoc files at the source. 🙃)

Copy link
Contributor

@karenzone karenzone Jul 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've scared me into not wanting to attempt to edit AsciiDoc files at the source.

Any fixes we make in logstash-docs would persist <except for some weird, almost edge case multi-release-during-the-same-stack-version-updates>. Even then, we could see the regression in the diff, and ideally, fix it permanently with your amazing regex skills.

Weird situations like these are one of the reasons that I feel so strongly that our solution should use incremental updates. If we make whatever is in github code persistent unless deliberately overwritten, then we don't have to worry about anything getting stomped ever.

I never meant to scare anybody! :-) There are boundaries, of course, but sometimes fixing in ADOC the best approach. We just need to be sure to fix anything we can in the plugin source.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird situations like these are one of the reasons that I feel so strongly that our solution should use incremental updates. If we make whatever is in github code persistent unless deliberately overwritten, then we don't have to worry about anything getting stomped ever.

What's the time commitment required to get to incremental updates? We need to balance effort vs output and I'm hesitant to put a lot of time into building an ideal temporary solution.

Copy link
Contributor

@karenzone karenzone Jul 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point, it's just a wish and a rudimentary idea:
"Generated jobs (VPR and LSR) create new, unique branches for content. When compared against baseline source content, we get a diff. We could take content from the newly created branch, ship it to logstash-docs-md, and run the conversion there (ADOC->MD), touching only changed files. This approach ensures that any manually touched up files persist."

If we think the regenerate-and-overwrite approach is a short-term solution, I'll go along with it. I'd still prefer not to introduce another way to handle redirects, and hard code the link instead.