[Internal]: Update Host-related documentation to reflect Schema Handling in OTel vs Elastic Schema

[roshan-elastic]

Description

We're updating the Hosts and Infrastructure Inventory UIs to ensure that Hosts observed via OTel can be analysed fully. We need to make this change because the inframetricsprocessor we previously relied on to reformat OTel data into the Elastic schema our UIs relied on it being deprecated.

In short, the following has been done:

Hosts UI

• Will have a schema selector to choose between different schemas (as will the inventory rules for 'hosts')
• The schema selector will modify the experience to work for the given schema (inc filtering out hosts not matching that schema and removing functionality like 'anomaly detection' which doesn't work for OTel Hosts)
• The individual host views will adapt to show the correct view based on the schema (inc metadata, metrics, alerts and processes)

Infra Inventory UI*

• When 'hosts' are selected, it will have the same 'schema selector' with same logic as per above

*We may wish to remove the schema selecting as part of this upcoming work so bear in mind by the time you update the documentation for this, we may have different requirements for the Infra Inventory

Resources

This feature was scoped and researched in Issue.

Parent Issues

• https://github.com/elastic/observability-dev/issues/4537

Notable Sub-Issues

• https://github.com/elastic/observability-dev/issues/4727
• [Infra][Hosts UI] Asset Details OTel metrics kibana#226356
• [Infra][Hosts UI] OTel Processes tab kibana#226360
• [Infra][Hosts UI] Anomaly Detection - OTel kibana#226339
• [Infra][Hosts UI] OTel Inventory Rules kibana#226337
• https://github.com/elastic/observability-dev/issues/4608

Notable PRs

• [ObsUX][HostsUI] Processes tab with semconv schema kibana#231283
• [ObsUX][Otel Hosts] Schema selection kibana#228418
• [Infra] Hide anomaly detection for Otel kibana#230827
• [ObsUX][Infra][Hosts UI] OTel Metadata kibana#231579
• [Infra] Inventory rules Otel support kibana#229487

We need to update the following sets of documentation:

• Host Metrics
• Host Docs (specifically troubleshooting, View Host Details and Filter the Hosts View.
• Detect Metric Anomalies
• Inventory Rules

Which documentation set does this change impact?

Elastic On-Prem and Cloud (all)

Feature differences

See the issue for details and also the description above.

We will need to explain the following to users:

1. How the schema selection works in the Hosts and Infra Inventory docs : We'll need a heading that we can refer to by URL via an # in the URL - we are going to point to this from the UI
2. Update the metrics reference to include the OTel metric definitions
3. Explain that Host Anomaly Detection won't work for OTel hosts
4. Explain that the Inventory Rules will have a schema selector when you select 'Hosts'
5. How the Host processes work in both schema modes*
6. How users configure the process collection in both modes* (I suggest we point users towards the System Integration docs for Elastic System Integration 'mode' and the EDOT docs for OpenTelemetry 'mode')

*We'll need a heading that we can refer to by URL via an # in the URL - we are going to point to this from the UI

What release is this request related to?

Target 9.1.3

Serverless release

Target 1st Sept

*Work is planned to be completed early iteration 116 (11-29 Aug) so it will go into production in the next stateful release and serverless released after that.

Collaboration model

The documentation team

Point of contact.

Main contact: @roshan-elastic

Stakeholders: @crespocarlos @MiriamAparicio

[roshan-elastic]

FYI @bmorelli25 we'll be pointing users from the UI to some short-links to the content this issue will be creating.

As a fallback, the short-links will point towards the standard Hosts docs but once the content has been updated I can easily update the short-links to point towards the new content (this means there's a chance the content we expect won't be there but this should only impact serverless customers if it takes a bit longer to complete this issue)

[bmorelli25]

cc @mdbirnstiehl for visibility

[roshan-elastic]

Hey @mdbirnstiehl - we're ready to merge the work in now (which should give us until 9.2 for stateful and likely 1 Sept for serverless for the docs to be ready):

(see PR)

Do you think we'll be able to get the docs updated by then?

I'll be around to discuss a strategy whenever you get onto this.

[mdbirnstiehl]

Hey @mdbirnstiehl - we're ready to merge the work in now (which should give us until 9.2 for stateful and likely 1 Sept for serverless for the docs to be ready):

(see PR)

Do you think we'll be able to get the docs updated by then?

I'll be around to discuss a strategy whenever you get onto this.

Hi @roshan-elastic, I'll do my best to get this done by Sept 1. Let me look over the ticket, and I'll set up some time for us if I need further explanation.