[Obs AI Assistant] Add LLM performance matrix docs

[viduni94]

Closes https://github.com/elastic/obs-ai-assistant-team/issues/347
Closes elastic/kibana#233110

This PR adds the LLM performance matrix and a link to the evaluation framework readme in the Observability AI Assistant docs.

The scores that were used to calculate the ratings are attached in the first issue linked above.

[github-actions]

🔍 Preview links for changed docs

• solutions/observability/llm-performance-matrix.md
• solutions/observability/connect-to-own-local-llm.md
• solutions/observability/observability-ai-assistant.md

[mdbirnstiehl]

A couple of suggestions, let me know if you have any questions or comments.

[mdbirnstiehl]

I would probably avoid putting dates in the docs.

[viduni94]

In scope and requirements of issue, there is a mention of

include “last updated” note.

@pmoust To clarify, should we include the last updated date of the performance matrix in the docs?

[pmoust]

@mdbirnstiehl we'd like to have some way to indicate that that's a "version" of our current supportability levels. If not a date, and given that we're not linking it to a stack release per se, what would you recommend @mdbirnstiehl ?

If no strong opinions, I'd just keep the date.

[benironside]

LGTM. Also @mdbirnstiehl this is another page we should consider consolidating since there's also a version for security

[arturoliduena]

LGTM

[viduni94]

LGTM. Also @mdbirnstiehl this is another page we should consider consolidating since there's also a version for security

Thanks @benironside
Would you be able to add a review to the PR if everything looks okay as Mike is on PTO?
Let me know if anything needs to change, happy to update.

[florent-leborgne]

LGTM with a few nits

[florent-leborgne]

Suggested change

	_Last updated: 15 September 2025_

Keeping manually maintained dates isn't something we do nor advise doing in the docs, because they're considered to match the latest release, not specific dates.

[viduni94]

Thanks @florent-leborgne
Could you check @pmoust's comment here - #2812 (comment)

Is there a way to link it to a stack release if we are removing the date?

[florent-leborgne]

The whole page will be marked as 9.2 thanks to the frontmatter

[viduni94]

Okay. Thanks @florent-leborgne

@pmoust are we okay with removing the date and only having the stack version we tested in?

[florent-leborgne]

It'll show like this when 9.2 is officially released

[viduni94]

@florent-leborgne

The requirement we have is to communicate the date we've evaluated the models with customers. This is important for Serverless too.
Is it okay to keep the date for this case?

We plan on updating these ratings whenever we come across a scenario in this comment.

cc: @pmoust

[viduni94]

If it helps, I can update it to say "the evaluations were done on the 15 September 2025
What do you think?

[florent-leborgne]

Yeah, more precise wording already sounds a bit better (even if manually maintained dates is still a bad practice in technical docs 😄). Something like this maybe?

Last LLM performance evaluation: 15 September 2025

• An FYI: We may surface automatically a "last updated" information on each docs page at some point, but we're very cautious with this wording (ex: fixing a typo or adding a new row doesn't necessarily mean that the entirety of the page was checked, validated, and updated at that date).
• Right now this is all new content, but if the results of these tests can vary per Stack version (or serverless) in the future, we'll have to think about how to present the information (one tab per version maybe or something similar), and about where to locate that date, to make sure it's attached to the right content/version on the page.
• Including such information at the start of the page implies that the entire matrix (all models) is checked and updated, not just a new model tested or so. If you're instead planning cases where only a small part of what shows in this doc will be tested, we may want to surface this date more granularly.
• If for some reason scenarios requiring to update this page become less frequent at some point, consider what to do with this date to avoid a "blog" effect, meaning a page written sometime in the past and becoming a liability because it's showing old dates.

Happy to discuss this further if you'd like to anticipate further updates, lmk :)

[pmoust]

@florent-leborgne @viduni94 to unblock the discussion here, I am ok to back away from having a "Last updated" date.
Let's remove the date, and continue the discussion outside of this github issue.
We shouldn't block merging on that.

[viduni94]

Thanks @pmoust and @florent-leborgne
I'll remove the date for now.

[florent-leborgne]

LGTM!