[Do not merge] [Obs AI Assistant] Adds docs for connecting to a local LLM with the Obs AI Assistant

[viduni94]

Closes https://github.com/elastic/obs-ai-assistant-team/issues/322

[DO NOT MERGE] This PR can only be merged once llama support is complete.

This PR adds documentation about how to connect to a local LLM with the Observability AI Assistant.

[viduni94]

Added a small disclaimer here - let me know if I should remove it.

[viduni94]

I'm waiting for the preview to be generated to double check the links and formatting.

[github-actions]

🔍 Preview links for changed docs

• solutions/observability/connect-to-own-local-llm.md
• solutions/observability/observability-ai-assistant.md

[viduni94]

@SrdjanLL

Is it a must to use Nginx for this purpose? If not, I'd suggest mentioning we have an example of setting up NGINX reverse proxy, but to be less prescriptive, because customers may be using other reverse proxies (HAProxy, Envoy Proxy, Caddy, etc.).

The reverse proxy configuration in the doc I've linked mentions the follows. While the user can use alternatives to Nginx for testing purposes, the recommendation is to use Nginx to collect telemetry via the Elastic's Nginx integration.

For testing, you can use alternatives to Nginx such as Azure Dev Tunnels or Ngrok, but using Nginx makes it easy to collect additional telemetry and monitor its status by using Elastic’s native Nginx integration. While this example uses cloud infrastructure, it could also be replicated locally without an internet connection.

[viduni94]

Hi @mdbirnstiehl
The PR is ready for review. I've addressed the comments from @benironside

Let me know if I need to add anything to target this for Serverless and ECH 9.2
Thank you

[arturoliduena]

Thanks, @viduni94, for adding this documentation. I have created a task to add vLLM documentation: elastic/kibana#232052

[arturoliduena]

@viduni94, should we add the minimum requirement to run llama3.3? WDYT?
e.g, Memory GPU, Nvidia drivers, disk size, etc

[viduni94]

@viduni94, should we add the minimum requirement to run llama3.3? WDYT? e.g, Memory GPU, Nvidia drivers, disk size, etc

It might not be necessary to add this because it's general information and not specific to Elastic.

[viduni94]

Thanks, @viduni94, for adding this documentation. I have created a task to add vLLM documentation: elastic/kibana#232052

Thanks for the review @arturoliduena
It might not be necessary (but a good to have) to add how to connect via vLLM. The most important part of the documentation is connector creation since that's the only part specific to Elastic. The other sections (connecting to LM studio) is added for completeness. Let's discuss with the team.

[mdbirnstiehl]

Thank you putt this together! I've added some comments and suggestions. Let me know if you have any questions.