You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: solutions/observability/apps/llm-observability.md
+7-6Lines changed: 7 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,8 +8,8 @@ While LLMs hold incredible transformative potential, they also bring complex cha
8
8
To keep your LLM-powered applications reliable, efficient, cost-effective, and easy to troubleshoot, Elastic provides a powerful LLM observability framework including key metrics, logs, and traces, along with pre-configured, out-of-the-box dashboards that deliver deep insights into model prompts and responses, performance, usage, and costs.
9
9
Elastic’s end-to-end LLM observability is delivered through the following methods:
10
10
11
-
- Metrics and logs ingestion for LLM APIs (via Elastic integrations)
12
-
- APM tracing for OpenAI Models (via instrumentation)
11
+
- Metrics and logs ingestion for LLM APIs (via [Elastic integrations](https://www.elastic.co/guide/en/integrations/current/introduction.html))
12
+
- APM tracing for OpenAI Models (via [instrumentation](https://github.com/elastic/opentelemetry))
13
13
14
14
## Metrics and logs ingestion for LLM APIs (via Elastic integrations)
15
15
@@ -36,20 +36,21 @@ Check these instructions on how to setup and collect OpenTelemetry data for your
36
36
37
37
## Use cases
38
38
39
-
Understand LLM performance and reliability
40
-
For an SRE team optimizing a customer support system powered by Azure OpenAI, Elastic’s Azure OpenAI integration provides critical insights. They can quickly identify which model variants experience higher latency or error rates, enabling smarter decisions on model deployment or even switching providers based on real-time performance metrics.
39
+
### Understand LLM performance and reliability
40
+
41
+
For an SRE team optimizing a customer support system powered by Azure OpenAI, Elastic’s [Azure OpenAI integration](https://www.elastic.co/guide/en/integrations/current/azure_openai.html) provides critical insights. They can quickly identify which model variants experience higher latency or error rates, enabling smarter decisions on model deployment or even switching providers based on real-time performance metrics.
41
42
42
43
[image]
43
44
44
45
### Troubleshoot OpenAI-powered applications
45
46
46
-
Consider an enterprise utilizing an OpenAI model for real-time user interactions. Encountering unexplained delays, an SRE can use OpenAI tracing to dissect the transaction pathway, identify if one specific API call or model invocation is the bottleneck, and monitor a request to see the exact prompt and response between the user and the LLM. Such insight is pivotal for prompt resolution.
47
+
Consider an enterprise utilizing an OpenAI model for real-time user interactions. Encountering unexplained delays, an SRE can use OpenAI tracing to dissect the transaction pathway, identify if one specific API call or model invocation is the bottleneck, and monitor a request to see the exact prompt and response between the user and the LLM.
47
48
48
49
[image]
49
50
50
51
### Addressing cost and usage concerns
51
52
52
-
For cost-sensitive deployments, being acutely aware of which LLM configurations are more cost-effective is crucial. Elastic’s dashboards, pre-configured to display model usage patterns, help mitigate unnecessary spending effectively. You can find out-of-thebox dashboards for Azure OpenAI, OpenAI, Amazon Bedrock, and Google VertexAI models.
53
+
For cost-sensitive deployments, being acutely aware of which LLM configurations are more cost-effective is crucial. Elastic’s dashboards, pre-configured to display model usage patterns, help mitigate unnecessary spending effectively. You can use out-of-the-box dashboards for Azure OpenAI, OpenAI, Amazon Bedrock, and Google VertexAI models.
0 commit comments