Skip to content

Commit c10ad22

Browse files
committed
Add integrations and instrumentaions tables
1 parent df0544e commit c10ad22

File tree

1 file changed

+24
-0
lines changed

1 file changed

+24
-0
lines changed

solutions/observability/apps/llm-observability.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,17 @@ Elastic’s LLM integrations now support the most widely adopted models, includi
2020
- [GCP Vertex AI](https://www.elastic.co/guide/en/integrations/current/gcp_vertexai.html)
2121
- [OpenAI](https://www.elastic.co/guide/en/integrations/current/openai.html)
2222

23+
Depending on the LLM provider you choose, the following table shows which source you can use and which type of data -- log or metrics -- you can collect.
24+
25+
| **LLM Provider** | **Source** | **Metrics** | **Logs** | **Notes** |
26+
|--------|------------|------------|
27+
| [AWS Bedrock][int-bedrock]| [AWS CloudWatch Logs][impl-bedrock] ||| GA |
28+
| [Azure OpenAI][int-azure]| [Azure Monitor and Event Hubs][impl-azure] ||| GA |
29+
| [GCP Vertex AI][int-vertexai] | [GCP Cloud Monitoring][impl-vertexai] || 🚧 | GA, we are not able to collect meaningful information for request/response from logs due to dynamic generation, GCP are aware of this issue, not ETA yet |
30+
| [OpenAI][int-openai]| [OpenAI Usage API][openai-usage] || 🚧 | GA, cannot collect prompt/response logs until OpenAI provides support. |
31+
| [OpenTelemetry][int-wip-otel] | OTLP | 🚧 | 🚧 | This would support Elastic extensions of otel's GenAI semantic conventions |
32+
33+
2334
## APM tracing for OpenAI models (via instrumentation)
2435

2536
Elastic offers specialized OpenTelemetry Protocol (OTLP) tracing for applications leveraging OpeAI models hosted on OpenAI, Azure, and Amazon Bedrock, providing a detailed view of request flows. This tracing capability captures critical insights, including the specific models used, request duration, errors encountered, token consumption per request, and the interaction between prompts and responses. Ideal for troubleshooting, APM tracing allows you to find exactly where the issue is happening with precision and efficiency in your OpenAI-powered application.
@@ -30,6 +41,19 @@ You can instrument the application with one of the following OpenTelemetry API:
3041
- [Node.js](https://github.com/elastic/elastic-otel-node)
3142
- [Java](https://github.com/elastic/elastic-otel-java)
3243

44+
EDOT includes many types of instrumentation. The following table shows the status of instrumentation relevant to GenAI on a per-language basis:
45+
46+
47+
| **SDK** | **Language** | **Instrumented Dependency** | **Traces** | **Metrics** | **Logs** | Status | **Notes** |
48+
|-------|-----|----|-----|------|------|-----|------|
49+
| OpenAI | Python | [openai][edot-openai-py]||||| Tested on OpenAI, Azure and Ollama |
50+
| OpenAI| JS/Node | [openai][edot-openai-js] ||||| Tested on OpenAI, Azure and Ollama|
51+
| OpenAI| Java| [com.openai:openai-java][edot-openai-java] ||||| Tested on OpenAI, Azure and Ollama|
52+
| Langchain| JS/Node| [@langchain/core][wip-edot-langchain-js] || 🚧| 🚧 | 🔒| Tested on OpenAI; Not yet finished |
53+
| (AWS) Boto| Python| [botocore][otel-bedrock-py]||||| Bedrock (not SageMaker) `InvokeModel*` and `Converse*` APIs Owner: Riccardo |
54+
| Cohere| Python| [cohere][wip-otel-cohere-py] | 🚧 | 🚧 | 🚧 | 🚧 | Owner: Leighton from Microsoft |
55+
| Google Cloud AI Platform | Python | [google-cloud-aiplatform][otel-vertexai-py] || 🚧| 🚧| 🚧 | Vertex (not Gemini); Clashes with OpenLLMetry package |
56+
3357
## Getting started
3458

3559
Check these instructions on how to setup and collect OpenTelemetry data for your LLM applications [create a link to https://github.com/elastic/opentelemetry/pull/100/files#diff-965570d21670c0ee4bba4b303960e5fe83b285f66b001ff8f31f0413f65a9d47 once the content is finalized and merged]

0 commit comments

Comments
 (0)