-
Notifications
You must be signed in to change notification settings - Fork 8
Open
Description
Description
When using LangChain with non-OpenAI providers (e.g., ChatSnowflake, ChatAnthropic), the gen_ai.request.model span attribute shows unknown_model instead of the actual model name. This makes it impossible to track which model was used for cost attribution, performance analysis, etc.
Environment
splunk-otel-instrumentation-langchain: 0.1.xlangchain-snowflake: 0.2.x (orlangchain-anthropic)- Provider: Snowflake Cortex (
ChatSnowflake)
Steps to Reproduce
from langchain_snowflake import ChatSnowflake
llm = ChatSnowflake(model="claude-3-5-sonnet", ...)
response = llm.invoke("Hello")
# Check span attributes in traceExpected Behavior
Span name: chat claude-3-5-sonnet
gen_ai.request.model: claude-3-5-sonnet
Actual Behavior
Span name: chat unknown_model
gen_ai.request.model: unknown_model
Evidence
Observed in production trace from Snowflake SPCS deployment:
- Trace ID:
90a9d9e471e7ba2a1c88f96e2bc46293 - Multiple
chat unknown_modelspans despite usingclaude-3-5-sonnet
📊 Trace Span Evidence
LLM span showing the bug:
{
"spanId": "ac8f798541d3812c",
"operationName": "chat unknown_model",
"tags": {
"gen_ai.request.model": "unknown_model",
"gen_ai.response.model": "claude-3-5-sonnet",
"gen_ai.provider.name": "snowflake",
"gen_ai.usage.input_tokens": 1108,
"gen_ai.usage.output_tokens": 99
}
}Key observation:
gen_ai.request.model: "unknown_model"❌ (should beclaude-3-5-sonnet)gen_ai.response.model: "claude-3-5-sonnet"✅ (correct from response)- The model is
claude-3-5-sonnetbut request model showsunknown_model
Full trace JSON: trace_flat_hierarchy_bug.json
Root Cause Analysis
The callback handler only checks invocation_params.model_name:
# Current code - only checks model_name (OpenAI convention)
model = invocation_params.get("model_name", "unknown_model")But ChatSnowflake and ChatAnthropic use invocation_params.model:
# What ChatSnowflake/ChatAnthropic provides
invocation_params = {"model": "claude-3-5-sonnet", ...}Provider Compatibility
| Provider | Field Used | Current Status |
|---|---|---|
OpenAI (ChatOpenAI) |
model_name |
✅ Works |
Snowflake (ChatSnowflake) |
model |
❌ Shows unknown_model |
Impact
- ❌ Cost tracking broken - can't attribute costs to specific models
- ❌ Performance analysis impossible - can't compare model latencies
- ❌ Span names misleading -
chat unknown_modelprovides no context - ❌ Dashboard filters broken - can't filter by model in Splunk O11y
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels