Skip to content

[Bug] gen_ai.request.model shows 'unknown_model' for non-OpenAI providers #124

@NikitaVoitov

Description

@NikitaVoitov

Description

When using LangChain with non-OpenAI providers (e.g., ChatSnowflake, ChatAnthropic), the gen_ai.request.model span attribute shows unknown_model instead of the actual model name. This makes it impossible to track which model was used for cost attribution, performance analysis, etc.

Environment

  • splunk-otel-instrumentation-langchain: 0.1.x
  • langchain-snowflake: 0.2.x (or langchain-anthropic)
  • Provider: Snowflake Cortex (ChatSnowflake)

Steps to Reproduce

from langchain_snowflake import ChatSnowflake

llm = ChatSnowflake(model="claude-3-5-sonnet", ...)
response = llm.invoke("Hello")
# Check span attributes in trace

Expected Behavior

Span name: chat claude-3-5-sonnet
gen_ai.request.model: claude-3-5-sonnet

Actual Behavior

Span name: chat unknown_model
gen_ai.request.model: unknown_model

Evidence

Observed in production trace from Snowflake SPCS deployment:

  • Trace ID: 90a9d9e471e7ba2a1c88f96e2bc46293
  • Multiple chat unknown_model spans despite using claude-3-5-sonnet
📊 Trace Span Evidence

LLM span showing the bug:

{
  "spanId": "ac8f798541d3812c",
  "operationName": "chat unknown_model",
  "tags": {
    "gen_ai.request.model": "unknown_model",
    "gen_ai.response.model": "claude-3-5-sonnet",
    "gen_ai.provider.name": "snowflake",
    "gen_ai.usage.input_tokens": 1108,
    "gen_ai.usage.output_tokens": 99
  }
}

Key observation:

  • gen_ai.request.model: "unknown_model" ❌ (should be claude-3-5-sonnet)
  • gen_ai.response.model: "claude-3-5-sonnet" ✅ (correct from response)
  • The model is claude-3-5-sonnet but request model shows unknown_model

Full trace JSON: trace_flat_hierarchy_bug.json

Image

Root Cause Analysis

The callback handler only checks invocation_params.model_name:

# Current code - only checks model_name (OpenAI convention)
model = invocation_params.get("model_name", "unknown_model")

But ChatSnowflake and ChatAnthropic use invocation_params.model:

# What ChatSnowflake/ChatAnthropic provides
invocation_params = {"model": "claude-3-5-sonnet", ...}

Provider Compatibility

Provider Field Used Current Status
OpenAI (ChatOpenAI) model_name ✅ Works
Snowflake (ChatSnowflake) model ❌ Shows unknown_model

Impact

  • Cost tracking broken - can't attribute costs to specific models
  • Performance analysis impossible - can't compare model latencies
  • Span names misleading - chat unknown_model provides no context
  • Dashboard filters broken - can't filter by model in Splunk O11y

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions