OTEL-based Python SDK v3 released in beta

[hassiebp]

We're introducing the Langfuse Python SDK v3 (OpenTelemetry-based) in beta.

This is a significant update to our Python SDK as it is now built on the OpenTelemetry (OTEL) standard and designed to improve developer experience and enhance integration capabilities for tracing your LLM applications.

Key Improvement: OpenTelemetry foundation

The foundation of the v3 SDK is OpenTelemetry, which brings several practical advantages:

• Standardized Context Propagation: OTEL automatically handles the propagation of trace and span context. This means when you create a new span or generation, it correctly nests under the currently active operation.
• Third-Party Library Compatibility: Libraries already instrumented with OpenTelemetry will integrate with the Langfuse SDK, with their spans being captured and correctly nested within your Langfuse traces.

Simplified Tracing and Global Client Access

The new SDK aims to make tracing more straightforward. Here's an example of how tracing can be implemented across different modules, leveraging automatic context propagation and global client access:

# main_app.py
from langfuse import get_client, observe
import other_module

# If no client is initialized, it will be initialized with the default configuration
# from the environment variables
langfuse = get_client()

@observe(name="user-request-pipeline")
def handle_user_request(user_query: str, user_id: str):
    # This span is now active. Enrich the trace with user info.
    langfuse.update_current_trace(user_id=user_id, tags=["experimental"])

    # Call a function from another module
    processed_data = other_module.process_data(user_query)

    # Update the root span
    langfuse.update_current_span(output={"final_result": processed_data})
    return processed_data

handle_user_request("Tell me a joke about OpenTelemetry", "user_123")

# other_module.py
from langfuse import get_client, observe

# Access the initialized Langfuse client globally
# If no client is initialized, it will be initialized with the default configuration
# from the environment variables
langfuse_client = get_client()

@observe(name="data-processing-step")
def process_data(query: str):
    # This span is automatically a child of "user-request-pipeline".

    with langfuse_client.start_as_current_generation(
        name="joke-generation-llm",
        model="gpt-4o",
        input=[{"role": "user", "content": query}]
    ) as generation:
        # Simulate LLM call
        joke = "Why did the OpenTelemetry collector break up with the span? It needed more space for its attributes!"
        generation.update(output=joke, usage_details={"input_tokens": 10, "output_tokens": 25})

    return joke

In this example:

• Langfuse is initialized once.
• The @observe decorator creates spans for handle_user_request and process_data.
• other_module.py accesses the same client instance via get_client().
• Spans are automatically nested due to OTEL's context propagation.
• Trace and span updates can be done contextually.

We encourage you to try the new Python SDK v3 beta and welcome your feedback here in this discussion!

Learn more

• Changelog post
• Python SDK V3 docs
• Upgrading from V2
• OpenTelemetry Specification
• OpenTelemetry Python SDK docs

[Leo310]

With v3, the integration with Langchain appears significantly more complicated compared to v2:

I’m having trouble understanding how this enhances the developer experience.
So I would appreciate it if you could explain the rationale behind this new approach.

[Leo310]

@hassiebp My use case was tracking feedback associated with specific chain runs. After executing a chain and submitting feedback, I wanted to be able to retrieve this feedback from langfuse later by using the ⁠run_id to identify the relevant trace.

[skekfab]

@hassiebp I am building an application based on Langchain to process user requests in an agentic fashion. For evaluation purposes, I need to get the Langfuse trace_id, which is required when creating scores in langfuse via langfuse.score(...). However, currently the documentation does not contain information about trace IDs regarding Langchain(Python v3). Are there any plans on upgrading the Langchain integration to v3 or is there a simple way to retrieve the automatically generated trace_id?

# In the API I somehow want to get the trace_id of the execution
config: RunnableConfig = {
    "callbacks": [CallbackHandler()]
}
input_user_query = {"messages": [{"role": "user", "content": user_query}]}
result = graph.invoke(input=input_user_query, config=config)

return MyApiResponse(
    result=result,
    trace_id='The trace_id produced by graph.invoke()'
)

[hassiebp]

@skekfab you may use something along these lines:

handler = CallbackHandler()

llm.invoke(messages, config={'callbacks': [handler]})
print(handler.last_trace_id) # -> stores the _last_ trace id observed by this handler; in concurrent environments, use a separate handler per invocation

[skekfab]

@hassiebp Thanks, the suggested solution works.

[bwinslow24]

@hassiebp From the documentation and this discussion #3901 it seems that we are suppose to reuse the callback handler for each trace. This is causing me issues since my application is processing many request asynchronously. This is leading to a race condition making the last_trace_id unreliable. I am currently just creating a new callback handler for each request and this solves the problem but I am running into the memory leak mentioned in the discussion. What is the proper way to get the trace id in my case?

[derevyan]

The V3 Python SDK is way too complicated, and using it with LangChain isn't fun at all anymore.

[hassiebp]

Thanks - we heard your feedback and have improvements for this on our immediate roadmap.

Could you please explain what exactly makes the v3 SDK more complicated for you?

[hassiebp]

@derevyan We have just released support for setting trace attributes in the Langchain chain invocation config metadata.

This is available from Python SDK version 3.0.2. See our docs for more details

Could you please upgrade and try again?

[derevyan]

Thanks a lot for taking action on it.
My thoughts - in my app main goal is to generate a response from an LLM. Sending metrics to Langfuse is a secondary, "fire-and-forget" task. If Langfuse is down or the metric fails to send application should keep works. i probably understand idea behind:
with langfuse.start_as_current_span(name="joke-chain") as span:
construction for more complex cases, but for simple request it to much for me and feels like im building my app around langfuse, also very well understand this point, just feels bad for me.
Approach:
langfuse_handler = CallbackHandler()
works just great for me in 99% of cases right now.
I really appreciate what you guys doing, great job, absolutely love to see how langfuse grows in this few years as company and project. But on other hand i really miss old simple and nice langfuse versions without all his features and stuff around, postgres and simple tracing all i need from this tool.

[hnykda]

I personally welcome the idea of going with OTEL, as we are migrating other stuff to OTEL as well, I appreciate you are trying to fit into existing ecosystem!

However, I struggled with integrating trace propagation from TS->Python.

[hassiebp]

We have plans to migrate our JS SDK to an OTEL backbone as well in the coming weeks 👍🏾

[hassiebp]

regarding the interop issue you faced, let's continue the discussion on the PR