Trace and span execution duration not accurate in Langfuse

[hkrpydev-hkr]

Describe your question

Hi,
We have a self hosted Langfuse infrastructure (v 3.124.1) on AWS EKS, and have a workflow where we from GenAI applications, we are abstracting the Langfuse observe decorator and creating a custom Python SDK, that will capture instrumentation messages (traces and span) and send it to SQS queue and then have a python process to process them in batches (say 50 messages/spans) and send traces to Langfuse, so the load on Langfuse infra is manageable.

As we are not directly sending traces from GenAI application to Langfuse using the @observe decorator, the execution time (start clock + end clock when span is written) captured in Langfuse UI traces and spans are inaccurate and are factoring in whole workflow duration and not showing just the function execution time (assuming a function corresponds to a span).

Also, we are not able to update the execution time at trace or span level in the Langfuse UI with the accurate execution time of the function run that we are capturing in the metadata. Is there anyway we ca do this?

Also, if the traces/spans are not sent to Langfuse immediately using @observe decorator or say if Langfuse or it's DB components are down, we may end up with same scenario where execution time shown in nested traces/span in Langfuse UI may be incorrect. Anyway, we can overcome this scenario or have any workarounds to override this execution time based on metadata info?

Langfuse Cloud or Self-Hosted?

Self-Hosted

If Self-Hosted

3.124.1

If Langfuse Cloud

No response

SDK and integration versions

No response

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[inkeep-ai-assistant[bot]]

---

Your issue stems from bypassing the Langfuse SDK's built-in timing mechanisms by queuing and batching observations asynchronously through SQS(1).

The Core Problem

The Langfuse SDK uses synchronous timestamps to track accurate latency(1). When you abstract away the @observe decorator and queue messages through SQS before sending them to Langfuse, you're breaking this timing chain. The SDK can no longer capture the actual function execution time because observations are created long after the function has completed(1).

Why Timing is Inaccurate

The execution duration shown in Langfuse UI is calculated based on when observations are created and ended in the Langfuse system, not when your actual functions ran(1). Your batching workflow introduces delays between:

1. Function execution completion
2. Message queuing in SQS
3. Batch processing
4. Finally sending to Langfuse

This means the timestamps Langfuse receives don't reflect your actual function execution times.

Potential Solutions

1. Capture Timestamps at Function Execution Time

Since you're building a custom SDK abstraction, you need to capture the actual start and end timestamps during function execution (before queuing to SQS) and include them in your queued messages. When you later create observations in Langfuse, you'll need to use these captured timestamps.

However, the provided documentation doesn't show explicit methods to override start/end times when creating observations manually. The SDK's context managers and decorators handle timing automatically(2).

2. Consider Using Manual Observations with Explicit Timing

The SDK supports manual observation creation with start_observation()(2). While the documentation shows you must call .end() explicitly(2), it doesn't detail whether you can pass custom timestamps to override the automatic timing.

3. Architectural Recommendation

For accurate timing with your batching approach, you would need to:

• Capture precise start/end timestamps during actual function execution
• Store these timestamps in your SQS messages
• Find a way to apply these timestamps when creating observations in Langfuse

Unfortunately, the provided documentation doesn't explicitly cover how to set custom start/end times on observations, which is what you'd need for your use case.

Regarding Downtime Scenarios

You're correct that if Langfuse or its components are down and traces are sent later, timing could be affected(1). However, the SDK's design with fully async requests and background processing is meant to minimize latency impact(1).

Recommendation

I cannot find explicit documentation on overriding observation start/end times in the provided sources. Your best path forward would be to:

1. Check the Langfuse Python SDK API reference for timestamp parameters on observation creation methods
2. Contact Langfuse support directly about your batching use case and timestamp override capabilities
3. Consider whether the batching architecture is necessary, given that the SDK already uses "fully async requests" that add "almost no latency"(1)

The SDK is designed to handle high-volume tracing efficiently without manual batching(1), so you might reconsider whether your SQS-based batching approach is necessary.

---

**

📚 Sources:

1. Langfuse SDKs
2. Instrumentation

---

Have another question? Just tag @inkeep.

[Lotte-Verheyden]

Hi @hkrpydev-hkr, was Inkeep bot's answer able to help you further?

[hkrpydev-hkr]

Hi @Lotte-Verheyden : I need further information, will appreciate your inputs on this.

Regarding potential solutions mentioned by Inkeep bot, I had some follow up questions:

• Are there any guidelines on how to manually create observations in Langfuse (not using Langfuse SDK or @observe decorator) and pass the corresponding python function execution duration timestamps during observation creation process?
• In the event Langfuse infrastructure is down (self hosted open source version) when we are creating observations, we need a mechanism to store the traces and spans externally like in AWS S3 etc and then restore them in Langfuse when Langfuse is up back. So, in such events we need a mechanism to override the time duration of observations, traces, spans. How can we do that?

Thanks in advance!

[Lotte-Verheyden]

Hi @hkrpydev-hkr,

• you can manually create observations via the API (see reference: https://api.reference.langfuse.com/), there is also a related Github thread about setting custom timestamps here: https://github.com/orgs/langfuse/discussions/10381#discussioncomment-14948814
• you can also write your traces and spans to another location, this thread should be able to help you further: https://github.com/orgs/langfuse/discussions/9049#discussioncomment-14433066