How to fetch the output and input tokens of a trace via API?

[fdb02983rhy]

Can I do that via API without a trace id? I can only get the total tokens by langfuse.fetch_traces().

[dosubot[bot]]

To fetch detailed trace information, including input and output tokens, you can use the following API endpoints provided by Langfuse:

• GET /api/public/traces: This endpoint allows you to fetch a list of traces.
• GET /api/public/traces/:traceId: This endpoint allows you to fetch a single trace by its ID, which may include detailed information such as input and output tokens ^[1].

If you're using the Langfuse Python SDK, you can log input and output tokens using the langfuse_context.update_current_observation method. Here's an example:

langfuse_context.update_current_observation(
    usage={
        "input": response.usage.input_tokens,
        "output": response.usage.output_tokens
    }
)

This method captures and logs the input and output tokens of a trace ^[2].

To continue talking to Dosu, mention @dosu.

[fdb02983rhy]

Thank you for the responses! Just to confirm, if I want to retrieve the output tokens for all traces, would the correct approach be to: (1) fetch all traces, and (2) use the trace IDs to obtain the output tokens?
It will be great if I can do those in one single API.

[marcklingen]

Do you just need the sum? If yes, I would go for the metrics api: https://langfuse.com/docs/analytics/daily-metrics-api

[fdb02983rhy]

I'm working on analyzing the performance of an API provider by looking at their response times. Specifically, I'd like to identify the slowest or most 'blocking' requests every day. I'm calculating this by looking at the output tokens divided by (initial latency + generation time since first token received), aka the lowest output tokens/sec. Do you have any good ideas fetching the necessary data from Langfuse via API? Any advices will be appreciated!

[marcklingen]

It makes sense that this is "expensive" through the API, as you will need to fetch all traces and observations. Some users replicate Langfuse data to their data warehouse (e.g., via Airbyte) to run these types of queries. Is this an option for you?

Through the Langfuse interface, you can sort the generations table by the tokens per second column. Is this helpful for your use case?

[fdb02983rhy]

Thanks, I will try!