Tracking LLM usage

[sky-2002]

Hi haystack team,
I have been using haystack for my workflows and I have components that make LLM API calls.
I wanted to ask how do I add tracking for these components so that I can see usage for every run.
Please point me to any resources or docs if any.

[sky-2002]

Also is there a way to cache component outputs, for use cases like starting from earlier state if some component fails etc, would love to know.

@julian-risch

[sjrl]

Hey @sky-2002 glad to hear you've been using Haystack! I'd recommend checking out our different tracing options like Langfuse. Depending on the LLM components that you are using you should see token tracking and cost tracking within the langfuse dashboard.

[sjrl]

For starting from an earlier state we do have a pipeline breakpoints feature coming soon to Haystack. You can find an experimental implementation of it in our haystack-experimental repo. Here is the discussion on it and here is a google colab for it.

[sky-2002]

Hey @SRJL
Thanks for the info, will check the options and breakpoints feature, thanks for the links and resources