How to keep track of the openAI tokens used by a session?

[saikarthikp9]

As I understand, I can create a tool to make API calls, so I can make API calls to my custom backend to store token usage. But how can I get this info from the Conversational Retrieval QA Chain output?

[mattbisme]

I've been trying to figure out the same thing.

[amansoniamazatic]

@saikarthikp9 @mattbisme Use LangSmith or LLM Monitor API to keep track of all input and output token usage.

[chungyau97]

Hi all, we will need to make some enhancement for this to work.

Extracting the value of ChatId and overide the value of LangSmith/LangFuse Project Name to keep track on each ChatEmbed sessions.

I have added this to our backlog.