callback to access http response headers

[deng232]

There is information like token remaining and modal name, etc. from the http response headers of the OpenAI http response. how to access that information in langchian? especially through callback?

[dosubot[bot]]

🤖

In LangChainJS, you can access information such as the token remaining and model name from the http response headers of the OpenAI http response using the OpenAILLMOutput interface and the tokenUsage object. The tokenUsage object contains the completionTokens, promptTokens, and totalTokens properties which can be used to track the token usage.

Here is an example of how you can access the token usage:

const data = await this.completionWithRetry(
  {
    ...params,
    stream: false,
    messages: messagesMapped,
  },
  {
    signal: options?.signal,
    ...options?.options,
  }
);
const {
  completion_tokens: completionTokens,
  prompt_tokens: promptTokens,
  total_tokens: totalTokens,
} = data?.usage ?? {};

if (completionTokens) {
  tokenUsage.completionTokens =
    (tokenUsage.completionTokens ?? 0) + completionTokens;
}

if (promptTokens) {
  tokenUsage.promptTokens = (tokenUsage.promptTokens ?? 0) + promptTokens;
}

if (totalTokens) {
  tokenUsage.totalTokens = (tokenUsage.totalTokens ?? 0) + totalTokens;
}

In the above code, completionWithRetry is a method that makes a request to the OpenAI API and returns the response. The usage property of the response contains the token usage information.

As for the model name, it is stored in the modelName property of the ChatOpenAI class. You can access it directly using this.modelName.

console.log(this.modelName); // prints the model name

If you want to use a callback to handle these values, you would need to pass a callback function to the method where these values are being calculated and call that callback with these values as arguments.

Please note that I wasn't able to find a built-in method or function within the LangChainJS framework to directly access HTTP response headers. If you need further assistance or if there's anything else you'd like to know, feel free to ask.

Sources

• langchain/src/chat_models/openai.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.