get_openai_callback() does not show the cost taken for gpt4o-mini model

[AarthiAlagammai]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_community.callbacks import get_openai_callback 

llm=AzureChatOpenAI()
cb = get_openai_callback()

Description

get_openai_callback() does not show the cost taken for gpt4o-mini model

System Info

pip freeze|grep langchain

langchain==0.3.0

[dosubot[bot]]

Hey @AarthiAlagammai! I'm here to help you with any bugs, questions, or contributions you have regarding LangChain. Let's tackle this together!

The get_openai_callback() function does not show the cost for the gpt4o-mini model in LangChain version 0.3.0 because the model name is not correctly standardized in the standardize_model_name function. The standardize_model_name function does not account for the gpt-4o-mini model, which results in the model name not matching any key in the MODEL_COST_PER_1K_TOKENS dictionary.

Here is the relevant part of the standardize_model_name function:

def standardize_model_name(
    model_name: str,
    is_completion: bool = False,
) -> str:
    """
    Standardize the model name to a format that can be used in the OpenAI API.

    Args:
        model_name: Model name to standardize.
        is_completion: Whether the model is used for completion or not.
            Defaults to False.

    Returns:
        Standardized model name.

    """
    model_name = model_name.lower()
    if ".ft-" in model_name:
        model_name = model_name.split(".ft-")[0] + "-azure-finetuned"
    if ":ft-" in model_name:
        model_name = model_name.split(":")[0] + "-finetuned-legacy"
    if "ft:" in model_name:
        model_name = model_name.split(":")[1] + "-finetuned"
    if is_completion and (
        model_name.startswith("gpt-4")
        or model_name.startswith("gpt-3.5")
        or model_name.startswith("gpt-35")
        or model_name.startswith("o1-")
        or ("finetuned" in model_name and "legacy" not in model_name)
    ):
        return model_name + "-completion"
    else:
        return model_name

To resolve this issue, the standardize_model_name function needs to be updated to handle the gpt-4o-mini model correctly ^[1].

To continue talking to Dosu, mention @dosu.