[Question] A huge mismatch in cost calculation between AI vendors dashboards and LibreChat's DB

[igodsewer]

Hi! I'm running LibreChat in Docker, version is 0.7.8.

I'm running LibreChat in a corporate environment with monthly AI budget around 5k$. As it's pretty big number, I was curious to make up a per-user cost allocation, and I discovered a huge mismatch between what AI vendors display in their billing panels and what is stored in LibreChat's database.

As per these docs, and Danny's explanation here, I see that I should simply divide tokenValue by 1000000 to get amount of $$ spent on certain transaction.

So, for testing purposes, I came up with a simple MongoDB aggregation presented below:

  {
    $match: {
      updatedAt: {
        $gt: ISODate("2025-05-01T00:00:00Z") // I only want data for current month - May
      }
    }
  },
  {
    $group: {
      _id: null,
      totalCost: {
        $sum: "$tokenValue"
      }
    }
  }
]

This aggregation gave me 5786033683 tokenValue, which translates to roughly 5786$ after applying this value to Danny's formula. However, my OpenAI and Google Cloud Platform dashboards shows 4165$ and 28$ (respectively) expenditures in May, which is almost 1500$ off of what is calculated based on values in the database. (GCP value might seem strange, but this is a true story - in our company Google's models are not used very often)

After some research I've found out that raw tokens numbers stored in LibreChat's database sometimes are different from what OpenAI's Tokenizer provides. For example, I've prompted OpenAI's GPT-4o with such message:

Hi! How are you?

Which was translated to 13 tokens, as per my LibreChat's "messages" and "transactions" collections. However, OpenAI's Tokenizer translated this into just 6 tokens.

So, my question is - what am I doing wrong? I suppose, there might be some more complicated logic or I'm doing math with wrong values, but I don't know where to look. Any assistance is appreciated. Thanks!

[danny-avila]

Token values are estimates and will never be 100% accurate and lean conservative, as you have found.

• The prices are static, and they may update in the real world before we can define new values
• Cached tokens will make OpenAI calculations more off than not, since this happens implicitly and we are not currently handling this

[igodsewer]

Hi!
Thanks for the quick reply. So, currently, there is no "good" way of properly calculating tokens and their cost?

Thanks,

[danny-avila]

I would argue what we are doing is already "good" since we first do our own calculation (likely off) and then replace it with API usage metadata if any is received (which is not always).

It can be better, especially resolving my second point of handling cached pricing for OpenAI specifically.