[Question]: Token Usage in agents

[mansiibm]

What is your question?

Hello, I had created an agent using Azure OpenAI , gpt-40-mini model. So, I am running my action using the following configuration -

openapi: 3.0.1
info:
title: Jira Issue API
description: API to retrieve Jira issue details
version: 1.0.0
servers:

• url: https:///rest/api/3
  paths:
  /issue/OS-663:
  get:
  summary: Get issue details
  description: Retrieve the details of a Jira issue by issue ID or key.
  parameters:
  - name: issueIdOrKey
  in: path
  required: true
  description: The ID or key of the issue.
  schema:
  type: string
  security:
  - basicAuth: []
  responses:
  '200':
  description: Successful response
  content:
  application/json:
  schema:
  type: object
  properties:
  expand:
  type: string
  id:
  type: string
  self:
  type: string
  format: uri
  key:
  type: string
  fields:
  type: object
  properties:
  # Example customization based on fields shown
  customfield_10556:
  type: object
  properties:
  cmdb:
  type: object
  properties:
  label:
  type: string
  objectKey:
  type: string
  customfield_10774:
  type: object
  properties:
  cmdb:
  type: object
  properties:
  label:
  type: string
  objectKey:
  type: string
  # Add more fields as needed
  '401':
  description: Unauthorized
  '404':
  description: Issue not found
  components:
  securitySchemes:
  basicAuth:
  type: http
  scheme: basic

So, in a fresh session , I ask the agent - get me issue details then the backend logs state-

2025-02-10T12:33:00.367725376Z {"completionTokens":29,"level":"debug","message":"[spendTokens] conversationId: a0efe278-220c-4b4d-a71f-0912a7cf84bf | Context: message | Token usage: ","promptTokens":58,"timestamp":"2025-02-10T12:33:00.367Z"}
2025-02-10T12:33:00.368683850Z {"completionTokens":345,"level":"debug","message":"[spendTokens] conversationId: a0efe278-220c-4b4d-a71f-0912a7cf84bf | Context: message | Token usage: ","promptTokens":24877,"timestamp":"2025-02-10T12:33:00.368Z"}

I cannot understand why two times the token usage is being shown.

Then when in the same chat session, I query for example- who is the reporter then it shows this-

2025-02-10T12:36:01.263283289Z {"completionTokens":51,"level":"debug","message":"[spendTokens] conversationId: a0efe278-220c-4b4d-a71f-0912a7cf84bf | Context: message | Token usage: ","promptTokens":25234,"timestamp":"2025-02-10T12:36:01.261Z"}

So, the questions are

1. Why such high token usage?
2. Why twice times the token usage is being shown for first query?
   Can it be optimised as I cannot spend these many token in a single session for a single user.

More Details

Steps to reproduce -

Create an agent with any endpoint
Add an action
Query using first prompt
Then check the logs, two usages are shown
And of high token value.

What is the main subject of your question?

Other

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[mansiibm]

Also I know using page format and body-format parameters I can narrow down but then the length of the function becomes invalid. Kindly help with the same, thanks in advance.

[danny-avila]

When using agents, multiple completions calls are made, and function definitions must be passed every time for the agent to intelligently delegate when/if tools should be used, that is the reason for high usage and multiple token transactions.

[mansiibm]

Hi Danny thankyou for replying , so using apis for actions not suggested ? What will be an optimised way to integrate apis or apps apart from mcp?

[danny-avila]

Why wouldn't actions be advised? In fact, if you can design your own API, which is simple to do with AI these days, it might be the best option. The key is to make the OpenAPI spec as simple as possible. It's a matter of use case and how well can you design an API for AI consumption.

[mansiibm]

Okay so like the spec can be optimised because apis I am using are standard, thanks for the input Danny :)

[mansiibm]

Hi Danny, I wanted to share few observations with you -

I have created two agents one with azure search tool and another with you tube tool. Used gpt-4o-mini from azure openai.

Tool	Observation
Azure Search Tool without file Search capability	My prompt includes- 11 tokens only as stated in logs .Complete logs for this single queryAs evident back to back 2 logs for completion tokens are being generated. Some 104150 tokens are going in prompt using the tool - azure search plugin.
Azure Search tool with file search capability	I begin with a fresh chat and query same thing as in above case ,I get only single output- single log of token usage and they have reduced to 209 prompt tokens.
You tube tool without file search	I again get 2 logs for tokens where final prompt tokens= 842
You tube tool with file search	Now the final prompt=1098

Based on above content-

1. Why two-three streams of the token's info get generated? Like at one time I used only you tube tool for the same agent as created and got this in the output.

2. How come using tools sometimes, over a lakh prompt tokens are being consumed? Shared above also.

3. If files are not involved and file search is enabled while using a tool, for the same query, then how is the token count variating?

4. What is the meaning of 2 logs of tokens? And which one represents actual consumption? Also this was tested and logged for a single user using librechat.