Skip to content

Handling non-OpenAI APIs #490

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Handling non-OpenAI APIs #490

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
aef49c3
Non-OpenAI APIs that implement a simmilar interface don't always incl…
annie444 Nov 21, 2025
2e58310
The LM Studio API doesn't always output token information
annie444 Nov 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions async-openai/src/types/responses/response.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1522,6 +1522,7 @@ pub struct ResponseLogProb {
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
pub struct OutputTextContent {
/// The annotations of the text output.
#[serde(default)]
pub annotations: Vec<Annotation>,
pub logprobs: Option<Vec<LogProb>>,
/// The text output from the model.
Expand Down Expand Up @@ -2431,21 +2432,22 @@ pub struct MCPApprovalRequest {
pub server_label: String,
}

#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Default)]
pub struct InputTokenDetails {
/// The number of tokens that were retrieved from the cache.
/// [More on prompt caching](https://platform.openai.com/docs/guides/prompt-caching).
pub cached_tokens: u32,
}

#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Default)]
pub struct OutputTokenDetails {
/// The number of reasoning tokens.
pub reasoning_tokens: u32,
}

/// Usage statistics for a response.
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Default)]
#[serde(default)]
pub struct ResponseUsage {
/// The number of input tokens.
pub input_tokens: u32,
Expand Down