Skip to content

[Firebase AI] Add thought summary and signature support #9192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

[Firebase AI] Add thought summary and signature support #9192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0f106fc
[Firebase AI] Add thought summary and signature support
andrewheard Aug 1, 2025
e8df63d
Run formatter
andrewheard Aug 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions common/api-review/ai.api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,8 @@ export interface EnhancedGenerateContentResponse extends GenerateContentResponse
functionCalls: () => FunctionCall[] | undefined;
inlineDataParts: () => InlineDataPart[] | undefined;
text: () => string;
// (undocumented)
thoughtSummary: () => string | undefined;
}

// @public
Expand Down Expand Up @@ -240,6 +242,10 @@ export interface FileDataPart {
inlineData?: never;
// (undocumented)
text?: never;
// (undocumented)
thought?: boolean;
// (undocumented)
thoughtSignature?: string;
}

// @public
Expand Down Expand Up @@ -294,6 +300,10 @@ export interface FunctionCallPart {
inlineData?: never;
// (undocumented)
text?: never;
// (undocumented)
thought?: boolean;
// (undocumented)
thoughtSignature?: string;
}

// @public
Expand Down Expand Up @@ -326,6 +336,10 @@ export interface FunctionResponsePart {
inlineData?: never;
// (undocumented)
text?: never;
// (undocumented)
thought?: boolean;
// (undocumented)
thoughtSignature?: string;
}

// @public
Expand Down Expand Up @@ -691,6 +705,10 @@ export interface InlineDataPart {
inlineData: GenerativeContentBlob;
// (undocumented)
text?: never;
// (undocumented)
thought?: boolean;
// (undocumented)
thoughtSignature?: string;
videoMetadata?: VideoMetadata;
}

Expand Down Expand Up @@ -957,10 +975,15 @@ export interface TextPart {
inlineData?: never;
// (undocumented)
text: string;
// (undocumented)
thought?: boolean;
// (undocumented)
thoughtSignature?: string;
}

// @public
export interface ThinkingConfig {
includeThoughts?: boolean;
thinkingBudget?: number;
}

Expand Down
17 changes: 13 additions & 4 deletions packages/ai/src/requests/response-helpers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ export function addHelpers(
}
);
}
return getText(response);
return getText(response, false);
} else if (response.promptFeedback) {
throw new AIError(
AIErrorCode.RESPONSE_ERROR,
Expand Down Expand Up @@ -160,13 +160,22 @@ export function addHelpers(
}

/**
* Returns all text found in all parts of first candidate.
* Returns all text from the first candidate's parts, filtering by whether they
* are part of the model's 'thought' process.
*
* @param response - The {@link GenerateContentResponse} from which to extract text.
* @param isThought - If `true`, extracts text from `thought` parts of the response,
* which represent the model's internal reasoning. If `false`, extracts text from
* regular response parts.
*/
export function getText(response: GenerateContentResponse): string {
export function getText(
response: GenerateContentResponse,
isThought: boolean
Copy link
Contributor

@hsubox76 hsubox76 Aug 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe instead of passing a specific boolean for thoughts we pass through a filter function here:

partFilter: () => boolean

and then when we call it in addHelpers() above it's like getText(response, (part) => !part.thought) for text() and it's getText(response, (part) => part.thought) for thoughtSummary(). And then in the condition below it's if (part.text && partFilter(part))

This makes it read clearly when it's called (or we can just add a comment before true and false). It might get unwieldy if we have other kinds of text parts in the future that have more complex logic though.

): string {
const textStrings = [];
if (response.candidates?.[0].content?.parts) {
for (const part of response.candidates?.[0].content?.parts) {
if (part.text) {
if (part.text && (part.thought ?? false) === isThought) {
textStrings.push(part.text);
}
}
Expand Down
10 changes: 10 additions & 0 deletions packages/ai/src/types/content.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ export interface TextPart {
inlineData?: never;
functionCall?: never;
functionResponse?: never;
thought?: boolean;
thoughtSignature?: string;
}

/**
Expand All @@ -62,6 +64,8 @@ export interface InlineDataPart {
* Applicable if `inlineData` is a video.
*/
videoMetadata?: VideoMetadata;
thought?: boolean;
thoughtSignature?: string;
}

/**
Expand Down Expand Up @@ -90,6 +94,8 @@ export interface FunctionCallPart {
inlineData?: never;
functionCall: FunctionCall;
functionResponse?: never;
thought?: boolean;
thoughtSignature?: string;
}

/**
Expand All @@ -101,6 +107,8 @@ export interface FunctionResponsePart {
inlineData?: never;
functionCall?: never;
functionResponse: FunctionResponse;
thought?: boolean;
thoughtSignature?: string;
}

/**
Expand All @@ -113,6 +121,8 @@ export interface FileDataPart {
functionCall?: never;
functionResponse?: never;
fileData: FileData;
thought?: boolean;
thoughtSignature?: string;
}

/**
Expand Down
9 changes: 9 additions & 0 deletions packages/ai/src/types/requests.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -294,4 +294,13 @@ export interface ThinkingConfig {
* feature or if the specified budget is not within the model's supported range.
*/
thinkingBudget?: number;

/**
* Whether to include "thought summaries" in the model's response.
*
* Thought summaries provide a brief overview of the model's internal thinking process,
* offering insight into how it arrived at the final answer. This can be useful for
* debugging, understanding the model's reasoning, and verifying its accuracy.
*/
includeThoughts?: boolean;
}
1 change: 1 addition & 0 deletions packages/ai/src/types/responses.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ export interface EnhancedGenerateContentResponse
*/
inlineDataParts: () => InlineDataPart[] | undefined;
functionCalls: () => FunctionCall[] | undefined;
thoughtSummary: () => string | undefined;
}

/**
Expand Down
Loading