Skip to content

Added optional support for callbacks on streamed response #322 #423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/famous-areas-punch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@google/generative-ai": minor
---

Add streaming callbacks (onData, onEnd, onError) to improve developer experience when handling streamed AI responses.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
node_modules
dist/
temp/
*.log
*.tgz
.DS_Store
testfiles
Expand Down
4 changes: 2 additions & 2 deletions common/api-review/generative-ai-server.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

```ts

/// <reference types="node" />

// Warning: (ae-incompatible-release-tags) The symbol "ArraySchema" is marked as @public, but its signature references "BaseSchema" which is marked as @internal
//
// @public
Expand Down Expand Up @@ -31,8 +33,6 @@ export interface BooleanSchema extends BaseSchema {
type: typeof SchemaType.BOOLEAN;
}

/// <reference types="node" />

// @public
export interface CachedContent extends CachedContentBase {
createTime?: string;
Expand Down
18 changes: 16 additions & 2 deletions common/api-review/generative-ai.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,18 @@ export interface CachedContentBase {
tools?: Tool[];
}

// Warning: (ae-internal-missing-underscore) The name "CallbacksRequestOptions" should be prefixed with an underscore because the declaration is marked as @internal
//
// @internal
export interface CallbacksRequestOptions {
// (undocumented)
onData?: (chunks: string) => void;
// (undocumented)
onEnd?: (response: string) => void;
// (undocumented)
onError?: (error: Error) => void;
}

// @public
export class ChatSession {
constructor(apiKey: string, model: string, params?: StartChatParams, _requestOptions?: RequestOptions);
Expand All @@ -96,7 +108,8 @@ export class ChatSession {
// (undocumented)
params?: StartChatParams;
sendMessage(request: string | Array<string | Part>, requestOptions?: SingleRequestOptions): Promise<GenerateContentResult>;
sendMessageStream(request: string | Array<string | Part>, requestOptions?: SingleRequestOptions): Promise<GenerateContentStreamResult>;
// Warning: (ae-incompatible-release-tags) The symbol "sendMessageStream" is marked as @public, but its signature references "CallbacksRequestOptions" which is marked as @internal
sendMessageStream(request: string | Array<string | Part>, requestOptions?: SingleRequestOptions, callbacks?: CallbacksRequestOptions): Promise<GenerateContentStreamResult>;
}

// @public
Expand Down Expand Up @@ -525,7 +538,8 @@ export class GenerativeModel {
countTokens(request: CountTokensRequest | string | Array<string | Part>, requestOptions?: SingleRequestOptions): Promise<CountTokensResponse>;
embedContent(request: EmbedContentRequest | string | Array<string | Part>, requestOptions?: SingleRequestOptions): Promise<EmbedContentResponse>;
generateContent(request: GenerateContentRequest | string | Array<string | Part>, requestOptions?: SingleRequestOptions): Promise<GenerateContentResult>;
generateContentStream(request: GenerateContentRequest | string | Array<string | Part>, requestOptions?: SingleRequestOptions): Promise<GenerateContentStreamResult>;
// Warning: (ae-incompatible-release-tags) The symbol "generateContentStream" is marked as @public, but its signature references "CallbacksRequestOptions" which is marked as @internal
generateContentStream(request: GenerateContentRequest | string | Array<string | Part>, requestOptions: SingleRequestOptions, callbacks: CallbacksRequestOptions): Promise<GenerateContentStreamResult>;
// (undocumented)
generationConfig: GenerationConfig;
// (undocumented)
Expand Down
101 changes: 95 additions & 6 deletions docs/reference/main/generative-ai.arrayschema.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,99 @@ export interface ArraySchema extends BaseSchema

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [items](./generative-ai.arrayschema.items.md) | | [Schema](./generative-ai.schema.md) | A schema describing the entries in the array. |
| [maxItems?](./generative-ai.arrayschema.maxitems.md) | | number | _(Optional)_ The maximum number of items in the array. |
| [minItems?](./generative-ai.arrayschema.minitems.md) | | number | _(Optional)_ The minimum number of items in the array. |
| [type](./generative-ai.arrayschema.type.md) | | typeof [SchemaType.ARRAY](./generative-ai.schematype.md) | |
<table><thead><tr><th>

Property


</th><th>

Modifiers


</th><th>

Type


</th><th>

Description


</th></tr></thead>
<tbody><tr><td>

[items](./generative-ai.arrayschema.items.md)


</td><td>


</td><td>

[Schema](./generative-ai.schema.md)


</td><td>

A schema describing the entries in the array.


</td></tr>
<tr><td>

[maxItems?](./generative-ai.arrayschema.maxitems.md)


</td><td>


</td><td>

number


</td><td>

_(Optional)_ The maximum number of items in the array.


</td></tr>
<tr><td>

[minItems?](./generative-ai.arrayschema.minitems.md)


</td><td>


</td><td>

number


</td><td>

_(Optional)_ The minimum number of items in the array.


</td></tr>
<tr><td>

[type](./generative-ai.arrayschema.type.md)


</td><td>


</td><td>

typeof [SchemaType.ARRAY](./generative-ai.schematype.md)


</td><td>


</td></tr>
</tbody></table>
63 changes: 59 additions & 4 deletions docs/reference/main/generative-ai.baseparams.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,63 @@ export interface BaseParams

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [generationConfig?](./generative-ai.baseparams.generationconfig.md) | | [GenerationConfig](./generative-ai.generationconfig.md) | _(Optional)_ |
| [safetySettings?](./generative-ai.baseparams.safetysettings.md) | | [SafetySetting](./generative-ai.safetysetting.md)<!-- -->\[\] | _(Optional)_ |
<table><thead><tr><th>

Property


</th><th>

Modifiers


</th><th>

Type


</th><th>

Description


</th></tr></thead>
<tbody><tr><td>

[generationConfig?](./generative-ai.baseparams.generationconfig.md)


</td><td>


</td><td>

[GenerationConfig](./generative-ai.generationconfig.md)


</td><td>

_(Optional)_


</td></tr>
<tr><td>

[safetySettings?](./generative-ai.baseparams.safetysettings.md)


</td><td>


</td><td>

[SafetySetting](./generative-ai.safetysetting.md)<!-- -->\[\]


</td><td>

_(Optional)_


</td></tr>
</tbody></table>
41 changes: 38 additions & 3 deletions docs/reference/main/generative-ai.batchembedcontentsrequest.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,42 @@ export interface BatchEmbedContentsRequest

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [requests](./generative-ai.batchembedcontentsrequest.requests.md) | | [EmbedContentRequest](./generative-ai.embedcontentrequest.md)<!-- -->\[\] | |
<table><thead><tr><th>

Property


</th><th>

Modifiers


</th><th>

Type


</th><th>

Description


</th></tr></thead>
<tbody><tr><td>

[requests](./generative-ai.batchembedcontentsrequest.requests.md)


</td><td>


</td><td>

[EmbedContentRequest](./generative-ai.embedcontentrequest.md)<!-- -->\[\]


</td><td>


</td></tr>
</tbody></table>
41 changes: 38 additions & 3 deletions docs/reference/main/generative-ai.batchembedcontentsresponse.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,42 @@ export interface BatchEmbedContentsResponse

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [embeddings](./generative-ai.batchembedcontentsresponse.embeddings.md) | | [ContentEmbedding](./generative-ai.contentembedding.md)<!-- -->\[\] | |
<table><thead><tr><th>

Property


</th><th>

Modifiers


</th><th>

Type


</th><th>

Description


</th></tr></thead>
<tbody><tr><td>

[embeddings](./generative-ai.batchembedcontentsresponse.embeddings.md)


</td><td>


</td><td>

[ContentEmbedding](./generative-ai.contentembedding.md)<!-- -->\[\]


</td><td>


</td></tr>
</tbody></table>
Loading