Live API v1 #9205

dlarocque · 2025-08-12T20:51:54Z

Adds support for the Public Preview Live API, with foundational methods to reach parity with Flutter, Android, and Unity.

API

LiveGenerativeModel class establishes a WebSocket connection and configures a LiveSession
LiveSession includes foundational methods for live content generation: receive(), send(), sendMediaChunks(), sendMediaStream()
Adds id property to FunctionCall and FunctionResponse, which is used by the Google AI backend to map function responses to calls. This property won't be in Vertex AI responses.

Testing

Unit tests: Uses a mock WebSocketHandler to simulate server messages.
Integration tests: Run against the live backend for both Vertex AI and Google AI. The function calling tests are very unreliable, so they will be skipped.

changeset-bot · 2025-08-12T20:51:58Z

⚠️ No Changeset found

Latest commit: 451c7b7

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

google-oss-bot · 2025-08-12T21:01:46Z

Size Report ¹

Affected Products

@firebase/ai
Type Base (2e03df7) Merge (b2f5347) Diff
browser 39.4 kB 47.2 kB +7.88 kB (+20.0%)
main 42.4 kB 50.6 kB +8.25 kB (+19.5%)
module 39.4 kB 47.2 kB +7.88 kB (+20.0%)
firebase
Type Base (2e03df7) Merge (b2f5347) Diff
firebase-ai.js 31.1 kB 37.3 kB +6.22 kB (+20.0%)

Test Logs

https://storage.googleapis.com/firebase-sdk-metric-reports/3NY4WIEmxX.html

google-oss-bot · 2025-08-12T21:13:42Z

Size Analysis Report ¹

Affected Products

`@firebase/ai`

AIError
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.40 kB 2.43 kB +32 B (+1.3%)
size-with-ext-deps 19.9 kB 20.0 kB +32 B (+0.2%)
AIErrorCode
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.40 kB 2.43 kB +32 B (+1.3%)
size-with-ext-deps 19.9 kB 20.0 kB +32 B (+0.2%)
AIModel
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 3.94 kB 3.97 kB +32 B (+0.8%)
size-with-ext-deps 21.5 kB 21.6 kB +32 B (+0.1%)
AnyOfSchema
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 4.72 kB 4.75 kB +32 B (+0.7%)
size-with-ext-deps 22.3 kB 22.3 kB +32 B (+0.1%)
ArraySchema
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 4.72 kB 4.75 kB +32 B (+0.7%)
size-with-ext-deps 22.3 kB 22.3 kB +32 B (+0.1%)
Backend
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.40 kB 2.43 kB +32 B (+1.3%)
size-with-ext-deps 19.9 kB 20.0 kB +32 B (+0.2%)
BackendType
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.40 kB 2.43 kB +32 B (+1.3%)
size-with-ext-deps 19.9 kB 20.0 kB +32 B (+0.2%)
BlockReason
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.50 kB 2.53 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.1 kB +32 B (+0.2%)
BooleanSchema
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 4.72 kB 4.75 kB +32 B (+0.7%)
size-with-ext-deps 22.3 kB 22.3 kB +32 B (+0.1%)

`ChatSession`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	16.9 kB	17.0 kB	+37 B (+0.2%)
size-with-ext-deps	34.6 kB	34.7 kB	+37 B (+0.1%)

Dependency

Type Base (2e03df7) Merge (b2f5347) Diff

variables

22 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
HarmSeverity
LANGUAGE_TAG
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
badFinishReasons
logger
name
responseLineRE
version

22 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_DOMAIN
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
HarmSeverity
LANGUAGE_TAG
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
badFinishReasons
logger
name
responseLineRE
version

+ DEFAULT_DOMAIN
- DEFAULT_BASE_URL

FinishReason
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.65 kB 2.68 kB +32 B (+1.2%)
size-with-ext-deps 20.2 kB 20.2 kB +32 B (+0.2%)
FunctionCallingMode
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.45 kB 2.48 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.0 kB +32 B (+0.2%)

`GenerativeModel`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	20.3 kB	20.3 kB	+37 B (+0.2%)
size-with-ext-deps	38.0 kB	38.1 kB	+37 B (+0.1%)

Dependency

Type Base (2e03df7) Merge (b2f5347) Diff

variables

22 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
HarmSeverity
LANGUAGE_TAG
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
badFinishReasons
logger
name
responseLineRE
version

22 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_DOMAIN
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
HarmSeverity
LANGUAGE_TAG
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
badFinishReasons
logger
name
responseLineRE
version

+ DEFAULT_DOMAIN
- DEFAULT_BASE_URL

GoogleAIBackend
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.40 kB 2.44 kB +32 B (+1.3%)
size-with-ext-deps 19.9 kB 20.0 kB +32 B (+0.2%)
HarmBlockMethod
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.46 kB 2.49 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.0 kB +32 B (+0.2%)
HarmBlockThreshold
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.58 kB 2.61 kB +32 B (+1.2%)
size-with-ext-deps 20.1 kB 20.1 kB +32 B (+0.2%)
HarmCategory
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.65 kB 2.68 kB +32 B (+1.2%)
size-with-ext-deps 20.2 kB 20.2 kB +32 B (+0.2%)
HarmProbability
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.48 kB 2.51 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.0 kB +32 B (+0.2%)
HarmSeverity
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.64 kB 2.67 kB +32 B (+1.2%)
size-with-ext-deps 20.2 kB 20.2 kB +32 B (+0.2%)
ImagenAspectRatio
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.51 kB 2.54 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.1 kB +32 B (+0.2%)
ImagenImageFormat
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.69 kB 2.72 kB +32 B (+1.2%)
size-with-ext-deps 20.2 kB 20.2 kB +32 B (+0.2%)

`ImagenModel`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	8.84 kB	8.88 kB	+37 B (+0.4%)
size-with-ext-deps	26.5 kB	26.5 kB	+37 B (+0.1%)

Dependency

Type Base (2e03df7) Merge (b2f5347) Diff

variables

13 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
LANGUAGE_TAG
PACKAGE_VERSION
Task
logger
name
version

13 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_DOMAIN
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
LANGUAGE_TAG
PACKAGE_VERSION
Task
logger
name
version

+ DEFAULT_DOMAIN
- DEFAULT_BASE_URL

ImagenPersonFilterLevel
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.49 kB 2.53 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.1 kB +32 B (+0.2%)
ImagenSafetyFilterLevel
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.57 kB 2.60 kB +32 B (+1.2%)
size-with-ext-deps 20.1 kB 20.1 kB +32 B (+0.2%)
IntegerSchema
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 4.72 kB 4.75 kB +32 B (+0.7%)
size-with-ext-deps 22.3 kB 22.3 kB +32 B (+0.1%)

`LiveGenerativeModel`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	?	8.39 kB	? (?)
size-with-ext-deps	?	26.0 kB	? (?)

Dependency

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
functions	?	assignRoleToPartsAndValidateSendMessageRequest decodeInstanceIdentifier formatNewContent formatSystemInstruction registerAI	?
classes	?	AIError AIModel AIService Backend GoogleAIBackend LiveGenerativeModel LiveSession VertexAIBackend WebSocketUrl	?
variables	?	AIErrorCode AI_TYPE BackendType DEFAULT_DOMAIN DEFAULT_LOCATION LiveResponseType Task logger name version	?
enums	?		?

External Dependency

Module	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
`@firebase/app`	?	_isFirebaseServerApp _registerComponent registerVersion	?
`@firebase/component`	?	Component	?
`@firebase/logger`	?	Logger	?
`@firebase/util`	?	FirebaseError	?

`LiveResponseType`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	?	2.55 kB	? (?)
size-with-ext-deps	?	20.1 kB	? (?)

Dependency

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
functions	?	decodeInstanceIdentifier registerAI	?
classes	?	AIError AIService Backend GoogleAIBackend VertexAIBackend	?
variables	?	AIErrorCode AI_TYPE BackendType DEFAULT_LOCATION LiveResponseType Task name version	?
enums	?		?

External Dependency

Module	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
`@firebase/app`	?	_registerComponent registerVersion	?
`@firebase/component`	?	Component	?
`@firebase/logger`	?	Logger	?
`@firebase/util`	?	FirebaseError	?

`LiveSession`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	?	4.92 kB	? (?)
size-with-ext-deps	?	22.5 kB	? (?)

Dependency

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
functions	?	assignRoleToPartsAndValidateSendMessageRequest decodeInstanceIdentifier formatNewContent registerAI	?
classes	?	AIError AIService Backend GoogleAIBackend LiveSession VertexAIBackend	?
variables	?	AIErrorCode AI_TYPE BackendType DEFAULT_LOCATION LiveResponseType Task logger name version	?
enums	?		?

External Dependency

Module	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
`@firebase/app`	?	_registerComponent registerVersion	?
`@firebase/component`	?	Component	?
`@firebase/logger`	?	Logger	?
`@firebase/util`	?	FirebaseError	?

Modality
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.53 kB 2.56 kB +32 B (+1.3%)
size-with-ext-deps 20.1 kB 20.1 kB +32 B (+0.2%)
NumberSchema
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 4.72 kB 4.75 kB +32 B (+0.7%)
size-with-ext-deps 22.3 kB 22.3 kB +32 B (+0.1%)
ObjectSchema
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 4.72 kB 4.75 kB +32 B (+0.7%)
size-with-ext-deps 22.3 kB 22.3 kB +32 B (+0.1%)
POSSIBLE_ROLES
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.45 kB 2.48 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.0 kB +32 B (+0.2%)
ResponseModality
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.44 kB 2.47 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.0 kB +32 B (+0.2%)
Schema
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 4.72 kB 4.75 kB +32 B (+0.7%)
size-with-ext-deps 22.3 kB 22.3 kB +32 B (+0.1%)
SchemaType
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.51 kB 2.54 kB +32 B (+1.3%)
size-with-ext-deps 20.0 kB 20.1 kB +32 B (+0.2%)
StringSchema
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 4.72 kB 4.75 kB +32 B (+0.7%)
size-with-ext-deps 22.3 kB 22.3 kB +32 B (+0.1%)
VertexAIBackend
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.40 kB 2.44 kB +32 B (+1.3%)
size-with-ext-deps 19.9 kB 20.0 kB +32 B (+0.2%)
getAI
Size
Type Base (2e03df7) Merge (b2f5347) Diff
size 2.76 kB 2.79 kB +32 B (+1.2%)
size-with-ext-deps 27.4 kB 27.5 kB +32 B (+0.1%)

`getGenerativeModel`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	20.4 kB	20.5 kB	+37 B (+0.2%)
size-with-ext-deps	38.2 kB	38.2 kB	+37 B (+0.1%)

Dependency

Type Base (2e03df7) Merge (b2f5347) Diff

variables

22 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
HarmSeverity
LANGUAGE_TAG
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
badFinishReasons
logger
name
responseLineRE
version

22 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_DOMAIN
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
HarmSeverity
LANGUAGE_TAG
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
badFinishReasons
logger
name
responseLineRE
version

+ DEFAULT_DOMAIN
- DEFAULT_BASE_URL

`getImagenModel`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	9.00 kB	9.04 kB	+37 B (+0.4%)
size-with-ext-deps	26.7 kB	26.7 kB	+37 B (+0.1%)

Dependency

Type Base (2e03df7) Merge (b2f5347) Diff

variables

13 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
LANGUAGE_TAG
PACKAGE_VERSION
Task
logger
name
version

13 dependencies

AIErrorCode
AI_TYPE
BackendType
DEFAULT_API_VERSION
DEFAULT_DOMAIN
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
LANGUAGE_TAG
PACKAGE_VERSION
Task
logger
name
version

+ DEFAULT_DOMAIN
- DEFAULT_BASE_URL

`getLiveGenerativeModel`

Size

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
size	?	10.8 kB	? (?)
size-with-ext-deps	?	28.5 kB	? (?)

Dependency

Type	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
functions	?	assignRoleToPartsAndValidateSendMessageRequest createWebSocketHandler decodeInstanceIdentifier formatNewContent formatSystemInstruction getLiveGenerativeModel registerAI	?
classes	?	AIError AIModel AIService Backend BrowserWebSocketHandler GoogleAIBackend LiveGenerativeModel LiveSession VertexAIBackend WebSocketUrl	?
variables	?	AIErrorCode AI_TYPE BackendType DEFAULT_DOMAIN DEFAULT_LOCATION LiveResponseType Task logger name version	?
enums	?		?

External Dependency

Module	Base (`2e03df7`)	Merge (`b2f5347`)	Diff
`@firebase/app`	?	_isFirebaseServerApp _registerComponent registerVersion	?
`@firebase/component`	?	Component	?
`@firebase/logger`	?	Logger	?
`@firebase/util`	?	FirebaseError	?

Test Logs

https://storage.googleapis.com/firebase-sdk-metric-reports/uQxoJLmKv3.html

Live API v1

5235782

update docs

0277497

dlarocque added 5 commits August 13, 2025 09:41

Docs and naming changes

683f696

Skip problematic Google AI tests

080342a

Fix mediaChunks errors

5252a40

Log warnings for 'close' event reasons

79da676

Add note about support to getLiveGenerativeModel docs

451c7b7

dlarocque marked this pull request as ready for review August 13, 2025 16:29

dlarocque requested review from a team as code owners August 13, 2025 16:29

dlarocque requested a review from hsubox76 August 13, 2025 16:31

Live API v1 #9205

Are you sure you want to change the base?

Live API v1 #9205

Conversation

dlarocque commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

API

Testing

Uh oh!

changeset-bot bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

google-oss-bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Size Report 1

Affected Products

@firebase/ai

firebase

Test Logs

Uh oh!

google-oss-bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Size Analysis Report 1

Affected Products

@firebase/ai

AIError

AIErrorCode

AIModel

AnyOfSchema

ArraySchema

Backend

BackendType

BlockReason

BooleanSchema

ChatSession

FinishReason

FunctionCallingMode

GenerativeModel

GoogleAIBackend

HarmBlockMethod

HarmBlockThreshold

HarmCategory

HarmProbability

HarmSeverity

ImagenAspectRatio

ImagenImageFormat

ImagenModel

ImagenPersonFilterLevel

ImagenSafetyFilterLevel

IntegerSchema

LiveGenerativeModel

LiveResponseType

LiveSession

Modality

NumberSchema

ObjectSchema

POSSIBLE_ROLES

ResponseModality

Schema

SchemaType

StringSchema

VertexAIBackend

getAI

getGenerativeModel

getImagenModel

getLiveGenerativeModel

Test Logs

Uh oh!

Uh oh!

dlarocque commented Aug 12, 2025 •

edited

Loading

changeset-bot bot commented Aug 12, 2025 •

edited

Loading

google-oss-bot commented Aug 12, 2025 •

edited

Loading

Size Report ¹

`@firebase/ai`

`firebase`

google-oss-bot commented Aug 12, 2025 •

edited

Loading

Size Analysis Report ¹

`@firebase/ai`

`AIError`

`AIErrorCode`

`AIModel`

`AnyOfSchema`

`ArraySchema`

`Backend`

`BackendType`

`BlockReason`

`BooleanSchema`

`ChatSession`

`FinishReason`

`FunctionCallingMode`

`GenerativeModel`

`GoogleAIBackend`

`HarmBlockMethod`

`HarmBlockThreshold`

`HarmCategory`

`HarmProbability`

`HarmSeverity`

`ImagenAspectRatio`

`ImagenImageFormat`

`ImagenModel`

`ImagenPersonFilterLevel`

`ImagenSafetyFilterLevel`

`IntegerSchema`

`LiveGenerativeModel`

`LiveResponseType`

`LiveSession`

`Modality`

`NumberSchema`

`ObjectSchema`

`POSSIBLE_ROLES`

`ResponseModality`

`Schema`

`SchemaType`

`StringSchema`

`VertexAIBackend`

`getAI`

`getGenerativeModel`

`getImagenModel`

`getLiveGenerativeModel`