-
Notifications
You must be signed in to change notification settings - Fork 1.7k
feat(ai): Live API #15309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
feat(ai): Live API #15309
Changes from 3 commits
Commits
Show all changes
126 commits
Select commit
Hold shift + click to select a range
441b4a7
[Firebase AI] Add starter types for Live API
andrewheard af76cbc
Add placeholder types for `LiveGenerativeModel` and `LiveSession`
andrewheard 1b5d791
Fix `BidiGenerateContentClientMessage` encoding
andrewheard da8c21b
Fix `BidiGenerateContentServerMessage` decoding
andrewheard aa3d148
Add `LiveGenerationConfig` and add to setup
andrewheard 60003c1
Add temporary state machine in `LiveSession`
andrewheard 3f285b8
Add `liveModel` static method to construct `LiveGenerativeModel`
andrewheard 737d823
Emit `responses` from `LiveSession`
andrewheard 91dd102
Temporarily display text Bidi responses in TestApp
andrewheard dd89e20
Refactor to use async/await and remove `URLSessionWebSocketDelegate`
andrewheard 962b57f
Add platform availability annotations
andrewheard 95e1908
Add `BidiGenerateContentServerMessage` availability annotation
andrewheard c6f3ba8
Add `AsyncWebSocket` wrapper for `URLSessionWebSocketTask`
andrewheard a9eb3d8
Update
daymxn b5b0107
Add support for SpeechConfig
daymxn 9b83676
Minor doc fixes
daymxn d5e375f
Add missing ID field
daymxn 543869e
Remove old todo comment
daymxn d342829
Add missing docs
daymxn 465006e
Add docs for app check extension
daymxn e51cc63
Add bug for session resumption
daymxn 3e58503
Use boolean to avoid warning
daymxn 55b8425
Log going away instead of error
daymxn fa4cf24
Merge branch 'main' into dlr/firebaseai-live-api
daymxn 0c23c7d
Add additional docs for tasks
daymxn 71acb55
Revert andrew's prototyping changes
daymxn 2ccca17
Add support for request options
daymxn 00ebe13
Minor doc fixes
daymxn 3c442a5
Revert "Analytics 12.3.0 (#15310)"
daymxn dba2bc0
Remove default from video realtime
daymxn 7902349
Add the close method
daymxn 24ef7f0
Add a todo about adding start
daymxn c252b0a
Fix missing id usage
daymxn b469e9c
Formatting
daymxn 12f9f27
Temporarily add support for function behavior
daymxn c4339fa
Add official support for transcripts
daymxn 9527026
Formatting
daymxn 48ce789
Add changelog entry
daymxn 240b94e
Add protoduration support
daymxn 5670c09
Remove unnecessary structs
daymxn a9f4961
Use unfair lock for closeError
daymxn e216c19
Use unfair lock for continuation finished
daymxn 1f545da
Make closeErorr immutable
daymxn 279b733
Merge branch 'main' into dlr/firebaseai-live-api
daymxn 6553b5a
Update some doc formatting
daymxn 8b32ed4
Fix duration compile error
daymxn 6678be2
Add docs to AsyncWebSocket
daymxn 58aab91
Remove realtime input config
daymxn 4c742d5
Add an error for lost connection
daymxn 2cd1544
Formatting
daymxn 4fce708
Remove function behavior stuff
daymxn dd81a84
Revert analytics changes from testing
daymxn a574eb4
Fix id decoding
daymxn 712b8b4
Rename id to functionId to avoid identifiable collision
daymxn 6ec278b
Fix available version to match usage
daymxn ad58739
Update reference to id field in docs
daymxn 0f4c55b
Update other doc references to id field
daymxn 6855efc
formatting
daymxn d1d878a
Fix old available version
daymxn e3fb8e8
Possibly fix concurrency issue
daymxn 866b32e
Allow params to be unnamed in realtime apis
daymxn 3b896bd
Add audio modifier to transcription names
daymxn 34f4275
Update FirebaseAI/Sources/Types/Internal/Live/LiveSessionService.swift
daymxn 30d9896
Update FirebaseAI/Sources/Types/Internal/Live/LiveSessionService.swift
daymxn 082ccbd
Update FirebaseAI/Sources/Types/Internal/Live/LiveSessionService.swift
daymxn 956942e
Update FirebaseAI/Sources/Types/Internal/Live/LiveSessionService.swift
daymxn 319ebc2
Update FirebaseAI/Sources/Types/Internal/Live/LiveSessionService.swift
daymxn 01f18f2
Add comments explaining nested task
daymxn 029ee0d
Use close instead on connect
daymxn 88fa653
Move live session error to public folder
daymxn 6332009
Use failable initializers
daymxn bba76c6
Formatting on comment
daymxn 7dacdc2
Update FirebaseAI/Sources/Types/Public/Live/LiveServerContent.swift
daymxn e8e6370
Update FirebaseAI/Sources/Types/Public/Live/LiveServerContent.swift
daymxn 57e2d40
Update docs for boolean value changes
daymxn ed06c34
Update FirebaseAI/Sources/Types/Public/Live/LiveServerContent.swift
daymxn 68472c4
Update docs for turn complete
daymxn 70c9f19
Update FirebaseAI/Sources/Types/Internal/Live/AsyncWebSocket.swift
daymxn d487279
Update FirebaseAI/Sources/Types/Internal/Live/BidiGenerateContentClie…
daymxn 474c646
Update FirebaseAI/Sources/Types/Internal/Live/BidiGenerateContentClie…
daymxn 29dfd73
Update FirebaseAI/Sources/Types/Internal/Live/BidiGenerateContentClie…
daymxn a7f5028
Update FirebaseAI/Sources/Types/Public/Live/LiveSessionErrors.swift
daymxn 2429404
Update FirebaseAI/Sources/Types/Public/Live/LiveSessionErrors.swift
daymxn 0939793
Update FirebaseAI/Sources/Types/Public/Live/LiveTranscription.swift
daymxn 407ee64
Update FirebaseAI/Sources/Types/Public/Live/SpeechConfig.swift
daymxn fe885aa
Update available versions to exclude watchos
daymxn 5977fe1
Add docs for audio response modality.
daymxn 6a6b2be
Link to refdocs instead
daymxn 9970582
Slight rewording + fix formatting on docs
daymxn 0ada63e
Add back encodable for client message
daymxn 814b4d4
Renaming GoAway to GoingAwayNotice for public api
daymxn 84adb36
lint
daymxn 166be93
Add workaround for `available(watchOS, unavailable)` being ignored
andrewheard 9228d9b
Add `available(watchOS, unavailable)` annotations to remaining types
andrewheard 140bfb8
Update FirebaseAI/Sources/FirebaseAI.swift
daymxn 5a1d836
Fix doc links + default to false for nullable bools
daymxn 86df993
Merge branch 'dlr/firebaseai-live-api' of github.com:firebase/firebas…
daymxn c5b1567
Rephrase live models in docs
daymxn 850914e
Migrate to protocol instead of enum
daymxn ba868a2
Update doc comments per rachel's input
daymxn 3788525
formatting
daymxn 31570e3
Update FirebaseAI/Sources/Types/Public/Live/LiveServerContent.swift
daymxn 12750b5
Merge branch 'dlr/firebaseai-live-api' of github.com:firebase/firebas…
daymxn feb838a
Update FirebaseAI/Sources/Types/Public/Live/LiveServerContent.swift
daymxn 91e13b8
Update FirebaseAI/Sources/Types/Public/Live/LiveServerContent.swift
daymxn c1de7cd
Update FirebaseAI/Sources/Types/Public/Live/LiveGenerationConfig.swift
daymxn ca3ebb9
Update FirebaseAI/Sources/Types/Public/Live/LiveGenerationConfig.swift
daymxn 6c999aa
Update FirebaseAI/Sources/Types/Public/Live/LiveGenerationConfig.swift
daymxn 6e80ac0
Update FirebaseAI/Sources/Types/Internal/Live/LiveSessionService.swift
daymxn 2b3e3c3
Only add underlying error if it's present
daymxn dc5942f
Merge branch 'dlr/firebaseai-live-api' of github.com:firebase/firebas…
daymxn 66ca3ca
minor comment fix
daymxn 74ea37d
Make sendVideo internal
daymxn adc35d2
Revert to enum
daymxn 27ce3fa
Rename live server message type
daymxn d2f48fd
Rename to LiveAudioTranscription
daymxn a589589
Rename function responses
daymxn a8c24ba
Minor doc formatting
daymxn 2ea027d
Log json payload if we don't have a mapping
daymxn 85f4c77
Surface setup task errors
daymxn 7d6b4de
Remove config suffix from speech param
daymxn 72c33d7
Use apiConfig endpoint
daymxn 339d5f3
Rename event to payload
daymxn dbcf303
Remove unnecessary conformance on AsyncWebSocket
daymxn 50aac8f
Add todo for location
daymxn 55cbf8d
Add hanging todo for service endpoints
daymxn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.