-
Notifications
You must be signed in to change notification settings - Fork 97
Gemini Live sample #53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summary of Changes
Hello @lethargicpanda, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces a new, fully functional sample application that demonstrates the capabilities of the Gemini Live API within an Android environment. The sample, a voice-controlled Todo list, serves as a practical example of how to integrate real-time conversational AI for interactive user experiences, expanding the existing AI catalog with a cutting-edge feature.
Highlights
- New Sample Application: I've added a new sample application named
gemini-live-todo
to showcase the integration of the Gemini Live API. This sample implements a simple Todo list application that can be interacted with via voice commands. - Gemini Live API Integration: The new Todo sample demonstrates how to initialize a Gemini Live session, define custom tools (functions like
addTodo
,removeTodo
,toggleTodoStatus
,getTodoList
), and handle function calls from the Gemini model to manage the Todo list state. It also configures speech output and system instructions for the model. - Android Compose UI: The sample features a modern Android UI built with Jetpack Compose, including a
TodoScreen
that displays the list of tasks and allows for manual input or voice-driven interaction via a floating action button. - Permissions and Dependencies: The application now requests the
RECORD_AUDIO
permission, which is essential for the Gemini Live API's voice input capabilities. I've also updatedlibs.versions.toml
and the app'sbuild.gradle.kts
to include necessary dependencies for Firebase AI, Compose Material3, and Hilt.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command>
or @gemini-code-assist <command>
. Below is a summary of the supported commands.
Feature | Command | Description |
---|---|---|
Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/
folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
The pull request introduces the 'Gemini Live Todo' sample, showcasing voice interaction with a to-do list. Review comments suggest improvements in ID generation, package names, state management, null handling, Java versioning, accessibility, and code clarity.
...mini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreenViewModel.kt
Show resolved
Hide resolved
...mini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreenViewModel.kt
Show resolved
Hide resolved
...og/samples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/data/Todo.kt
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Show resolved
Hide resolved
...mini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreenViewModel.kt
Outdated
Show resolved
Hide resolved
...mini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreenViewModel.kt
Outdated
Show resolved
Hide resolved
...mini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreenViewModel.kt
Outdated
Show resolved
Hide resolved
...og/samples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/data/Todo.kt
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Outdated
Show resolved
Hide resolved
...amples/gemini-live-todo/src/main/java/com/android/ai/samples/geminilivetodo/ui/TodoScreen.kt
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, let's add tests in future as well
viewModelScope.launch { | ||
Log.d(TAG, "Start Gemini Live initialization") | ||
val liveGenerationConfig = liveGenerationConfig { | ||
speechConfig = SpeechConfig(voice = Voice("FENRIR")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Would be great to document options for Voice as well.
Add UiState.Error to manage Live Sesssion connection error
a18189d
to
32712ff
Compare
} | ||
|
||
@Composable | ||
fun TodoInput(text: String, onTextChange: (String) -> Unit, onAddClick: () -> Unit) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Every composable function except top level screen composables should take a modifier parameter
} | ||
|
||
@SuppressLint("MissingPermission") | ||
fun toggleLiveSession(activity: Activity) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You shouldn't ever pass an Activity into the view model because it causes memory leaks.
Permissions requests are done in the UI layer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I opened a separate PR to address this one and the other comment:
#63
} | ||
is TodoScreenUiState.Success -> { | ||
val todos = (uiState as TodoScreenUiState.Success).todos | ||
LazyColumn(modifier = Modifier.fillMaxSize()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just noticed that this section is completely copy pasted, you should extract it. But also, I don't think you even need the higher level UIState error, you never use it and you have the error in the LiveSessionState
data object NotReady : LiveSessionState | ||
data object Ready : LiveSessionState | ||
data object Running : LiveSessionState | ||
data object Error : LiveSessionState |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add an error message here
val liveSessionState: LiveSessionState, | ||
) : TodoScreenUiState | ||
|
||
data class Error( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above, I don't think you need this.
Add new sample showcasing the Gemini Live API used to support a Todo app.