Skip to content

AudioHandler callback to modify AudioTrack and AudioRecord #7496

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

AudioHandler callback to modify AudioTrack and AudioRecord #7496

wants to merge 6 commits into from

Conversation

@VinayGuthal
Copy link
Contributor

@VinayGuthal VinayGuthal commented Oct 21, 2025 •
edited
Loading

This PR adds a callback function audioHandler which would apply the necessary configurations or modifications to the AudioTrack and the AudioRecord objects used by the startAudioCoversation function.
This PR also adds a new configuration class called conversation config which could be sent to the startAudioConversation function which allows users to specify different aspects of the conversation.

@gemini-code-assist
Copy link
Contributor

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 21, 2025 •
edited
Loading

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

@google-oss-bot
Copy link
Contributor

1 Warning
⚠️ Did you forget to add a changelog entry? (Add the 'no-changelog' label to the PR to silence this warning.)

Generated by 🚫 Danger

@VinayGuthal VinayGuthal changed the title add the user handler AudioHandler callback to modify AudioTrack and AudioRecord Oct 21, 2025
@VinayGuthal VinayGuthal requested a review from rlazo October 21, 2025 18:27
@google-oss-bot
Copy link
Contributor

The public api surface has changed for the subproject firebase-ai:
error: Attempted to change parameter name from audioHandler to transcriptHandler in method com.google.firebase.ai.java.LiveSessionFutures.startAudioConversation [ParameterNameChange]
error: Removed method com.google.firebase.ai.java.LiveSessionFutures.startAudioConversation(kotlin.jvm.functions.Function1<? super com.google.firebase.ai.type.FunctionCallPart,com.google.firebase.ai.type.FunctionResponsePart>,kotlin.jvm.functions.Function2<? super android.media.AudioRecord,? super android.media.AudioTrack,kotlin.Unit>) [RemovedMethod]
error: Attempted to change parameter name from audioHandler to transcriptHandler in method com.google.firebase.ai.java.LiveSessionFutures.startAudioConversation [ParameterNameChange]
error: Removed method com.google.firebase.ai.java.LiveSessionFutures.startAudioConversation(kotlin.jvm.functions.Function2<? super com.google.firebase.ai.type.Transcription,? super com.google.firebase.ai.type.Transcription,kotlin.Unit>,kotlin.jvm.functions.Function2<? super android.media.AudioRecord,? super android.media.AudioTrack,kotlin.Unit>) [RemovedMethod]
error: Attempted to change parameter name from audioHandler to transcriptHandler in method com.google.firebase.ai.java.LiveSessionFutures.startAudioConversation [ParameterNameChange]
error: Removed method com.google.firebase.ai.java.LiveSessionFutures.startAudioConversation(kotlin.jvm.functions.Function1<? super com.google.firebase.ai.type.FunctionCallPart,com.google.firebase.ai.type.FunctionResponsePart>,kotlin.jvm.functions.Function2<? super com.google.firebase.ai.type.Transcription,? super com.google.firebase.ai.type.Transcription,kotlin.Unit>,kotlin.jvm.functions.Function2<? super android.media.AudioRecord,? super android.media.AudioTrack,kotlin.Unit>) [RemovedMethod]
error: Removed method com.google.firebase.ai.java.LiveSessionFutures.startAudioConversation(kotlin.jvm.functions.Function2<? super com.google.firebase.ai.type.Transcription,? super com.google.firebase.ai.type.Transcription,kotlin.Unit>,kotlin.jvm.functions.Function2<? super android.media.AudioRecord,? super android.media.AudioTrack,kotlin.Unit>,boolean) [RemovedMethod]
error: Removed method com.google.firebase.ai.java.LiveSessionFutures.startAudioConversation(kotlin.jvm.functions.Function1<? super com.google.firebase.ai.type.FunctionCallPart,com.google.firebase.ai.type.FunctionResponsePart>,kotlin.jvm.functions.Function2<? super com.google.firebase.ai.type.Transcription,? super com.google.firebase.ai.type.Transcription,kotlin.Unit>,kotlin.jvm.functions.Function2<? super android.media.AudioRecord,? super android.media.AudioTrack,kotlin.Unit>,boolean) [RemovedMethod]
error: Added class com.google.firebase.ai.type.ConversationConfig [AddedClass]
error: Added class com.google.firebase.ai.type.ConversationConfigKt [AddedClass]
error: Added method com.google.firebase.ai.type.LiveSession.startAudioConversation(com.google.firebase.ai.type.ConversationConfig,kotlin.coroutines.Continuation<? super kotlin.Unit>) [AddedMethod]
error: Removed method com.google.firebase.ai.type.LiveSession.startAudioConversation(kotlin.jvm.functions.Function1<? super com.google.firebase.ai.type.FunctionCallPart,com.google.firebase.ai.type.FunctionResponsePart>,kotlin.jvm.functions.Function2<? super com.google.firebase.ai.type.Transcription,? super com.google.firebase.ai.type.Transcription,kotlin.Unit>,kotlin.jvm.functions.Function2<? super android.media.AudioRecord,? super android.media.AudioTrack,kotlin.Unit>,boolean,kotlin.coroutines.Continuation<? super kotlin.Unit>) [RemovedMethod]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

@rlazo
Copy link
Collaborator

rlazo commented Oct 22, 2025

/gemini review

gemini-code-assist[bot]
gemini-code-assist bot reviewed Oct 22, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a ConversationConfig class to provide a more flexible way of configuring audio conversations. It also adds an audioHandler callback to allow custom modifications to AudioRecord and AudioTrack objects. The changes are well-structured, and the new ConversationConfig with its builder and Kotlin DSL is a great addition for API usability. I've made a few suggestions to improve immutability and code style.

rlazo
rlazo reviewed Oct 22, 2025
View reviewed changes
@google-oss-bot
Copy link
Contributor

google-oss-bot commented Oct 22, 2025 •
edited
Loading

Coverage Report 1

Affected Products

No changes between base commit (4194057) and merge commit (95fe5bb).

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/Mi9y1trunt.html

@VinayGuthal VinayGuthal requested a review from rlazo October 22, 2025 17:13
rlazo
rlazo requested changes Oct 22, 2025
View reviewed changes
firebase-ai/src/main/kotlin/com/google/firebase/ai/type/AudioHelper.kt
*/
@RequiresPermission(Manifest.permission.RECORD_AUDIO)
fun build(): AudioHelper {
fun build(audioHandler: ((AudioRecord, AudioTrack) -> Unit)? = null): AudioHelper {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update the kdoc to include new parameter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rlazo rlazo rlazo requested changes

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@VinayGuthal @google-oss-bot @rlazo