Skip to content

Conversation

@PratimMallick
Copy link
Contributor

@PratimMallick PratimMallick commented Nov 12, 2025

Goal

This PR adds support for capturing and transmitting screen audio during screen sharing on Android. Previously, when users shared their screen, only the video was transmitted. This change enables capturing system audio (e.g., from media playback, other apps) and mixing it with microphone audio before transmission.

Please note that the screen audio is not send if user mutes their microphone since the local audio track is unpublished when muting.

Implementation

Screen Audio Capture:

  • Added screen audio capture using AudioRecord with AudioPlaybackCaptureConfiguration (Android Q+)
  • Implemented on-demand audio reading directly from AudioRecord using READ_BLOCKING mode
  • Screen audio is read synchronously when requested by the WebRTC audio callback

Audio Mixing:

  • Created addAndConvertBuffers() utility function to mix microphone and screen audio buffers
  • Integrated mixing logic in StreamPeerConnectionFactory.setAudioBufferCallback():
    • Audio filters are applied to microphone audio first
    • Screen audio is then mixed with filtered microphone audio
    • When microphone is muted, screen audio is muted as well
    • When microphone is enabled, both are mixed together

Audio Track Management:

  • Added includeAudio parameter to Call.startScreenSharing() (defaults to false)
  • Added audioEnabled StateFlow to ScreenShareManager to track screen share audio state separately from video

Key Changes:

  • MediaManager.ScreenShareManager: Added screen audio capture infrastructure and audioEnabled StateFlow
  • StreamPeerConnectionFactory: Added audio mixing logic with proper filter application order
  • Demo app: Updated to pass includeAudio=true when starting screen sharing

🎨 UI Changes

None

Testing

On the demo app :

  1. Start screen sharing with includeAudio=true and verify system audio is captured and transmitted
  2. Mute microphone while screen sharing with audio enabled - verify screen audio also stops on the other side
  3. Unmute microphone while screen sharing - verify both microphone and screen audio are mixed and transmitted
  4. Join call with microphone muted, then start screen sharing with audio - verify audio track is not heard on the other side but once mic is unmuted it is transmitted

… audio screen sharing is present

2. If the user was muted while screen sharing, stopping screen sharing properly unpublishes the audio stream and sets the mute state.
@PratimMallick PratimMallick requested a review from a team as a code owner November 12, 2025 08:48
@PratimMallick PratimMallick added the pr:new-feature Adds new functionality label Nov 12, 2025
@PratimMallick PratimMallick marked this pull request as draft November 12, 2025 08:49
@github-actions
Copy link
Contributor

github-actions bot commented Nov 12, 2025

PR checklist ✅

All required conditions are satisfied:

  • Title length is OK (or ignored by label).
  • At least one pr: label exists.
  • Sections ### Goal, ### Implementation, and ### Testing are filled.

🎉 Great job! This PR is ready for review.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 12, 2025

SDK Size Comparison 📏

SDK Before After Difference Status
stream-video-android-core 11.93 MB 11.93 MB 0.00 MB 🟢
stream-video-android-ui-xml 5.68 MB 5.70 MB 0.02 MB 🟢
stream-video-android-ui-compose 6.27 MB 6.27 MB 0.00 MB 🟢

@PratimMallick PratimMallick changed the title Add possibility to share screen audio when starting screensharing Add possibility to share screen audio when screensharing is enabled Nov 12, 2025
@PratimMallick PratimMallick marked this pull request as ready for review November 12, 2025 13:54
Copy link
Contributor

@rahul-lohra rahul-lohra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@aleksandar-apostolov
Copy link
Contributor

We are not yet merging this, as there are some discussions around SFU changes.

@sonarqubecloud
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
1.1% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@aleksandar-apostolov aleksandar-apostolov merged commit 4409dae into develop Nov 27, 2025
18 of 21 checks passed
@aleksandar-apostolov aleksandar-apostolov deleted the audio_share branch November 27, 2025 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr:new-feature Adds new functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants