Conversation
|
Windows and Mac build successful in Unity Cloud! You can find a link to the downloadable artifact below. |
cyaiox
force-pushed
the
feat/enhance-livekit-track-selector
branch
from
4609f6b to
46fbe57
Compare
cyaiox
added
force-build
cyaiox
force-pushed
the
feat/enhance-livekit-track-selector
branch
from
46fbe57 to
dd905a9
Compare
cyaiox
added
force-build
|
@claude review |
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
|
Tested! Functionality works as expected ✅ 
7347-evi.mp4
None of these block the functionality, just leaving them on record! 11.03.2026_11.08.39_REC.mp4
|
cyaiox
force-pushed
the
feat/enhance-livekit-track-selector
branch
from
390bb2e to
66e96cc
Compare
cyaiox
added
force-build
…r livekit user streams Add participant metadata (name, speaking, trackName, width, height) to GetActiveVideoStreamsResponse JSON output. Enable automatic audio track resolution in LivekitPlayer when opening a user stream by pairing video source to the correct audio source (camera→microphone, screenshare→screenshare audio).
Parse displayName from participant.Metadata JSON (used by unauthenticated castV2 users), falling back to participant.Name then participant.Identity.
When a specific user stream track (livekit-video://identity/sid) is unpublished, fallback to current-stream instead of retrying the same dead address every frame. This lets EnsureVideoIsPlaying and EnsureAudioIsPlaying self-heal by scanning for any available track.
Instead of pairing one audio track to the selected video track, LivekitPlayer now plays every audio track in the room — so all participants are heard regardless of which video is displayed.
When LivekitPlayer plays in CurrentStream mode, the video now follows the dominant speaker using room.ActiveSpeakers with a 1.5s debounce to prevent flickering. UserStream mode remains pinned to the specified participant. Also adds docs/cast.md documenting the full cast feature.
cyaiox
force-pushed
the
feat/enhance-livekit-track-selector
branch
from
43e595a to
ccdc25a
Compare
cyaiox
added
force-build
|
@claude review |
|
Claude finished @m3taphysics's task in 2m 53s —— View job Code Review
Issues Found🔴 Performance —
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| } | ||
| } | ||
|
|
||
| if (!anyDied && audioSources.Count > 0) return; |
There was a problem hiding this comment.
Performance bug — participants lock acquired every frame when room has no audio
When audioSources.Count == 0 (e.g. room has no audio tracks yet, or after Stop()), anyDied stays false, so !anyDied && audioSources.Count > 0 evaluates to false and the method always falls through to ReleaseAllAudioSources() + OpenAllAudioStreams(). OpenAllAudioStreams acquires lock(room.Participants) and iterates all participants — every frame, for the lifetime of the player when no audio is present.
Fix: track a separate flag for "attempted to open audio, found nothing".
| if (!anyDied && audioSources.Count > 0) return; | |
| if (!anyDied && audioSources.Count > 0) return; | |
| if (audioSources.Count == 0 && !anyDied) | |
| { | |
| // No sources and nothing died — room has no audio tracks. | |
| // Only re-check when the frame budget allows (or when first opened). | |
| OpenAllAudioStreams(); | |
| return; | |
| } |
Simpler fix: add an audioInitialized bool that is set to true in OpenMedia/OpenAllAudioStreams, and only skip the redundant re-poll when audioInitialized && audioSources.Count == 0.
| /// Finds the audio track paired to a specific video track from the same participant. | ||
| /// Available for future targeted audio scenarios. | ||
| /// </summary> | ||
| private Weak<AudioStream> FindPairedAudio(string identity, string videoSid) |
There was a problem hiding this comment.
Dead code — FindPairedAudio is never called
This private method has no callers. Per the project's code standards: "Don't create helpers, utilities, or abstractions for one-time operations. Don't design for hypothetical future requirements."
The docs note it's reserved for "future targeted audio scenarios", but shipping dead code adds maintenance burden and cognitive noise. Remove it until it's actually needed.
There was a problem hiding this comment.
I kept it in case it will be required in the future, as livekit sends Video and Audio in separate tracks, this utility helps to match the participant Video with their respective Audio.
| // If a specific user stream died, fallback to current-stream (first available track) | ||
| if (playingAddress.Value.IsUserStream(out _)) | ||
| { | ||
| OpenMedia(LivekitAddress.CurrentStream()); |
There was a problem hiding this comment.
Audio disruption on every video fallback
OpenMedia(LivekitAddress.CurrentStream()) calls CloseCurrentStream() first, which calls ReleaseAllAudioSources() — releasing and re-pooling all audio MonoBehaviour objects — then immediately re-acquires them in OpenAllAudioStreams(). This causes a brief audio dropout on every video recovery cycle, even when the audio streams are perfectly healthy.
Consider splitting the video and audio reset paths so a dying video track doesn't interrupt audio that is still alive:
| OpenMedia(LivekitAddress.CurrentStream()); | |
| // Video stream died — swap to first available track without resetting audio | |
| if (playingAddress.Value.IsUserStream(out _)) | |
| { | |
| currentVideoIdentity = null; | |
| currentVideoStream = FirstVideoTrackingIdentity(); | |
| playingAddress = LivekitAddress.CurrentStream(); | |
| videoSwitchedAtTime = UnityEngine.Time.realtimeSinceStartup; | |
| playerState = PlayerState.PLAYING; | |
| return; | |
| } |
| if (!string.IsNullOrEmpty(displayName)) | ||
| return displayName; | ||
| } | ||
| catch (JsonException) { } |
There was a problem hiding this comment.
Silent exception swallowing — invalid metadata is silently ignored
catch (JsonException) { } discards all parse failures without any trace. If a participant sends malformed metadata this will silently fall through to the Name/Identity fallback, making it very hard to diagnose in the field.
| catch (JsonException) { } | |
| catch (JsonException e) { ReportHub.LogWarning(ReportCategory.COMMS, $"Failed to parse participant metadata for display name: {e.Message}"); } |
github-actions
bot
requested review from
AnsisMalins,
Ludmilafantaniella and
mikhail-dcl
… rescan, log metadata parse failures - Add ReopenVideoStream() to recover video without tearing down audio - Throttle empty-audio rescan to 2s interval to avoid per-frame lock - Replace silent JsonException catch with ReportHub.LogWarning
cyaiox
added
force-build
|
@claude review |
|
Claude finished @cyaiox's task in 5m 33s —— View job Code Review
Progress since last reviewThe three critical issues from the previous review are now resolved:
Remaining Issues🔴 Race condition —
|
|
|
||
| string? dominantSpeaker = null; | ||
|
|
||
| foreach (string speakerIdentity in room.ActiveSpeakers) |
There was a problem hiding this comment.
Potential race condition — room.ActiveSpeakers iterated without a lock
Every other participant collection access in this file uses lock (room.Participants). room.ActiveSpeakers is a shared collection updated from the LiveKit networking thread, but it is accessed here without any synchronisation. If it is mutated concurrently the foreach can throw InvalidOperationException or silently skip entries.
Either wrap in a lock (matching whatever lock ActiveSpeakers uses), use a snapshot, or confirm in a code comment that the collection is already thread-safe. Fix this →
| /// Finds the audio track paired to a specific video track from the same participant. | ||
| /// Available for future targeted audio scenarios. | ||
| /// </summary> | ||
| private Weak<AudioStream> FindPairedAudio(string identity, string videoSid) |
There was a problem hiding this comment.
Dead code — FindPairedAudio has no callers
Per CLAUDE.md: "Don't create helpers, utilities, or abstractions for one-time operations. Don't design for hypothetical future requirements."
The XML doc says "available for future targeted audio scenarios" but shipping uncalled private methods adds maintenance burden: it must be kept compiling, updated if surrounding types change, and confuses readers who try to trace call graphs. Remove until there's a concrete call site. Fix this →
OpenAllAudioStreams now skips tracks that already have a source, so it can be called periodically without disrupting healthy audio. Dead sources are removed individually. EnsureVideoIsPlaying also drives audio discovery since PBVideoPlayer entities don't trigger UpdateAudioStream.
cyaiox
added
force-build
4 participants
Pull Request Description
What does this PR change?
Enhances the LiveKit media player to support Cast scenarios with multiple participants, richer stream metadata, and automatic recovery when tracks disconnect.
Changes
getActiveVideoStreamsresponse, adding:to the JSON output, enabling scenes to display participant info and build stream-selection UIs
Test Instructions
Prerequisites
Streaming Theatreto .zone/.orgTest Steps
a. Click on the Showcase button
b. Should be an empty list
a. Should be one participant
b. Click on the Showcase dropdown -> Select the camera option
c. The Video screen should stream your camera
a. Should be one participant
b. Click on the Showcase dropdown -> Select the screen option
c. The Video screen should stream your screen
d. Stop sharing your screen
e. The video screen should stream the first available stream(your camera)
Additional Testing Notes
Quality Checklist
Code Review Reference
Please review our Code Review Standards before submitting.
Screenshots