feat: add microphone protocal#123
Open
Yundi339 wants to merge 1 commit intomoonlight-stream:masterfrom
Open
Conversation
Author
|
Our microphone implementation is based on VBCable. While it shares functional similarities with another ongoing Sunshine Pull Request, the underlying mechanisms differ. We have considered merging the complete microphone codebase into the upstream repository; however, several challenges remain:
Given these factors, our current priority is to continue refining these capabilities. Should the opportunity arise to integrate with the community version in the future, we intend to perform a complete rewrite of the microphone logic to ensure it meets community standards and maintains long-term maintainability. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The following is translated by AI.
Hi, I'm the developer of the Sunshine-Foundation and moonlight-vplus.
Moonlight has always been a one-way street for audio/video - downstream only, no microphone upstream. We implemented a microphone transmission protocol in Moonlight VPlus, and it's been in production with many users.
This PR contributes the code to the community, aiming to establish a unified specification. Better to standardize now before wider adoption to avoid microphone protocol conflicts down the road.
Everything defined here is open for modification - we'll follow the official guidelines.
Protocol Overview
Client captures mic → Opus encode → UDP send to host. Supports both plaintext and encrypted modes.
Packet Format
Followed by Opus-encoded audio (optionally encrypted). Total packet <= 1400 bytes.
Key Constants
Encryption Support
We use Sunshine-Foundation and Moonlight VPlus. The old version only supported plaintext MIC data transmission; the new version adds encrypted transmission.
For smooth upgrades (some users may still be on older versions), we implemented compatibility handling: RTSP handshake identifies whether both sides support encryption. If one side doesn't support it, falls back to plaintext. Developers can also force MIC to use encrypted transmission in code for absolute security.
remoteInputAesKeyriKeyId + sequenceNumber(big-endian) in first 4 bytesDesign Notes
MIC_PACKET_MAGICThis value is sent in every packet's SSRC field. Sunshine currently doesn't validate it.
Design intent: If multiple mic implementations emerge (different codecs, configs), this value can identify client type and enable plugin-based processing.
Reserved Interfaces
These are defined but not currently used, reserved for scenarios where mic needs to be dynamically started/stopped via control stream:
FEC and Packet Loss
Opus library has built-in FEC and packet loss concealment, default FEC level is 1. Not explicitly configured in code - can be added to RTSP handshake if stronger FEC protection is needed.
Files Changed
MicrophoneStream.cLimelight.henableMicconfig and encryption flagLimelight-internal.hInput.hConnection.cRtspConnection.cSdpGenerator.cCompatibility