Skip to content

feat: add microphone protocal#123

Open
Yundi339 wants to merge 1 commit intomoonlight-stream:masterfrom
Yundi339:master
Open

feat: add microphone protocal#123
Yundi339 wants to merge 1 commit intomoonlight-stream:masterfrom
Yundi339:master

Conversation

@Yundi339
Copy link

The following is translated by AI.

Hi, I'm the developer of the Sunshine-Foundation and moonlight-vplus.

Moonlight has always been a one-way street for audio/video - downstream only, no microphone upstream. We implemented a microphone transmission protocol in Moonlight VPlus, and it's been in production with many users.

This PR contributes the code to the community, aiming to establish a unified specification. Better to standardize now before wider adoption to avoid microphone protocol conflicts down the road.

Everything defined here is open for modification - we'll follow the official guidelines.


Protocol Overview

Client captures mic → Opus encode → UDP send to host. Supports both plaintext and encrypted modes.

Packet Format

typedef struct {
    uint8_t  flags;           // Reserved, currently 0x00
    uint8_t  packetType;      // Fixed 0x61 (Opus audio)
    uint16_t sequenceNumber;  // Little-endian
    uint32_t timestamp;       // Little-endian
    uint32_t ssrc;            // Client identifier, little-endian
} MICROPHONE_PACKET_HEADER;  // 12 bytes

Followed by Opus-encoded audio (optionally encrypted). Total packet <= 1400 bytes.

Key Constants

#define MIC_PACKET_TYPE_OPUS  0x61        // Audio type
#define MAX_MIC_PACKET_SIZE   1400        // Max packet size
#define MIC_PACKET_MAGIC      0x12345678  // Client identifier
#define SS_ENC_MICROPHONE     0x08        // Encryption flag

Encryption Support

We use Sunshine-Foundation and Moonlight VPlus. The old version only supported plaintext MIC data transmission; the new version adds encrypted transmission.

For smooth upgrades (some users may still be on older versions), we implemented compatibility handling: RTSP handshake identifies whether both sides support encryption. If one side doesn't support it, falls back to plaintext. Developers can also force MIC to use encrypted transmission in code for absolute security.

  • Algorithm: AES-128-CBC, reuses remoteInputAesKey
  • IV Generation: riKeyId + sequenceNumber (big-endian) in first 4 bytes
  • Negotiation Logic: If audio encryption is enabled, mic encryption is automatically enabled; if one side doesn't support encryption, falls back to plaintext

Design Notes

MIC_PACKET_MAGIC

This value is sent in every packet's SSRC field. Sunshine currently doesn't validate it.

Design intent: If multiple mic implementations emerge (different codecs, configs), this value can identify client type and enable plugin-based processing.

Reserved Interfaces

These are defined but not currently used, reserved for scenarios where mic needs to be dynamically started/stopped via control stream:

#define SS_MICROPHONE_MAGIC   0x55000008
#define MIC_CONTROL_START     0x01
#define MIC_CONTROL_STOP      0x02

FEC and Packet Loss

Opus library has built-in FEC and packet loss concealment, default FEC level is 1. Not explicitly configured in code - can be added to RTSP handshake if stronger FEC protection is needed.


Files Changed

File Description
MicrophoneStream.c NEW - Core implementation
Limelight.h Added enableMic config and encryption flag
Limelight-internal.h Internal function declarations
Input.h Reserved interface definitions
Connection.c Lifecycle management
RtspConnection.c RTSP handshake
SdpGenerator.c Encryption negotiation

Compatibility

  • Mic initialization failure doesn't affect the connection, just mic unavailable
  • Legacy Sunshine handshake failure auto-disables mic feature
  • Encryption negotiation failure falls back to plaintext

@Yundi339
Copy link
Author

Our microphone implementation is based on VBCable. While it shares functional similarities with another ongoing Sunshine Pull Request, the underlying mechanisms differ. We have considered merging the complete microphone codebase into the upstream repository; however, several challenges remain:

  • Feature Coupling: The microphone logic is deeply integrated with our other proprietary features, making it difficult to isolate for a clean merge.
  • Stability and Iteration: Having undergone several version updates, we believe the features require further maturation and optimization through continued real-world testing.
  • Technical Refinement: Certain aspects, such as the Acoustic Echo Cancellation (AEC), have not yet reached peak performance. Additionally, two-way microphone communication between Sunshine and Moonlight is currently unsupported.

Given these factors, our current priority is to continue refining these capabilities. Should the opportunity arise to integrate with the community version in the future, we intend to perform a complete rewrite of the microphone logic to ensure it meets community standards and maintains long-term maintainability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant