Skip to content

Commit 59eda4a

Browse files
chenosaurusladvoc
chenosaurus
and
ladvoc
authored
Add example for local audio streaming (#676)
* initial commit * add params for identity and room * set default for playback * fix remote participant playback * update readme * fix * update readme * adding local audio example * cleanup readme * debug output * remove log * add ability to select channel * add room db meter and clean up * hooking up APM * Update cargo lock * Update cargo lock --------- Co-authored-by: Jacob Gelman <3182119+ladvoc@users.noreply.github.com>
1 parent e6ae543 commit 59eda4a

File tree

9 files changed

+3064
-1067
lines changed

9 files changed

+3064
-1067
lines changed

‎examples/Cargo.lock‎

Lines changed: 1563 additions & 1067 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
[workspace]
22
members = [
33
"basic_room",
4+
"local_audio",
45
"mobile",
56
"save_to_disk",
67
"wgpu_room",

‎examples/local_audio/Cargo.toml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
[package]
2+
name = "local_audio"
3+
version = "0.1.0"
4+
edition = "2021"
5+
6+
[dependencies]
7+
tokio = { version = "1", features = ["full"] }
8+
env_logger = "0.10"
9+
livekit = { path = "../../livekit", features = ["rustls-tls-native-roots"]}
10+
livekit-api = { path = "../../livekit-api", features = ["rustls-tls-native-roots"]}
11+
libwebrtc = { path = "../../libwebrtc" }
12+
log = "0.4"
13+
cpal = "0.15"
14+
anyhow = "1.0"
15+
clap = { version = "4.0", features = ["derive"] }
16+
futures-util = "0.3"

‎examples/local_audio/README.md‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Local Audio Capture Example
2+
3+
This example demonstrates how to capture audio from a local microphone and stream it to a LiveKit room while simultaneously playing back audio from other participants. It provides a complete bidirectional audio experience with real-time level monitoring.
4+
5+
## Features
6+
7+
- **Bidirectional Audio**: Capture from local microphone and play back remote participants
8+
- **Device Selection**: Choose specific input/output devices or use system defaults
9+
- **Real-time Level Meter**: Visual dB meter showing local microphone levels
10+
- **Audio Processing**: Echo cancellation, noise suppression, and auto gain control (enabled by default)
11+
- **Volume Control**: Adjustable playback volume for remote participants
12+
- **Audio Mixing**: Combines audio from multiple remote participants
13+
- **Format Support**: Handles F32, I16, and U16 sample formats
14+
- **Cross-platform**: Works on Windows, macOS, and Linux
15+
16+
## Prerequisites
17+
18+
1. **Rust**: Install Rust 1.70+ from [rustup.rs](https://rustup.rs/)
19+
2. **LiveKit Server**: Access to a LiveKit server instance
20+
3. **Audio Devices**: Working microphone and speakers/headphones
21+
4. **System Permissions**: Audio device access permissions
22+
23+
### Platform-specific Requirements
24+
25+
- **macOS**: Grant microphone permissions in System Preferences → Privacy & Security → Microphone
26+
- **Windows**: Ensure audio drivers are installed and microphone is not in use by other applications
27+
- **Linux**: May need ALSA or PulseAudio libraries (`sudo apt install libasound2-dev` on Ubuntu/Debian)
28+
29+
## Setup
30+
31+
1. **LiveKit Connection Details** (choose one method):
32+
33+
**Option A: Environment Variables**
34+
```bash
35+
export LIVEKIT_URL="wss://your-livekit-server.com"
36+
export LIVEKIT_API_KEY="your-api-key"
37+
export LIVEKIT_API_SECRET="your-api-secret"
38+
```
39+
40+
**Option B: CLI Arguments**
41+
Pass connection details directly to the command (see examples below)
42+
43+
**Note**: CLI arguments take precedence over environment variables. You can mix both methods - for example, set API credentials via environment variables but override the URL via CLI.
44+
45+
2. **Build the Example**:
46+
47+
```bash
48+
cd examples/local_audio
49+
cargo build --release
50+
```
51+
52+
## Usage
53+
54+
### List Available Audio Devices
55+
56+
```bash
57+
cargo run -- --list-devices
58+
```
59+
60+
Example output:
61+
```
62+
Available Input Devices:
63+
───────────────────────────────────────────────────────────────
64+
1. MacBook Pro Microphone
65+
├─ Sample Rate: 8000-48000 Hz
66+
├─ Channels: 1-2
67+
└─ Formats: F32, I16
68+
69+
2. USB Microphone
70+
├─ Sample Rate: 44100-48000 Hz
71+
├─ Channels: 1-2
72+
└─ Formats: F32, I16
73+
74+
Default Input Device: MacBook Pro Microphone
75+
76+
Available Output Devices:
77+
───────────────────────────────────────────────────────────────
78+
1. MacBook Pro Speakers
79+
├─ Sample Rate: 8000-48000 Hz
80+
├─ Channels: 2
81+
└─ Formats: F32, I16
82+
83+
2. USB Headphones
84+
├─ Sample Rate: 44100-48000 Hz
85+
├─ Channels: 2
86+
└─ Formats: F32, I16
87+
88+
Default Output Device: MacBook Pro Speakers
89+
```
90+
91+
### Basic Usage
92+
93+
Stream audio with default settings (using environment variables):
94+
95+
```bash
96+
cargo run
97+
```
98+
99+
Using CLI arguments for connection details:
100+
101+
```bash
102+
cargo run -- \
103+
--url "wss://your-project.livekit.cloud" \
104+
--api-key "your-api-key" \
105+
--api-secret "your-api-secret"
106+
```
107+
108+
Join a specific room with custom identity:
109+
110+
```bash
111+
cargo run -- \
112+
--url "wss://your-project.livekit.cloud" \
113+
--api-key "your-api-key" \
114+
--api-secret "your-api-secret" \
115+
--room-name "my-meeting" \
116+
--identity "john-doe"
117+
```
118+
119+
### Advanced Configuration
120+
121+
```bash
122+
cargo run -- \
123+
--url "wss://your-project.livekit.cloud" \
124+
--api-key "your-api-key" \
125+
--api-secret "your-api-secret" \
126+
--input-device "USB Microphone" \
127+
--output-device "USB Headphones" \
128+
--sample-rate 44100 \
129+
--channels 2 \
130+
--volume 0.8 \
131+
--room-name "conference-room"
132+
```
133+
134+
### Capture-Only Mode
135+
136+
Disable audio playback and only capture:
137+
138+
```bash
139+
cargo run -- \
140+
--url "wss://your-project.livekit.cloud" \
141+
--api-key "your-api-key" \
142+
--api-secret "your-api-secret" \
143+
--no-playback
144+
```
145+
146+
## Command Line Options
147+
148+
| Option | Description | Default |
149+
|--------|-------------|---------|
150+
| `--list-devices` | List available audio devices and exit | - |
151+
| `--input-device <NAME>` | Input device name | System default |
152+
| `--output-device <NAME>` | Output device name | System default |
153+
| `--sample-rate <HZ>` | Sample rate in Hz | 48000 |
154+
| `--channels <COUNT>` | Number of channels | 1 |
155+
| `--echo-cancellation` | Enable echo cancellation | true |
156+
| `--noise-suppression` | Enable noise suppression | true |
157+
| `--auto-gain-control` | Enable auto gain control | true |
158+
| `--no-playback` | Disable audio playback (capture only) | false |
159+
| `--volume <LEVEL>` | Playback volume (0.0 to 1.0) | 1.0 |
160+
| `--identity <NAME>` | LiveKit participant identity | "rust-audio-streamer" |
161+
| `--room-name <NAME>` | LiveKit room name | "audio-room" |
162+
| `--url <URL>` | LiveKit server URL | From LIVEKIT_URL env var |
163+
| `--api-key <KEY>` | LiveKit API key | From LIVEKIT_API_KEY env var |
164+
| `--api-secret <SECRET>` | LiveKit API secret | From LIVEKIT_API_SECRET env var |

‎examples/local_audio/src/audio_capture.rs‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
use anyhow::{anyhow, Result};
2+
use cpal::traits::{DeviceTrait, StreamTrait};
3+
use cpal::{Device, SampleFormat, Stream, StreamConfig, SizedSample};
4+
use log::{error, info, warn};
5+
use std::sync::{
6+
atomic::{AtomicBool, Ordering},
7+
Arc,
8+
};
9+
use tokio::sync::mpsc;
10+
11+
pub struct AudioCapture {
12+
_stream: Stream,
13+
is_running: Arc<AtomicBool>,
14+
}
15+
16+
impl AudioCapture {
17+
pub async fn new(
18+
device: Device,
19+
config: StreamConfig,
20+
sample_format: SampleFormat,
21+
audio_tx: mpsc::UnboundedSender<Vec<i16>>,
22+
db_tx: Option<mpsc::UnboundedSender<f32>>,
23+
channel_index: u32, // New: Index of the channel to capture
24+
num_input_channels: u32, // New: Total number of channels in input
25+
) -> Result<Self> {
26+
let is_running = Arc::new(AtomicBool::new(true));
27+
let is_running_clone = is_running.clone();
28+
29+
let stream = match sample_format {
30+
SampleFormat::F32 => Self::create_input_stream::<f32>(device, config, audio_tx, db_tx, is_running_clone, channel_index, num_input_channels)?,
31+
SampleFormat::I16 => Self::create_input_stream::<i16>(device, config, audio_tx, db_tx, is_running_clone, channel_index, num_input_channels)?,
32+
SampleFormat::U16 => Self::create_input_stream::<u16>(device, config, audio_tx, db_tx, is_running_clone, channel_index, num_input_channels)?,
33+
sample_format => {
34+
return Err(anyhow!("Unsupported sample format: {:?}", sample_format));
35+
}
36+
};
37+
38+
stream.play()?;
39+
info!("Audio capture stream started");
40+
41+
Ok(AudioCapture {
42+
_stream: stream,
43+
is_running,
44+
})
45+
}
46+
47+
fn create_input_stream<T>(
48+
device: Device,
49+
config: StreamConfig,
50+
audio_tx: mpsc::UnboundedSender<Vec<i16>>,
51+
db_tx: Option<mpsc::UnboundedSender<f32>>,
52+
is_running: Arc<AtomicBool>,
53+
channel_index: u32, // New: Index of the channel to capture
54+
num_input_channels: u32, // New: Total number of channels in input
55+
) -> Result<Stream>
56+
where
57+
T: SizedSample + Send + 'static,
58+
{
59+
let stream = device.build_input_stream(
60+
&config,
61+
move |data: &[T], _: &cpal::InputCallbackInfo| {
62+
if !is_running.load(Ordering::Relaxed) {
63+
return;
64+
}
65+
66+
// Extract samples from the selected channel (assuming interleaved format)
67+
let converted: Vec<i16> = data.iter()
68+
.skip(channel_index as usize)
69+
.step_by(num_input_channels as usize)
70+
.map(|&sample| Self::convert_sample_to_i16(sample))
71+
.collect();
72+
73+
// Calculate and send dB level if channel is available (now on selected channel only)
74+
if let Some(ref db_sender) = db_tx {
75+
let db_level = crate::db_meter::calculate_db_level(&converted);
76+
if let Err(e) = db_sender.send(db_level) {
77+
warn!("Failed to send dB level: {}", e);
78+
}
79+
}
80+
81+
if let Err(e) = audio_tx.send(converted) {
82+
warn!("Failed to send audio data: {}", e);
83+
}
84+
},
85+
move |err| {
86+
error!("Audio input stream error: {}", err);
87+
},
88+
None,
89+
)?;
90+
91+
Ok(stream)
92+
}
93+
94+
fn convert_sample_to_i16<T: SizedSample>(sample: T) -> i16 {
95+
if std::mem::size_of::<T>() == std::mem::size_of::<f32>() {
96+
let sample_f32 = unsafe { std::mem::transmute_copy::<T, f32>(&sample) };
97+
(sample_f32.clamp(-1.0, 1.0) * i16::MAX as f32) as i16
98+
} else if std::mem::size_of::<T>() == std::mem::size_of::<i16>() {
99+
unsafe { std::mem::transmute_copy::<T, i16>(&sample) }
100+
} else if std::mem::size_of::<T>() == std::mem::size_of::<u16>() {
101+
let sample_u16 = unsafe { std::mem::transmute_copy::<T, u16>(&sample) };
102+
((sample_u16 as i32) - (u16::MAX as i32 / 2)) as i16
103+
} else {
104+
0
105+
}
106+
}
107+
108+
pub fn stop(&self) {
109+
self.is_running.store(false, Ordering::Relaxed);
110+
}
111+
}
112+
113+
impl Drop for AudioCapture {
114+
fn drop(&mut self) {
115+
self.stop();
116+
}
117+
}

0 commit comments

Comments
 (0)