Skip to content

Commit 6dd5ad2

Browse files
committed
[Bugfix] (qwen3_tts): enable batched offline inference by fixing tensor slicing
Signed-off-by: Roman Koshkin <roman.koshkin@sbintuitions.co.jp>
1 parent 0b577a7 commit 6dd5ad2

File tree

6 files changed

+865
-53
lines changed

6 files changed

+865
-53
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,3 +245,5 @@ tmp_test
245245

246246
# output files
247247
*.wav
248+
examples/offline_inference/qwen3_tts/test.py
249+
examples/online_serving/qwen3_tts/Untitled.ipynb

READMEmy.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
2+
3+
```bash
4+
cd /lustre/users/rkoshkin
5+
git clone https://github.com/vllm-project/vllm-omni.git
6+
cd vllm-omni
7+
uv venv --python 3.10 --seed
8+
source .venv/bin/activate
9+
cd ..
10+
git clone https://github.com/vllm-project/vllm.git
11+
cd vllm
12+
git checkout v0.16.0
13+
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.16.0/vllm-0.16.0-cp38-abi3-manylinux_2_31_x86_64.whl
14+
uv pip install -e .
15+
cd ../vllm-omni
16+
uv pip install -e .
17+
```
18+
19+
20+
21+
```bash
22+
# edit /lustre/users/rkoshkin/vllm-omni/vllm_omni/model_executor/stage_configs/qwen3_tts.yaml AS NECESSARY
23+
cd examples/online_serving/qwen3_tts
24+
./run_server.sh Base
25+
```

0 commit comments

Comments
 (0)