Online Fbank with High CPU Usage (No Cuda for OnlineFbank)

[hsn-zeinali]

Hi there,

We are working on Online ASR. For the feature extraction part, we initially placed it on the GPU, similar to the offline version, but it resulted in errors. On the CPU, however, we observe high CPU usage when handling a large number of requests, whereas in the offline mode, this issue does not occur since processing is done on the GPU.

Why is GPU support not available for the online mode?

Do you have any plans to add this feature in the near future?

[csukuangfj]

please give more details with logs.

[hsn-zeinali]

Sorry for the late reply. Below is a code snippet that demonstrates the issue I'm encountering. It appears that the OnlineFbank feature extractor is hardcoded to work only on CPU, even when explicitly setting the device to CUDA:

import kaldifeat
opts = kaldifeat.FbankOptions()
opts.device = 'cuda'
opts.frame_opts.dither = 0
opts.frame_opts.snip_edges = False
opts.frame_opts.samp_freq = 16000
opts.frame_opts.frame_length_ms = 25
opts.frame_opts.frame_shift_ms = 10
opts.frame_opts.max_feature_vectors = 45 * 2
opts.mel_opts.num_bins = 80
opts.mel_opts.high_freq = -400
online_fbank = kaldifeat.OnlineFbank(opts)

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.12/site-packages/kaldifeat-1.25.5.dev20250122+cuda12.4.torch2.4.0-py3.12-linux-x86_64.egg/kaldifeat/fbank.py", line 18, in __init__
    super().__init__(opts)
  File "/opt/anaconda3/lib/python3.12/site-packages/kaldifeat-1.25.5.dev20250122+cuda12.4.torch2.4.0-py3.12-linux-x86_64.egg/kaldifeat/online_feature.py", line 23, in __init__
    assert opts.device.type == "cpu"
           ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Environment Info:

kaldifeat==1.25.5.dev20250122+cuda12.4.torch2.4.0

It would be helpful to clarify whether GPU support for OnlineFbank is currently planned or if this limitation is by design.

Thanks! 🙏

[Adibian]

please give more details with logs.

I’m facing the same problem. I'd be grateful if you could let me know if you find a solution.