Skip to content

fix: Whisper stt model #4352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zhanweizhang7 merged 1 commit into from
Nov 12, 2025
Merged

fix: Whisper stt model #4352

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/models_provider/impl/vllm_model_provider/credential/whisper_stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

class VLLMWhisperModelParams(BaseForm):
Language = forms.TextInputField(
TooltipLabel(_('Language'),
TooltipLabel(_('language'),
_("If not passed, the default value is 'zh'")),
required=True,
default_value='zh',
Copy link
Contributor Author

@shaohuzhang1 shaohuzhang1 Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TooltipLabel field should be replaced with 'TooltipText'. Here's the corrected version of your code:

@@ -13,7 +13,7 @@ class VLLMWhisperModelParams(BaseForm):
     Language = forms.TextInputField(
-        TooltipLabel(_('Language'),
+        TooltipText(_('language'),
                      _("If not passed, the default value is 'zh'")),
         required=True,
         default_value='zh',
     )

Explanation:

  1. Error: The use of TooltipLabel in the tooltip_args= parameter would result in an error because there is no such argument supported by forms.TextInputField.
  2. Change: Replace TooltipLabel with TooltipText, which seems to be the intended usage for providing tooltips in Django forms.
  3. Corrected Code Snippet: The corrected snippet now correctly sets up the tooltip for the "Language" input field using TooltipText.

This change ensures that the tooltip will display correctly when rendering the form in the user interface.

Expand Down
4 changes: 2 additions & 2 deletions apps/models_provider/impl/vllm_model_provider/model/whisper_sst.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,11 @@ def speech_to_text(self, audio_file):
api_key=self.api_key,
base_url=base_url
)

buf = audio_file.read()
filter_params = {k: v for k, v in self.params.items() if k not in {'model_id', 'use_local', 'streaming'}}
transcription_params = {
'model': self.model,
'file': audio_file,
'file': buf,
'language': 'zh',
}
result = client.audio.transcriptions.create(
Copy link
Contributor Author

@shaohuzhang1 shaohuzhang1 Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code has a minor issue with how the file parameter is being handled in the API call. The audio_file.read() method reads all the content of the file into memory before passing it to the API, which can be inefficient for large files.

Here’s a corrected version of your function:

def speech_to_text(self, audio_file):
    api_key = self.api_key
    base_url = "your_base_url"  # Replace with the actual URL

    client = google.cloud.speech_v2.SpeechClient(credentials=self.credentials, region_name='us-east1')
    
    filter_params = {k: v for k, v in self.params.items() if k not in {'model_id', 'use_local', 'streaming'}}

    transcription_params = {
        'model': self.model,
        'config': {
            'enable_word_confidence': True,
            'encoding': media_audio.RecognitionConfig.AudioEncoding.LINEAR16,
            'sample_rate_hertz': 44_100,
            'language_code': 'zh-CN'
        },
        'audio': {
            'content': (buf := audio_file.read())
        }
    }

    response = client.long_running_recognize(transcription_params)

    return response

Key Changes:

  1. Buffer Read: Instead of reading the entire file at once, audio_file.read() is used within a generator expression (buf := audio_file.read()). This allows processing the file in chunks, which is memory-efficient and avoids loading the entire file into RAM.

  2. Configuration Updates:

    • Added media_audio.RecognitionConfig.AudioEncoding.LINEAR16 encoding type as required by some Google Cloud APIs.
    • Set 'sample_rate_hertz' to 44_100 Hz based on common settings for Chinese speech recognition.

  3. Transcription Parameters: Updated the transcription_params dictionary to include both the configuration (config) and audio data (audio). Since GCP's Speech-to-Text service now uses long-running operations, you need to use client.long_running_recognize.

This should resolve the potential issues related to memory usage and ensure proper formatting according to the latest requirements from Google Cloud Speech-to-Text.

Expand Down
Loading