Skip to content

feat: STT model params #4047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zhanweizhang7 wants to merge 1 commit into from
Closed

feat: STT model params #4047

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions apps/models_provider/impl/aliyun_bai_lian_model_provider/credential/omni_stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
from models_provider.base_model_provider import BaseModelCredential, ValidCode
from django.utils.translation import gettext as _

# class AliyunBaiLianOmiSTTModelParams(BaseForm):
# CueWord = forms.TextInputField(
# TooltipLabel(_('CueWord'), _('If not passed, the default value is What is this audio saying? Only answer the audio content')),
# required=True,
# default_value='这段音频在说什么，只回答音频的内容',
# )
class AliyunBaiLianOmiSTTModelParams(BaseForm):
CueWord = forms.TextInputField(
TooltipLabel(_('CueWord'), _('If not passed, the default value is What is this audio saying? Only answer the audio content')),
required=True,
default_value='这段音频在说什么，只回答音频的内容',
)


class AliyunBaiLianOmiSTTModelCredential(BaseForm, BaseModelCredential):
Expand Down Expand Up @@ -70,4 +70,4 @@ def encryption_dict(self, model: Dict[str, object]) -> Dict[str, object]:

def get_model_params_setting_form(self, model_name):

pass
return AliyunBaiLianOmiSTTModelParams()
Copy link
Contributor

@shaohuzhang1 shaohuzhang1 Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no immediate irregularities or major issues with the provided code snippet. However, there's a small improvement you can make to enhance readability:

from models_provider.base_model_provider import BaseModelCredential, ValidCode
from django.utils.translation import gettext_lazy as _

class AliyunBaiLianOmiSTTModelParams(forms.Form):  # Change base class from BaseForm to forms.Form

    CueWord = forms.TextInputField(
        tooltip_label=_('CueWord'),
        help_text=_("If not passed, the default value is '这段音频在说什么，只回答音频的内容'"),
        required=True,
    )

class AliyunBaiLianOmiSTTModelCredential(forms.ModelForm, BaseModelCredential):  # Add ModelForm mixin
    credential_type_code_choices = (
        ('ali_bailian_omi_stt', "Aliyun BaiLian OMI STT Credential"),
    )  # Ensure this part isn't missing

    def encryption_dict(self, model: Dict[str, object]) -> Dict[str, object]:
        pass

    def get_model_params_setting_form(self, model_name):
        # If you need additional customization for form fields, add them here.
        return AliyunBaiLianOmiSTTModelParams()

Changes Made:

  1. Base Class Update: Changed BaseForm in AliyunBaiLianOmiSTTModelParams to forms.Form. This change is standard practice when using Django's built-in form library.

  2. Attributes Formatting: Updated TooltipLabel and required=True to use Django's preferred attribute names (tooltip_label, help_text) and improved their formatting slightly for better clarity and consistency.

  3. ModelForm Mixin: Added the forms.ModelForm mix into AliyunBaiLianOmiSTTModelCredential, which provides some convenience methods (like cleaned_data) and allows handling of related models more directly by automatically creating a form tied to the model instance.

  4. Additional Documentation: Provided an example on how to customize the form further if needed.

These changes should make the code more readable and align closely with best practices used in Django development.

2 changes: 1 addition & 1 deletion apps/models_provider/impl/aliyun_bai_lian_model_provider/model/omni_stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def speech_to_text(self, audio_file):
"format": "mp3",
},
},
{"type": "text", "text": '这段音频在说什么，只回答音频的内容'},
{"type": "text", "text": self.params.get('CueWord')},
],
},
],
Copy link
Contributor

@shaohuzhang1 shaohuzhang1 Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code snippet seems to be part of an API request that sends a transcribed text message along with an audio file. Some potential issues and optimizations:

  1. Parameter Usage: The self.params dictionary is used without checking if it contains the key 'CueWord'. This could lead to KeyError if the parameter is not present.

  2. Text Content: The text "这段音频在说什么，只回答音频的内容" is hardcoded within the response. If you want to dynamically inject this text based on the value from the parameters, adjust accordingly.

  3. Optimization: Ensure that all necessary environment variables or dependencies (like Google Cloud Speech-to-Text API client) are properly set up and initialized before making requests. Also, consider using asynchronous programming if dealing with multiple requests at once.

Here's an example of how you might modify the function to handle these considerations:

def speech_to_text(self, audio_file):
    # Assuming params is already defined and contains necessary configurations
    prompt = {
        "language_code": "zh-CN",
        "media_content_type": "audio/mp3",
        "input": {
            "content": open(audio_file, 'rb').read(),
        },
        "output_config": {
            "audio_encoding": "LINEAR16",  # Adjust encoding according to your needs
        },
                }
        
    return prompt

# Example usage:
# prompt_dict = my_instance.speech_to_text("path/to/audio.mp3")

Key Points:

  • Check for Parameter Existence: Before accessing params.get('CueWord'), check its presence.
  • Dynamic Content: Replace static content with dynamic references to parameters where appropriate.
  • Environment Setup: Verify and initialize dependencies needed for API calls.
  • Asynchronous Processing: Consider asynchronous processing to improve performance when handling multiple requests concurrently.

Expand Down
66 changes: 33 additions & 33 deletions apps/models_provider/impl/tencent_model_provider/credential/stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,38 +8,38 @@
from models_provider.base_model_provider import BaseModelCredential, ValidCode


# class TencentSSTModelParams(BaseForm):
# EngSerViceType = forms.SingleSelect(
# TooltipLabel(_('Engine model type'), _('If not passed, the default value is 16k_zh (Chinese universal)')),
# required=True,
# default_value='16k_zh',
# option_list=[
# {"value": "8k_zh", "label": _("Chinese telephone universal")},
# {"value": "8k_en", "label": _("English telephone universal")},
# {"value": "16k_zh", "label": _("Commonly used in Chinese")},
# {"value": "16k_zh-PY", "label": _("Chinese, English, and Guangdong")},
# {"value": "16k_zh_medical", "label": _("Chinese medical")},
# {"value": "16k_en", "label": _("English")},
# {"value": "16k_yue", "label": _("Cantonese")},
# {"value": "16k_ja", "label": _("Japanese")},
# {"value": "16k_ko", "label": _("Korean")},
# {"value": "16k_vi", "label": _("Vietnamese")},
# {"value": "16k_ms", "label": _("Malay language")},
# {"value": "16k_id", "label": _("Indonesian language")},
# {"value": "16k_fil", "label": _("Filipino language")},
# {"value": "16k_th", "label": _("Thai")},
# {"value": "16k_pt", "label": _("Portuguese")},
# {"value": "16k_tr", "label": _("Turkish")},
# {"value": "16k_ar", "label": _("Arabic")},
# {"value": "16k_es", "label": _("Spanish")},
# {"value": "16k_hi", "label": _("Hindi")},
# {"value": "16k_fr", "label": _("French")},
# {"value": "16k_de", "label": _("German")},
# {"value": "16k_zh_dialect", "label": _("Multiple dialects, supporting 23 dialects")}
# ],
# value_field='value',
# text_field='label'
# )
class TencentSSTModelParams(BaseForm):
EngSerViceType = forms.SingleSelect(
TooltipLabel(_('Engine model type'), _('If not passed, the default value is 16k_zh (Chinese universal)')),
required=True,
default_value='16k_zh',
option_list=[
{"value": "8k_zh", "label": _("Chinese telephone universal")},
{"value": "8k_en", "label": _("English telephone universal")},
{"value": "16k_zh", "label": _("Commonly used in Chinese")},
{"value": "16k_zh-PY", "label": _("Chinese, English, and Guangdong")},
{"value": "16k_zh_medical", "label": _("Chinese medical")},
{"value": "16k_en", "label": _("English")},
{"value": "16k_yue", "label": _("Cantonese")},
{"value": "16k_ja", "label": _("Japanese")},
{"value": "16k_ko", "label": _("Korean")},
{"value": "16k_vi", "label": _("Vietnamese")},
{"value": "16k_ms", "label": _("Malay language")},
{"value": "16k_id", "label": _("Indonesian language")},
{"value": "16k_fil", "label": _("Filipino language")},
{"value": "16k_th", "label": _("Thai")},
{"value": "16k_pt", "label": _("Portuguese")},
{"value": "16k_tr", "label": _("Turkish")},
{"value": "16k_ar", "label": _("Arabic")},
{"value": "16k_es", "label": _("Spanish")},
{"value": "16k_hi", "label": _("Hindi")},
{"value": "16k_fr", "label": _("French")},
{"value": "16k_de", "label": _("German")},
{"value": "16k_zh_dialect", "label": _("Multiple dialects, supporting 23 dialects")}
],
value_field='value',
text_field='label'
)

class TencentSTTModelCredential(BaseForm, BaseModelCredential):
REQUIRED_FIELDS = ["SecretId", "SecretKey"]
Expand Down Expand Up @@ -87,4 +87,4 @@ def encryption_dict(self, model):
SecretKey = forms.PasswordInputField('SecretKey', required=True)

def get_model_params_setting_form(self, model_name):
pass
return TencentSSTModelParams()
Copy link
Contributor

@shaohuzhang1 shaohuzhang1 Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code snippet looks largely intact and contains no immediate syntax errors or logical issues, but there are some improvements that can be made:

  1. Variable Names: Ensure consistency in variable names throughout the file to improve readability.

  2. Comments Formatting: The comments follow snake_case format which is more common in Python than underscore_separated_names like EngSerViceType. Convert them accordingly if desired.

  3. Docstring Consistency: It's beneficial to maintain consistent docstrings across functions and classes. Add appropriate comments explaining what each method does.

Here’s a revised version of the code with improved naming conventions and additional comments:

from models_provider.base_model_provider import BaseModelCredential, ValidCode

class TencentSSTModelParams(BaseForm):
    eng_service_type = forms.SingleSelect(
        TooltipLabel(_('Engine model type'), _('If not passed, the default value is 16k_zh (Chinese universal)')),
        required=True,
        default_value='16k_zh',
        option_list=[
            {"value": "8k_zh", "label": _("Chinese telephone universal")},
            {"value": "8k_en", "label": _("English telephone universal")},
            {"value": "16k_zh", "label": _("Commonly used in Chinese")},
            {"value": "16k_zh-py", "label": _("Chinese, English, and Guangdong")},
            {"value": "16k_zh_medical", "label": _("Chinese medical")},
            {"value": "16k_en", "label": _("English")},
            {"value": "16k_yue", "label": _("Cantonese")},
            {"value": "16k_ja", "label": _("Japanese")},
            {"value": "16k_ko", "label": _("Korean")},
            {"value": "16k_vi", "label": _("Vietnamese")},
            {"value": "16k_ms", "label": _("Malay language")},
            {"value": "16k_id", "label": _("Indonesian language")},
            {"value": "16k_fil", "label": _("Filipino language")},
            {"value": "16k_th", "label": _(" Thai")},
            {"value": "16k_pt", "label": _("Portuguese")},
            {"value": "16k_tr", "label": _("Turkish")},
            {"value": "16k_ar", "label": _("Arabic")},
            {"value": "16k_es", "label": _("Spanish")},
            {"value": "16k_hi", "label": _("Hindi")},
            {"value": "16k_fr", "label": _("French")},
            {"value": "16k_de", "label": _("German")},
            {"value": "16k_zh_dialect", "label": _("Multiple dialects, supporting 23 dialects")}
        ],
        value_field="value",
        text_field="label"
    )

class TencentSTTModelCredential(BaseForm, BaseModelCredential):
   REQUIRED_FIELDS = ["secret_id", "secret_key"]

    secret_id = forms.CharField('secret_id', required=True)
    secret_key = forms.PasswordInputField('secret_key', required=True)

    def get_model_params_setting_form(self, model_name):
        # Retrieve and return specific configuration form for the given model
        return TencentSSTModelParams()

Additional Remarks:

  • Use lowercase eng_service_type instead of EngSerViceType.
  • Add an empty line after method definitions and before function calls for better formatting.
  • Consider adding a comment at the top of the file to provide context about what the module or class does.

These changes make the code cleaner and easier to maintain.

2 changes: 1 addition & 1 deletion apps/models_provider/impl/tencent_model_provider/model/stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def speech_to_text(self, audio_file):
# 实例化一个请求对象,每个接口都会对应一个request对象
req = models.SentenceRecognitionRequest()
params = {
"EngSerViceType": '16k_zh',
"EngSerViceType": self.params.get('EngSerViceType'),
"SourceType": 1,
"VoiceFormat": "mp3",
"Data": _v.decode(),
Expand Down
Loading