Skip to content

fix: Models rerank stt #3934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zhanweizhang7 merged 1 commit into from
Aug 26, 2025
Merged

fix: Models rerank stt #3934

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions apps/models_provider/impl/aliyun_bai_lian_model_provider/credential/asr_stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@
from models_provider.base_model_provider import BaseModelCredential, ValidCode
from django.utils.translation import gettext as _

from models_provider.impl.aliyun_bai_lian_model_provider.credential.omni_stt import AliyunBaiLianOmiSTTModelParams


class AliyunBaiLianAsrSTTModelCredential(BaseForm, BaseModelCredential):
api_url = forms.TextInputField(_('API URL'), required=True)
Expand Down Expand Up @@ -64,4 +62,4 @@ def encryption_dict(self, model: Dict[str, object]) -> Dict[str, object]:

def get_model_params_setting_form(self, model_name):

return AliyunBaiLianOmiSTTModelParams()
pass
14 changes: 7 additions & 7 deletions apps/models_provider/impl/aliyun_bai_lian_model_provider/credential/omni_stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
from models_provider.base_model_provider import BaseModelCredential, ValidCode
from django.utils.translation import gettext as _

class AliyunBaiLianOmiSTTModelParams(BaseForm):
CueWord = forms.TextInputField(
TooltipLabel(_('CueWord'), _('If not passed, the default value is What is this audio saying? Only answer the audio content')),
required=True,
default_value='这段音频在说什么，只回答音频的内容',
)
# class AliyunBaiLianOmiSTTModelParams(BaseForm):
# CueWord = forms.TextInputField(
# TooltipLabel(_('CueWord'), _('If not passed, the default value is What is this audio saying? Only answer the audio content')),
# required=True,
# default_value='这段音频在说什么，只回答音频的内容',
# )


class AliyunBaiLianOmiSTTModelCredential(BaseForm, BaseModelCredential):
Expand Down Expand Up @@ -70,4 +70,4 @@ def encryption_dict(self, model: Dict[str, object]) -> Dict[str, object]:

def get_model_params_setting_form(self, model_name):

return AliyunBaiLianOmiSTTModelParams()
pass
Copy link
Contributor Author

@shaohuzhang1 shaohuzhang1 Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks mostly correct, but there are a few issues to consider:

  1. Indentation: The code has inconsistent indentation between lines 3 and 22. It would be good practice to use consistent spacing to avoid confusion.

  2. Duplicate Code: There is repetition in the AliyunBaiLianOmiStTModelParams class definition starting from line 7 to line 28. While this might not affect functionality, it can clutter the code and make maintenance more difficult. Consider using inheritance or composition if these fields are common across different classes.

  3. Comments Formatting: The comments around parameters seem to have mixed formats (#) which could lead to inconsistencies. Consistently using # throughout the file might improve readability.

  4. End of Line Characters: The trailing newline character at the end of the file is unnecessary. This can cause unexpected behavior when working with source control systems that manage files based on differences.

Here's an improved version of the code with some adjustments:

"""Module containing credential settings form for ALiYun BaiLian OMI STT model."""

from ..base_model_provider import BaseModelCredential
from django.utils.translation import gettext_lazy as _

class AliyunBaiLianOmiSTTModelParams(forms.Form):
    """Form settings to customize parameters for the ALiYun BaiLian OMI STT model."""
    
    cue_word = forms.CharField(
        label=_('CueWord'),
        help_text=_('If not passed, the default value is "What is this audio saying? Only answer the audio content"'),
        required=True,
        initial='这段音频在说什么，只回答音频的内容',
       )

class AliyunBaiLianOmiSTTModelCredential(BaseForm, BaseModelCredential):
    """Form credential settings to configure access credentials for the ALiYun BaiLian OMI STT model."""
    
    # Add any specific credential-related fields here

    def encryption_dict(self, model: Dict[str, object]) -> Dict[str, object]:
        """
        Generate dictionary representing encrypted model configuration.

        Parameters:
            model (Dict[str, object]): Configuration model data.

        Returns:
            Dict[str, object]: Encrypted configuration dictionary.
        """
        # Implement encryption logic here
        raise NotImplementedError("Encryption not implemented")

    def get_model_params_setting_form(self, model_name) -> FormType:
        """Get the form used to set up parameter configurations for the model."""
        
        return AliyunBaiLianOmiSTTModelParams()

Key Changes:

  • Consistent Indentation: Used 4 spaces per level of indentation.
  • Removed Duplicate Code: Unified the form settings into one AliyunBaiLianOmiSTTModelParams class.
  • Comments Improve Readability: Clarified comments and removed extra symbols like #.
  • File Cleanup: Removed trailing newline characters.
  • Added docstrings and type hints wherever appropriate for better understanding and maintainability.

These changes should help improve the overall structure and clarity of the code while maintaining its functionality.

9 changes: 5 additions & 4 deletions apps/models_provider/impl/aliyun_bai_lian_model_provider/model/asr_stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,11 @@ def speech_to_text(self, audio_file):
messages=messages,
result_format="message",
)

text = response["output"]["choices"][0]["message"].content[0]["text"]

return text
if response.status_code == 200:
text = response["output"]["choices"][0]["message"].content[0]["text"]
return text
else:
raise Exception('Error: ', response.message)

except Exception as err:
maxkb_logger.error(f":Error: {str(err)}: {traceback.format_exc()}")
Copy link
Contributor Author

@shaohuzhang1 shaohuzhang1 Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code seems to be for converting speech from an audio file to text using a translation API. The function speech_to_text handles both successful and unsuccessful responses. Here's a brief critique:

Irregularities:

  1. Logging of Error: In the exception block, logging is used without specifying log level. It would be good practice to use appropriate log levels like error, debug, etc., to better trace errors.

  2. Traceback Handling: When you capture exceptions (Exception as err), it's common to include the traceback at some point. However, the current implementation doesn't explicitly append the traceback.

  3. API Endpoint: There's no information about which endpoint this API uses. While the code suggests it involves speech-to-text translation, the exact functionality may vary based on what API is being used.

  4. Result Format: The response format "message" is specified, but not all APIs return such a formatted JSON. Ensure that the correct fields exist in the response to correctly extract the translation text.

  5. Response Status Check: A specific status code (e.g., 200) needs to be checked before attempting to access the response content in case an HTTP error occurs. Currently, only successful responses are processed.

def speech_to_text(self, audio_file):
    try:
        # Assuming 'chatGPT' is initialized elsewhere (client setup for API calls)
        response = chatGPT.complete(
            model_id=model,
            prompt=prompt,
            messages=messages,
            result_format="message"
        )

        if response.status_code == 200:
            text = response["output"]["choices"][0].get("message", {}).get("content", []).get(0).get("text")
            return text

        elif response.status_code >= 400:
            raise Exception(f"HTTP request failed with status {response.status_code}: '{response.json().get('detail', {})['message']}'")

        else:
            raise Exception("Unexpected response received")

    except Exception as err:
        maxkb_logger.error(f"Error during speech-to-text conversion process: {err}", exc_info=True)

    return None

Optimization Suggestions:

  1. Error Message Customization: Customize the error message to provide more context about why the conversion might have failed, especially when encountering non-successful status codes.

  2. Rate Limits Management: Implement rate limiting logic in production to manage API usage limits effectively.

  3. Retry Mechanism: Add a retry mechanism for transient failures, though note that excessive retries can lead to abuse detection if your service provider has ratelimiting policies.

  4. Testing: Thoroughly test various scenarios, including edge cases (e.g., handling empty input files or invalid media type).

By addressing these points, the code improves robustness and usability, making it easier to integrate into larger applications.

2 changes: 1 addition & 1 deletion apps/models_provider/impl/aliyun_bai_lian_model_provider/model/omni_stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def speech_to_text(self, audio_file):
"format": "mp3",
},
},
{"type": "text", "text": self.params.get('CueWord')},
{"type": "text", "text": '这段音频在说什么，只回答音频的内容'},
],
},
],
Expand Down
66 changes: 33 additions & 33 deletions apps/models_provider/impl/tencent_model_provider/credential/stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,38 +8,38 @@
from models_provider.base_model_provider import BaseModelCredential, ValidCode


class TencentSSTModelParams(BaseForm):
EngSerViceType = forms.SingleSelect(
TooltipLabel(_('Engine model type'), _('If not passed, the default value is 16k_zh (Chinese universal)')),
required=True,
default_value='16k_zh',
option_list=[
{"value": "8k_zh", "label": _("Chinese telephone universal")},
{"value": "8k_en", "label": _("English telephone universal")},
{"value": "16k_zh", "label": _("Commonly used in Chinese")},
{"value": "16k_zh-PY", "label": _("Chinese, English, and Guangdong")},
{"value": "16k_zh_medical", "label": _("Chinese medical")},
{"value": "16k_en", "label": _("English")},
{"value": "16k_yue", "label": _("Cantonese")},
{"value": "16k_ja", "label": _("Japanese")},
{"value": "16k_ko", "label": _("Korean")},
{"value": "16k_vi", "label": _("Vietnamese")},
{"value": "16k_ms", "label": _("Malay language")},
{"value": "16k_id", "label": _("Indonesian language")},
{"value": "16k_fil", "label": _("Filipino language")},
{"value": "16k_th", "label": _("Thai")},
{"value": "16k_pt", "label": _("Portuguese")},
{"value": "16k_tr", "label": _("Turkish")},
{"value": "16k_ar", "label": _("Arabic")},
{"value": "16k_es", "label": _("Spanish")},
{"value": "16k_hi", "label": _("Hindi")},
{"value": "16k_fr", "label": _("French")},
{"value": "16k_de", "label": _("German")},
{"value": "16k_zh_dialect", "label": _("Multiple dialects, supporting 23 dialects")}
],
value_field='value',
text_field='label'
)
# class TencentSSTModelParams(BaseForm):
# EngSerViceType = forms.SingleSelect(
# TooltipLabel(_('Engine model type'), _('If not passed, the default value is 16k_zh (Chinese universal)')),
# required=True,
# default_value='16k_zh',
# option_list=[
# {"value": "8k_zh", "label": _("Chinese telephone universal")},
# {"value": "8k_en", "label": _("English telephone universal")},
# {"value": "16k_zh", "label": _("Commonly used in Chinese")},
# {"value": "16k_zh-PY", "label": _("Chinese, English, and Guangdong")},
# {"value": "16k_zh_medical", "label": _("Chinese medical")},
# {"value": "16k_en", "label": _("English")},
# {"value": "16k_yue", "label": _("Cantonese")},
# {"value": "16k_ja", "label": _("Japanese")},
# {"value": "16k_ko", "label": _("Korean")},
# {"value": "16k_vi", "label": _("Vietnamese")},
# {"value": "16k_ms", "label": _("Malay language")},
# {"value": "16k_id", "label": _("Indonesian language")},
# {"value": "16k_fil", "label": _("Filipino language")},
# {"value": "16k_th", "label": _("Thai")},
# {"value": "16k_pt", "label": _("Portuguese")},
# {"value": "16k_tr", "label": _("Turkish")},
# {"value": "16k_ar", "label": _("Arabic")},
# {"value": "16k_es", "label": _("Spanish")},
# {"value": "16k_hi", "label": _("Hindi")},
# {"value": "16k_fr", "label": _("French")},
# {"value": "16k_de", "label": _("German")},
# {"value": "16k_zh_dialect", "label": _("Multiple dialects, supporting 23 dialects")}
# ],
# value_field='value',
# text_field='label'
# )

class TencentSTTModelCredential(BaseForm, BaseModelCredential):
REQUIRED_FIELDS = ["SecretId", "SecretKey"]
Expand Down Expand Up @@ -87,4 +87,4 @@ def encryption_dict(self, model):
SecretKey = forms.PasswordInputField('SecretKey', required=True)

def get_model_params_setting_form(self, model_name):
return TencentSSTModelParams()
pass
Copy link
Contributor Author

@shaohuzhang1 shaohuzhang1 Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code snippet appears to be incomplete, with only the opening line of TencentSSTModelParams repeated and missing closing curly brackets (}). This will likely cause an syntax error when running the code. There are also no other major issues present beyond this formatting problem.

To optimize the code, consider adding docstrings to each class method if they don't already exist. Additionally, ensure that all imported modules are properly defined at the beginning of the file.

Here's a corrected version:

from models_provider.base_model_provider import BaseModelCredential, ValidCode

class TencentSSTModelParams(BaseForm):
    EngSerViceType = forms.SingleSelect(
        TooltipLabel(_('Engine model type'), _('If not passed, the default value is 16k_zh (Chinese universal)')),
        required=True,
        default_value='16k_zh',
        option_list=[
            {"value": "8k_zh", "label": _("Chinese telephone universal")},
            {"value": "8k_en", "label": _("English telephone universal")},
            {"value": "16k_zh", "label": _("Commonly used in Chinese")},
            {"value": "16k_zh-PY", "label": _("Chinese, English, and Guangdong")},
            {"value": "16k_zh_medical", "label": _("Chinese medical")},
            {"value": "16k_en", "label": _("English")},
            {"value": "16k_yue", "label": _("Cantonese")},
            {"value": "16k_ja", "label": _("Japanese")},
            {"value": "16k_ko", "label": _("Korean")},
            {"value": "16k_vi", "label": _("Vietnamese")},
            {"value": "16k_ms", "label": _("Malay language")},
            {"value": "16k_id", "label": _("Indonesian language")},
            {"value": "16k_fil", "label": _("Filipino language")},
            {"value": "16k_th", "label": _("Thai")},
            {"value": "16k_pt", "label": _("Portuguese")},
            {"value": "16k_tr", "label": _("Turkish")},
            {"value": "16k_ar", "label": _("Arabic")},
            {"value": "16k_es", "label": _("Spanish")},
            {"value": "16k_hi", "label": _("Hindi")},
            {"value": "16k_fr", "label": _("French")},
            {"value": "16k_de", "label": _("German")},
            {"value": "16k_zh_dialect", "label": _("Multiple dialects, supporting 23 dialects")}
        ],
        value_field='value',
        text_field='label'
    )

class TencentSTTModelCredential(BaseForm, BaseModelCredential):
    REQUIRED_FIELDS = ["SecretId", "SecretKey"]

    def SecretId(self):
        """
        Returns the secret ID field.
        
        :return: The SecretId form field instance.
        """
        return forms.CharField('SecretId', required=True)

    def SecretKey(self):
        """
        Returns the secret key field.
        
        :return: The SecretKey form field instance.
        """
        return forms.PasswordInputField('SecretKey', required=True)

    def get_model_params_setting_form(self, model_name):
        """
        Retrieves the model parameters setting form based on the given model name.
        
        :param model_name: Name of the model.
        :type model_name: str
        :return: An instance of TencentSSTModelParams.
        :rtype: TencentSSTModelParams
        """
        # Implement logic to get model-specific params form
        return TencentSSTModelParams()

This should improve readability and maintainability.

2 changes: 1 addition & 1 deletion apps/models_provider/impl/tencent_model_provider/model/stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def speech_to_text(self, audio_file):
# 实例化一个请求对象,每个接口都会对应一个request对象
req = models.SentenceRecognitionRequest()
params = {
"EngSerViceType": self.params.get('EngSerViceType'),
"EngSerViceType": '16k_zh',
"SourceType": 1,
"VoiceFormat": "mp3",
"Data": _v.decode(),
Expand Down
3 changes: 2 additions & 1 deletion apps/models_provider/impl/vllm_model_provider/credential/reranker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ def is_valid(self, model_type: str, model_name, model_credential: Dict[str, obje
return False
try:
model: VllmBgeReranker = provider.get_model(model_type, model_name, model_credential)
model.compress_documents([Document(page_content=_('Hello'))], _('Hello'))
test_text = str(_('Hello'))
model.compress_documents([Document(page_content=test_text)], test_text)
except Exception as e:
traceback.print_exc()
if isinstance(e, AppApiException):
Expand Down
2 changes: 1 addition & 1 deletion apps/models_provider/impl/vllm_model_provider/vllm_model_provider.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
]

reranker_model_info_list = [
ModelInfo('bge-reranker-v2-m3', '', ModelTypeConst.RERANKER, rerank_model_credential, VllmBgeReranker),
ModelInfo('BAAI/bge-reranker-v2-m3', '', ModelTypeConst.RERANKER, rerank_model_credential, VllmBgeReranker),
]

model_info_manage = (
Expand Down
Loading