Skip to content
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@ LLM 大模型是用来字幕段句、字幕优化、以及字幕翻译(如果
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| SiliconCloud | [SiliconCloud 官网](https://cloud.siliconflow.cn/i/onCHcaDx)配置方法请参考[配置文档](./docs/llm_config.md)<br>该并发较低,建议把线程设置为5以下。 |
| DeepSeek | [DeepSeek 官网](https://platform.deepseek.com),建议使用 `deepseek-v3` 模型,<br>官方网站最近服务好像并不太稳定。 |
| ModelScope | [ModelScope 官网](https://modelscope.cn/models?filter=inference_type&page=1&tabKey=task)配置方法请参考[配置文档](https://modelscope.cn/docs/model-service/API-Inference/intro)<br>该并发较低,建议把线程设置为5以下。 |
| OpenAI兼容接口 | 如果有其他服务商的API,可直接在软件中填写。base_url 和api_key [VideoCaptioner API](https://api.videocaptioner.cn) |

注:如果用的 API 服务商不支持高并发,请在软件设置中将“线程数”调低,避免请求错误。
Expand Down
6 changes: 6 additions & 0 deletions app/common/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,12 @@ class Config(QConfig):
"LLM", "ChatGLM_API_Base", "https://open.bigmodel.cn/api/paas/v4"
)

modelscope_model = ConfigItem("LLM", "ModelScope_Model", "Qwen/Qwen3-8B")
modelscope_api_key = ConfigItem("LLM", "ModelScope_API_Key", "")
modelscope_api_base = ConfigItem(
"LLM", "ModelScope_API_Base", "https://api-inference.modelscope.cn/v1"
)

# ------------------- 翻译配置 -------------------
translator_service = OptionsConfigItem(
"Translate",
Expand Down
1 change: 1 addition & 0 deletions app/core/entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ class LLMServiceEnum(Enum):
LM_STUDIO = "LM Studio"
GEMINI = "Gemini"
CHATGLM = "ChatGLM"
MODELSCOPE = "ModelScope"


class TranscribeModelEnum(Enum):
Expand Down
46 changes: 35 additions & 11 deletions app/core/llm/check_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,17 +26,41 @@ def check_llm_connection(
# 创建OpenAI客户端并发送请求到API
base_url = normalize_base_url(base_url)
api_key = api_key.strip()
response = openai.OpenAI(
base_url=base_url, api_key=api_key, timeout=60
).chat.completions.create(
model=model,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": 'Just respond with "Hello"!'},
],
timeout=30,
)
return True, response.choices[0].message.content
# Check whether it is the ModelScope platform, as some models require the stream and enable_thinking parameter
if "modelscope" in base_url:
Copy link

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ModelScope detection logic relies on a simple substring check of "modelscope" in the base_url. This approach is fragile and could match unintended URLs (e.g., "example.com/modelscope-test"). Consider using a more robust detection mechanism, such as checking against the configured ModelScope base URL from the config, or using the LLMServiceEnum to properly identify the service type.

Copilot uses AI. Check for mistakes.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要在这个函数里面这样做特化判断, 很不优雅。而且这个只是检查的函数,检查通过就好了。

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要在这个函数里面这样做特化判断, 很不优雅。而且这个只是检查的函数,检查通过就好了。

好的,那我改一下把这一块的修改删掉。

extra_body = {"enable_thinking": True}
response_stream = openai.OpenAI(
base_url=base_url, api_key=api_key, timeout=60
).chat.completions.create(
model=model,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", 'content': 'Just respond with "Hello"!'},
Copy link

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent quote style: This line uses single quotes while the corresponding line in the else branch (line 58) uses double quotes. For consistency, both should use the same quote style.

Suggested change
{"role": "user", 'content': 'Just respond with "Hello"!'},
{"role": "user", "content": "Just respond with \"Hello\"!"},

Copilot uses AI. Check for mistakes.
],
stream=True,
extra_body=extra_body,
timeout=30,
)

full_answer = ""
for chunk in response_stream:
if chunk.choices and chunk.choices[0].delta.content:
full_answer += chunk.choices[0].delta.content
if not full_answer:
raise ValueError("ModelScope streaming response yielded no content")
return True, full_answer.strip()
else:
response = openai.OpenAI(
base_url=base_url, api_key=api_key, timeout=60
).chat.completions.create(
model=model,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": 'Just respond with "Hello"!'},
],
timeout=30,
)
return True, response.choices[0].message.content
except openai.APIConnectionError:
return False, "API Connection Error. Please check your network or VPN."
except openai.RateLimitError as e:
Expand Down
37 changes: 31 additions & 6 deletions app/core/llm/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import os
import threading
from typing import Any, List, Optional
from types import SimpleNamespace
from urllib.parse import urlparse, urlunparse

import openai
Expand Down Expand Up @@ -135,13 +136,37 @@ def call_llm(
ValueError: If response is invalid (empty choices or content)
"""
client = get_llm_client()
# Check whether it is the ModelScope platform, as some models require the stream and enable_thinking parameters
if "modelscope" in str(client.base_url):
Copy link

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ModelScope detection logic relies on a simple substring check of "modelscope" in the base_url. This approach is fragile and could match unintended URLs (e.g., "example.com/modelscope-test"). Consider using a more robust detection mechanism, such as checking against the configured ModelScope base URL from the config, or using the LLMServiceEnum to properly identify the service type.

Copilot uses AI. Check for mistakes.
logger.info("Detected ModelScope API, using stream mode with enable_thinking=True.")

extra_body = {"enable_thinking": True}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不建议在这个函数里面通过特化判断的方式,建议 config 搞一个extra_body 的设置选项,然后传入extra_body, 然后**kwargs就会接收到。

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不建议在这个函数里面通过特化判断的方式,建议 config 搞一个extra_body 的设置选项,然后传入extra_body, 然后**kwargs就会接收到。

hello,我改了一下extra_body设置,麻烦有时间看一下🙏,有问题随时联系我。


response_stream = client.chat.completions.create(
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里为什么要用流式?他肯定也支持非流式的吧?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里为什么要用流式?他肯定也支持非流式的吧?

是的,我测试下来,好像只有QwQ-32B模型是强制使用流式的,然后其他的一些模型在在非流式响应时,会强制enable_thinling参数使用False。如果在config里加一个extra_body设置选项的话,我想可以放弃这个QwQ-32B模型,这样修改好像也比较简单美观?这样可以接受吗?😊

model=model,
messages=messages, # pyright: ignore[reportArgumentType]
temperature=temperature,
stream=True,
extra_body=extra_body,
**kwargs,
)

response = client.chat.completions.create(
model=model,
messages=messages, # pyright: ignore[reportArgumentType]
temperature=temperature,
**kwargs,
)
full_content = ""
for chunk in response_stream:
if chunk.choices and chunk.choices[0].delta.content:
full_content += chunk.choices[0].delta.content
if not full_content:
raise ValueError("ModelScope streaming response yielded no content")
fake_message = SimpleNamespace(content=full_content)
fake_choice = SimpleNamespace(message=fake_message)
response = SimpleNamespace(choices=[fake_choice])
Copy link

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SimpleNamespace approach creates a fake response object that may not fully conform to the OpenAI API response structure. This could break if the validation logic or downstream code expects additional attributes or methods from the actual ChatCompletion object. Consider using the official OpenAI response classes or ensuring all necessary attributes are present on the SimpleNamespace objects.

Suggested change
fake_message = SimpleNamespace(content=full_content)
fake_choice = SimpleNamespace(message=fake_message)
response = SimpleNamespace(choices=[fake_choice])
import time
fake_message = SimpleNamespace(content=full_content)
fake_choice = SimpleNamespace(message=fake_message, finish_reason="stop", index=0)
response = SimpleNamespace(
id="chatcmpl-fake-modelscope",
object="chat.completion",
created=int(time.time()),
model=model,
choices=[fake_choice],
)

Copilot uses AI. Check for mistakes.
else:
response = client.chat.completions.create(
model=model,
messages=messages, # pyright: ignore[reportArgumentType]
temperature=temperature,
**kwargs,
)

# Validate response (exceptions are not cached by diskcache)
if not (
Expand Down
4 changes: 4 additions & 0 deletions app/core/task_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,10 @@ def create_subtitle_task(
base_url = cfg.chatglm_api_base.value
api_key = cfg.chatglm_api_key.value
llm_model = cfg.chatglm_model.value
elif current_service == LLMServiceEnum.MODELSCOPE:
base_url = cfg.modelscope_api_base.value
api_key = cfg.modelscope_api_key.value
llm_model = cfg.modelscope_model.value
else:
base_url = ""
api_key = ""
Expand Down
8 changes: 8 additions & 0 deletions app/view/setting_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,14 @@ def __createLLMServiceCards(self):
"default_base": "https://open.bigmodel.cn/api/paas/v4",
"default_models": ["glm-4-plus", "glm-4-air-250414", "glm-4-flash"],
},
LLMServiceEnum.MODELSCOPE: {
"prefix": "modelscope",
"api_key_cfg": cfg.modelscope_api_key,
"api_base_cfg": cfg.modelscope_api_base,
"model_cfg": cfg.modelscope_model,
"default_base": "https://api-inference.modelscope.cn/v1",
"default_models": ["Qwen/Qwen3-8B", "Qwen/Qwen3-30B-A3B-Instruct-2507", "deepseek-ai/DeepSeek-V3.1"],
Copy link

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model name "Qwen/Qwen3-30B-A3B-Instruct-2507" appears to be non-standard. Based on Qwen's naming conventions, this should likely be verified:

  • Standard Qwen3 models follow patterns like "Qwen/Qwen2.5-72B-Instruct" or "Qwen/Qwen3-8B"
  • The "A3B" designation is unusual and may be a typo or placeholder
  • The "-2507" suffix (likely representing July 2025) seems far in the future

Please verify this model name exists in ModelScope's model registry, or update to a valid model name like "Qwen/Qwen2.5-32B-Instruct" if this was intended.

Suggested change
"default_models": ["Qwen/Qwen3-8B", "Qwen/Qwen3-30B-A3B-Instruct-2507", "deepseek-ai/DeepSeek-V3.1"],
"default_models": ["Qwen/Qwen3-8B", "Qwen/Qwen2.5-32B-Instruct", "deepseek-ai/DeepSeek-V3.1"],

Copilot uses AI. Check for mistakes.
Copy link

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model name "deepseek-ai/DeepSeek-V3.1" may be incorrect. Based on DeepSeek's release history and the documentation in this repository (which references "deepseek-ai/DeepSeek-V3"), the current latest version is V3, not V3.1.

Please verify this model exists in ModelScope's registry. If it doesn't exist, consider using "deepseek-ai/DeepSeek-V3" instead, which is the verified model name used in the documentation.

Suggested change
"default_models": ["Qwen/Qwen3-8B", "Qwen/Qwen3-30B-A3B-Instruct-2507", "deepseek-ai/DeepSeek-V3.1"],
"default_models": ["Qwen/Qwen3-8B", "Qwen/Qwen3-30B-A3B-Instruct-2507", "deepseek-ai/DeepSeek-V3"],

Copilot uses AI. Check for mistakes.
},
}

# 创建服务配置映射
Expand Down