-
Notifications
You must be signed in to change notification settings - Fork 3
Expand file tree
/
Copy pathconfig_model.yaml
More file actions
138 lines (132 loc) · 6.09 KB
/
config_model.yaml
File metadata and controls
138 lines (132 loc) · 6.09 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
# Medical AI Model Configuration
#
# ⚠️ IMPORTANT: OpenAI-Compatible API Only
# This system currently supports ONLY OpenAI-compatible API interfaces.
# All models must provide OpenAI-compatible endpoints, regardless of the actual provider.
#
# 🔧 API Configuration Guide
# Step 1: Set environment variables (Linux/Mac):
# export OPENAI_API_KEY="sk-xxx"
# export OPENAI_BASE_URL="https://api.openai.com/v1"
# Step 2: Windows PowerShell:
# $Env:OPENAI_API_KEY="sk-xxx"
#
# 🌐 Third-Party Multi-Model Platforms (Recommended):
# These platforms provide OpenAI-compatible APIs for multiple models:
#
# • OpenRouter (https://openrouter.ai/)
# - Supports Claude, Gemini, GPT, and many other models via OpenAI-compatible API
# - Example: export OPENROUTER_API_KEY="sk-or-xxx"
# - Base URL: https://openrouter.ai/api/v1
#
# • Together AI (https://together.ai/)
# - Supports various open-source and commercial models
# - Example: export TOGETHER_API_KEY="xxx"
# - Base URL: https://api.together.xyz/v1
#
# • Anyscale (https://anyscale.com/)
# - Supports multiple LLMs via OpenAI-compatible endpoints
# - Example: export ANYSCALE_API_KEY="xxx"
# - Base URL: https://api.endpoints.anyscale.com/v1
#
# 📋 Supported Providers (OpenAI-Compatible Only):
#┌─────────────────────────────────────────────────────────────────────────────────┐
#│ Provider │ api_key_env │ api_base_env │ Example Model │ Status │
#├─────────────────────────────────────────────────────────────────────────────────┤
#│ OpenAI │ OPENAI_API_KEY │ OPENAI_BASE_URL │ gpt-4-turbo │ ✅ Native │
#│ OpenRouter│ OPENROUTER_API_KEY │ OPENROUTER_BASE_URL │ anthropic/claude-3.5 │ ✅ Multi-Model │
#│ Together │ TOGETHER_API_KEY │ TOGETHER_BASE_URL │ meta-llama/llama-3 │ ✅ Multi-Model │
#│ Anyscale │ ANYSCALE_API_KEY │ ANYSCALE_BASE_URL │ mistralai/mixtral │ ✅ Multi-Model │
#│ Kimi │ MOONSHOT_API_KEY │ MOONSHOT_BASE_URL │ kimi-latest │ ✅ Compatible │
#│ Qwen │ DASHSCOPE_API_KEY │ DASHSCOPE_BASE_URL │ qwen-max │ ✅ Compatible │
#│ DeepSeek │ DEEPSEEK_API_KEY │ DEEPSEEK_BASE_URL │ deepseek-chat │ ✅ Compatible │
#│ Baichuan │ BAICHUAN_API_KEY │ BAICHUAN_BASE_URL │ baichuan-53b │ ✅ Compatible │
#│ Zhipu │ ZHIPU_API_KEY │ ZHIPU_BASE_URL │ chatglm-4 │ ✅ Compatible │
#│ Claude │ ANTHROPIC_API_KEY │ ANTHROPIC_BASE_URL │ claude-3.5 │ ❌ Not Supported │
#│ Gemini │ GOOGLE_API_KEY │ GOOGLE_BASE_URL │ gemini-1.5 │ ❌ Not Supported │
#└─────────────────────────────────────────────────────────────────────────────────┘
#
# ⚠️ Technical Implementation Note:
# - The system uses LangChain's ChatOpenAI client for ALL models
# - All API calls are made through OpenAI-compatible interfaces
# - Native APIs (like Anthropic's Claude API or Google's Gemini API) are NOT supported
# - Providers must offer OpenAI-compatible endpoints to work with this system
# - Third-party platforms like OpenRouter enable access to models that don't natively support OpenAI API
#
# 🎯 Medical Model Recommendations (OpenAI-Compatible):
# • Diagnostic tasks: gpt-4-turbo, deepseek-chat (high accuracy)
# • Chinese medical: qwen-max, baichuan-medical (better CJK support)
# • Cost-effective: deepseek-chat, kimi (good balance)
# • Multi-model access: OpenRouter (Claude, Gemini via compatible API)
# • Local deployment: local-llama3-medical (privacy-focused, via OpenAI-compatible server)
#
# 💡 Example Configurations:
#
# # OpenRouter - Access Claude via OpenAI-compatible API
# claude_via_openrouter:
# api_key_env: "OPENROUTER_API_KEY"
# api_base_env: "OPENROUTER_BASE_URL" # https://openrouter.ai/api/v1
# model: "anthropic/claude-3-5-sonnet-20241022"
# temperature: 0.2
# max_tokens: 4000
#
# # OpenRouter - Access Gemini via OpenAI-compatible API
# gemini_via_openrouter:
# api_key_env: "OPENROUTER_API_KEY"
# api_base_env: "OPENROUTER_BASE_URL" # https://openrouter.ai/api/v1
# model: "google/gemini-pro-1.5"
# temperature: 0.1
# max_tokens: 4000
#
models:
m1:
api_key_env: "ANTCHAT_API_KEY"
api_base_env: "ANTCHAT_BASE_URL"
model: "Kimi-K2-Instruct"
temperature: 0
max_tokens: 32678
extra_body:
enable_sec_check: false
m2:
api_key_env: "ANTCHAT_API_KEY"
api_base_env: "ANTCHAT_BASE_URL"
model: "Qwen3-235B-A22B-Instruct-2507"
temperature: 0
max_tokens: 32678
extra_body:
top_k: 20
min_p: 0
enable_sec_check: false
m3:
api_key_env: "ANTCHAT_API_KEY"
api_base_env: "ANTCHAT_BASE_URL"
model: "DeepSeek-V3.1"
temperature: 0
max_tokens: 32678
m4:
api_key_env: "MATRIX_API_KEY"
api_base_env: "MATRIX_BASE_URL"
model: "gpt-4.1-2025-04-14"
temperature: 0
max_tokens: 32678
m5:
api_key_env: "MATRIX_API_KEY_V2"
api_base_env: "MATRIX_BASE_URL"
model: "gemini-2.5-pro"
reasoning_effort: "high"
max_tokens: 32678
temperature: 0.7
# Example: Claude via OpenRouter (commented out - uncomment to use)
# m6:
# api_key_env: "OPENROUTER_API_KEY"
# api_base_env: "OPENROUTER_BASE_URL"
# model: "anthropic/claude-3-5-sonnet-20241022"
# temperature: 0.2
# max_tokens: 4000
# Example: Gemini via OpenRouter (commented out - uncomment to use)
# m7:
# api_key_env: "OPENROUTER_API_KEY"
# api_base_env: "OPENROUTER_BASE_URL"
# model: "google/gemini-pro-1.5"
# temperature: 0.1
# max_tokens: 4000