Skip to content

Commit d82d463

Browse files
tbphptbphp
committed
feat: 优化超时配置
1 parent 3232d34 commit d82d463

File tree

8 files changed

+137
-80
lines changed

8 files changed

+137
-80
lines changed

‎.env.example‎

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,9 +32,6 @@ MAX_RETRIES=3
3232
# 上游 API 地址
3333
OPENAI_BASE_URL=https://api.openai.com
3434

35-
# 请求超时时间（毫秒）
36-
REQUEST_TIMEOUT=30000
37-
3835
# ===========================================
3936
# 性能优化配置
4037
# ===========================================
@@ -76,3 +73,30 @@ ENABLE_CORS=true
7673

7774
# 允许的来源（逗号分隔，* 表示允许所有）
7875
ALLOWED_ORIGINS=*
76+
ALLOWED_METHODS=GET,POST,PUT,DELETE,OPTIONS
77+
ALLOWED_HEADERS=*
78+
ALLOW_CREDENTIALS=false
79+
80+
# ===========================================
81+
# 超时配置
82+
# ===========================================
83+
# 服务器读取超时时间（秒）
84+
SERVER_READ_TIMEOUT=120
85+
86+
# 服务器写入超时时间（秒）
87+
SERVER_WRITE_TIMEOUT=1800
88+
89+
# 服务器空闲超时时间（秒）
90+
SERVER_IDLE_TIMEOUT=120
91+
92+
# 服务器优雅关闭超时时间（秒）
93+
SERVER_GRACEFUL_SHUTDOWN_TIMEOUT=60
94+
95+
# 请求超时时间（秒）
96+
REQUEST_TIMEOUT=30
97+
98+
# 响应超时时间（秒）- 控制TLS握手和响应头接收超时
99+
RESPONSE_TIMEOUT=30
100+
101+
# 空闲连接超时时间（秒）- 控制连接池中空闲连接的生存时间
102+
IDLE_CONN_TIMEOUT=120

‎README.md‎

Lines changed: 27 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -92,18 +92,23 @@ cp .env.example .env
9292

9393
### Key Configuration Options
9494

95-
| Setting | Environment Variable | Default | Description |
96-
| ------------------- | --------------------- | ------------------------ | ------------------------------- |
97-
| Server Port | `PORT` | 3000 | Server listening port |
98-
| Server Host | `HOST` | 0.0.0.0 | Server binding address |
99-
| Keys File | `KEYS_FILE` | keys.txt | API keys file path |
100-
| Start Index | `START_INDEX` | 0 | Starting key index for rotation |
101-
| Blacklist Threshold | `BLACKLIST_THRESHOLD` | 1 | Error count before blacklisting |
102-
| Upstream URL | `OPENAI_BASE_URL` | `https://api.openai.com` | OpenAI-compatible API base URL. Supports multiple, comma-separated URLs for load balancing. |
103-
| Request Timeout | `REQUEST_TIMEOUT` | 30000 | Request timeout in milliseconds |
104-
| Auth Key | `AUTH_KEY` | - | Optional authentication key |
105-
| CORS | `ENABLE_CORS` | true | Enable CORS support |
106-
| Max Connections | `MAX_SOCKETS` | 50 | Maximum HTTP connections |
95+
| Setting | Environment Variable | Default | Description |
96+
| ----------------------- | ---------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------- |
97+
| Server Port | `PORT` | 3000 | Server listening port |
98+
| Server Host | `HOST` | 0.0.0.0 | Server binding address |
99+
| Keys File | `KEYS_FILE` | keys.txt | API keys file path |
100+
| Start Index | `START_INDEX` | 0 | Starting key index for rotation |
101+
| Blacklist Threshold | `BLACKLIST_THRESHOLD` | 1 | Error count before blacklisting |
102+
| Upstream URL | `OPENAI_BASE_URL` | `https://api.openai.com` | OpenAI-compatible API base URL. Supports multiple, comma-separated URLs for load balancing. |
103+
| Auth Key | `AUTH_KEY` | - | Optional authentication key |
104+
| CORS | `ENABLE_CORS` | true | Enable CORS support |
105+
| Server Read Timeout | `SERVER_READ_TIMEOUT` | 120 | HTTP server read timeout in seconds |
106+
| Server Write Timeout | `SERVER_WRITE_TIMEOUT` | 1800 | HTTP server write timeout in seconds |
107+
| Server Idle Timeout | `SERVER_IDLE_TIMEOUT` | 120 | HTTP server idle timeout in seconds |
108+
| Graceful Shutdown | `SERVER_GRACEFUL_SHUTDOWN_TIMEOUT` | 60 | Graceful shutdown timeout in seconds |
109+
| Request Timeout | `REQUEST_TIMEOUT` | 30 | Request timeout in seconds |
110+
| Response Timeout | `RESPONSE_TIMEOUT` | 30 | Response timeout in seconds (TLS handshake & response header) |
111+
| Idle Connection Timeout | `IDLE_CONN_TIMEOUT` | 120 | Idle connection timeout in seconds |
107112

108113
### Configuration Examples
109114

@@ -126,16 +131,16 @@ OPENAI_BASE_URL=https://your-resource.openai.azure.com
126131
```bash
127132
OPENAI_BASE_URL=https://api.your-provider.com
128133
# Use provider-specific API keys
129-
```
130-
131-
#### Multi-Target Load Balancing
132-
133-
```bash
134-
# Use a comma-separated list of target URLs
135-
OPENAI_BASE_URL=https://gateway.ai.cloudflare.com/v1/.../openai,https://api.openai.com/v1,https://api.another-provider.com/v1
136-
```
137-
138-
## API Key Validation
134+
```
135+
136+
#### Multi-Target Load Balancing
137+
138+
```bash
139+
# Use a comma-separated list of target URLs
140+
OPENAI_BASE_URL=https://gateway.ai.cloudflare.com/v1/.../openai,https://api.openai.com/v1,https://api.another-provider.com/v1
141+
```
142+
143+
## API Key Validation
139144

140145
The project includes a high-performance API key validation tool:
141146

‎README_CN.md‎

Lines changed: 27 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -92,18 +92,23 @@ cp .env.example .env
9292

9393
### 主要配置选项
9494

95-
| 配置项 | 环境变量 | 默认值 | 说明 |
96-
| ---------- | --------------------- | ------------------------ | ------------------------ |
97-
| 服务器端口 | `PORT` | 3000 | 服务器监听端口 |
98-
| 服务器主机 | `HOST` | 0.0.0.0 | 服务器绑定地址 |
99-
| 密钥文件 | `KEYS_FILE` | keys.txt | API 密钥文件路径 |
100-
| 起始索引 | `START_INDEX` | 0 | 密钥轮换起始索引 |
101-
| 拉黑阈值 | `BLACKLIST_THRESHOLD` | 1 | 拉黑前的错误次数 |
102-
| 上游地址 | `OPENAI_BASE_URL` | `https://api.openai.com` | OpenAI 兼容 API 基础地址。支持多个地址，用逗号分隔 |
103-
| 请求超时 | `REQUEST_TIMEOUT` | 30000 | 请求超时时间（毫秒） |
104-
| 认证密钥 | `AUTH_KEY` | - | 可选的认证密钥 |
105-
| CORS | `ENABLE_CORS` | true | 启用 CORS 支持 |
106-
| 最大连接数 | `MAX_SOCKETS` | 50 | 最大 HTTP 连接数 |
95+
| 配置项 | 环境变量 | 默认值 | 说明 |
96+
| -------------- | ---------------------------------- | ------------------------ | -------------------------------------------------- |
97+
| 服务器端口 | `PORT` | 3000 | 服务器监听端口 |
98+
| 服务器主机 | `HOST` | 0.0.0.0 | 服务器绑定地址 |
99+
| 密钥文件 | `KEYS_FILE` | keys.txt | API 密钥文件路径 |
100+
| 起始索引 | `START_INDEX` | 0 | 密钥轮换起始索引 |
101+
| 拉黑阈值 | `BLACKLIST_THRESHOLD` | 1 | 拉黑前的错误次数 |
102+
| 上游地址 | `OPENAI_BASE_URL` | `https://api.openai.com` | OpenAI 兼容 API 基础地址。支持多个地址，用逗号分隔 |
103+
| 认证密钥 | `AUTH_KEY` | - | 可选的认证密钥 |
104+
| CORS | `ENABLE_CORS` | true | 启用 CORS 支持 |
105+
| 服务器读取超时 | `SERVER_READ_TIMEOUT` | 120 | HTTP 服务器读取超时时间（秒） |
106+
| 服务器写入超时 | `SERVER_WRITE_TIMEOUT` | 1800 | HTTP 服务器写入超时时间（秒） |
107+
| 服务器空闲超时 | `SERVER_IDLE_TIMEOUT` | 120 | HTTP 服务器空闲超时时间（秒） |
108+
| 优雅关闭超时 | `SERVER_GRACEFUL_SHUTDOWN_TIMEOUT` | 60 | 服务器优雅关闭超时时间（秒） |
109+
| 请求超时 | `REQUEST_TIMEOUT` | 30 | 请求超时时间（秒） |
110+
| 响应超时 | `RESPONSE_TIMEOUT` | 30 | 响应超时时间（秒）- 控制 TLS 握手和响应头接收 |
111+
| 空闲连接超时 | `IDLE_CONN_TIMEOUT` | 120 | 空闲连接超时时间（秒） |
107112

108113
### 配置示例
109114

@@ -126,16 +131,16 @@ OPENAI_BASE_URL=https://your-resource.openai.azure.com
126131
```bash
127132
OPENAI_BASE_URL=https://api.your-provider.com
128133
# 使用提供商特定的 API 密钥
129-
```
130-
131-
#### 多目标负载均衡
132-
133-
```bash
134-
# 使用逗号分隔多个目标地址
135-
OPENAI_BASE_URL=https://gateway.ai.cloudflare.com/v1/.../openai,https://api.openai.com/v1,https://api.another-provider.com/v1
136-
```
137-
138-
## API 密钥验证
134+
```
135+
136+
#### 多目标负载均衡
137+
138+
```bash
139+
# 使用逗号分隔多个目标地址
140+
OPENAI_BASE_URL=https://gateway.ai.cloudflare.com/v1/.../openai,https://api.openai.com/v1,https://api.another-provider.com/v1
141+
```
142+
143+
## API 密钥验证
139144

140145
项目包含高性能的 API 密钥验证工具：
141146

‎cmd/gpt-load/main.go‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -61,10 +61,10 @@ func main() {
6161
server := &http.Server{
6262
Addr: fmt.Sprintf("%s:%d", serverConfig.Host, serverConfig.Port),
6363
Handler: router,
64-
ReadTimeout: 120 * time.Second, // Increased read timeout for large file uploads
65-
WriteTimeout: 1800 * time.Second, // Increased write timeout for streaming responses
66-
IdleTimeout: 120 * time.Second, // Increased idle timeout for connection reuse
67-
MaxHeaderBytes: 1 << 20, // 1MB header limit
64+
ReadTimeout: time.Duration(serverConfig.ReadTimeout) * time.Second,
65+
WriteTimeout: time.Duration(serverConfig.WriteTimeout) * time.Second,
66+
IdleTimeout: time.Duration(serverConfig.IdleTimeout) * time.Second,
67+
MaxHeaderBytes: 1 << 20, // 1MB header limit
6868
}
6969

7070
// Start server
@@ -89,7 +89,7 @@ func main() {
8989
logrus.Info("Shutting down server...")
9090

9191
// Give outstanding requests a deadline for completion
92-
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
92+
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(serverConfig.GracefulShutdownTimeout)*time.Second)
9393
defer cancel()
9494

9595
// Attempt graceful shutdown
@@ -203,7 +203,9 @@ func displayStartupInfo(configManager types.ConfigManager) {
203203
logrus.Infof(" Blacklist threshold: %d errors", keysConfig.BlacklistThreshold)
204204
logrus.Infof(" Max retries: %d", keysConfig.MaxRetries)
205205
logrus.Infof(" Upstream URL: %s", openaiConfig.BaseURL)
206-
logrus.Infof(" Request timeout: %dms", openaiConfig.Timeout)
206+
logrus.Infof(" Request timeout: %ds", openaiConfig.RequestTimeout)
207+
logrus.Infof(" Response timeout: %ds", openaiConfig.ResponseTimeout)
208+
logrus.Infof(" Idle connection timeout: %ds", openaiConfig.IdleConnTimeout)
207209

208210
authStatus := "disabled"
209211
if authConfig.Enabled {

‎internal/config/manager.go‎

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ type Constants struct {
3030
var DefaultConstants = Constants{
3131
MinPort: 1,
3232
MaxPort: 65535,
33-
MinTimeout: 1000,
34-
DefaultTimeout: 30000,
33+
MinTimeout: 1,
34+
DefaultTimeout: 30,
3535
DefaultMaxSockets: 50,
3636
DefaultMaxFreeSockets: 10,
3737
}
@@ -62,8 +62,12 @@ func NewManager() (types.ConfigManager, error) {
6262

6363
config := &Config{
6464
Server: types.ServerConfig{
65-
Port: parseInteger(os.Getenv("PORT"), 3000),
66-
Host: getEnvOrDefault("HOST", "0.0.0.0"),
65+
Port: parseInteger(os.Getenv("PORT"), 3000),
66+
Host: getEnvOrDefault("HOST", "0.0.0.0"),
67+
ReadTimeout: parseInteger(os.Getenv("SERVER_READ_TIMEOUT"), 120),
68+
WriteTimeout: parseInteger(os.Getenv("SERVER_WRITE_TIMEOUT"), 1800),
69+
IdleTimeout: parseInteger(os.Getenv("SERVER_IDLE_TIMEOUT"), 120),
70+
GracefulShutdownTimeout: parseInteger(os.Getenv("SERVER_GRACEFUL_SHUTDOWN_TIMEOUT"), 60),
6771
},
6872
Keys: types.KeysConfig{
6973
FilePath: getEnvOrDefault("KEYS_FILE", "keys.txt"),
@@ -72,8 +76,10 @@ func NewManager() (types.ConfigManager, error) {
7276
MaxRetries: parseInteger(os.Getenv("MAX_RETRIES"), 3),
7377
},
7478
OpenAI: types.OpenAIConfig{
75-
BaseURLs: parseArray(os.Getenv("OPENAI_BASE_URL"), []string{"https://api.openai.com"}),
76-
Timeout: parseInteger(os.Getenv("REQUEST_TIMEOUT"), DefaultConstants.DefaultTimeout),
79+
BaseURLs: parseArray(os.Getenv("OPENAI_BASE_URL"), []string{"https://api.openai.com"}),
80+
RequestTimeout: parseInteger(os.Getenv("REQUEST_TIMEOUT"), DefaultConstants.DefaultTimeout),
81+
ResponseTimeout: parseInteger(os.Getenv("RESPONSE_TIMEOUT"), 30),
82+
IdleConnTimeout: parseInteger(os.Getenv("IDLE_CONN_TIMEOUT"), 120),
7783
},
7884
Auth: types.AuthConfig{
7985
Key: os.Getenv("AUTH_KEY"),
@@ -88,7 +94,6 @@ func NewManager() (types.ConfigManager, error) {
8894
},
8995
Performance: types.PerformanceConfig{
9096
MaxConcurrentRequests: parseInteger(os.Getenv("MAX_CONCURRENT_REQUESTS"), 100),
91-
RequestTimeout: parseInteger(os.Getenv("REQUEST_TIMEOUT"), DefaultConstants.DefaultTimeout),
9297
EnableGzip: parseBoolean(os.Getenv("ENABLE_GZIP"), true),
9398
},
9499
Log: types.LogConfig{
@@ -173,8 +178,8 @@ func (m *Manager) Validate() error {
173178
}
174179

175180
// Validate timeout
176-
if m.config.OpenAI.Timeout < DefaultConstants.MinTimeout {
177-
validationErrors = append(validationErrors, fmt.Sprintf("request timeout cannot be less than %dms", DefaultConstants.MinTimeout))
181+
if m.config.OpenAI.RequestTimeout < DefaultConstants.MinTimeout {
182+
validationErrors = append(validationErrors, fmt.Sprintf("request timeout cannot be less than %ds", DefaultConstants.MinTimeout))
178183
}
179184

180185
// Validate upstream URL format
@@ -212,7 +217,9 @@ func (m *Manager) DisplayConfig() {
212217
logrus.Infof(" Blacklist threshold: %d errors", m.config.Keys.BlacklistThreshold)
213218
logrus.Infof(" Max retries: %d", m.config.Keys.MaxRetries)
214219
logrus.Infof(" Upstream URLs: %s", strings.Join(m.config.OpenAI.BaseURLs, ", "))
215-
logrus.Infof(" Request timeout: %dms", m.config.OpenAI.Timeout)
220+
logrus.Infof(" Request timeout: %ds", m.config.OpenAI.RequestTimeout)
221+
logrus.Infof(" Response timeout: %ds", m.config.OpenAI.ResponseTimeout)
222+
logrus.Infof(" Idle connection timeout: %ds", m.config.OpenAI.IdleConnTimeout)
216223

217224
authStatus := "disabled"
218225
if m.config.Auth.Enabled {

‎internal/handler/handler.go‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -176,8 +176,10 @@ func (h *Handler) GetConfig(c *gin.Context) {
176176
"max_retries": keysConfig.MaxRetries,
177177
},
178178
"openai": gin.H{
179-
"base_url": openaiConfig.BaseURL,
180-
"timeout": openaiConfig.Timeout,
179+
"base_url": openaiConfig.BaseURL,
180+
"request_timeout": openaiConfig.RequestTimeout,
181+
"response_timeout": openaiConfig.ResponseTimeout,
182+
"idle_conn_timeout": openaiConfig.IdleConnTimeout,
181183
},
182184
"auth": gin.H{
183185
"enabled": authConfig.Enabled,
@@ -192,9 +194,17 @@ func (h *Handler) GetConfig(c *gin.Context) {
192194
},
193195
"performance": gin.H{
194196
"max_concurrent_requests": perfConfig.MaxConcurrentRequests,
195-
"request_timeout": perfConfig.RequestTimeout,
196197
"enable_gzip": perfConfig.EnableGzip,
197198
},
199+
"timeout_config": gin.H{
200+
"request_timeout_s": openaiConfig.RequestTimeout,
201+
"response_timeout_s": openaiConfig.ResponseTimeout,
202+
"idle_conn_timeout_s": openaiConfig.IdleConnTimeout,
203+
"server_read_timeout_s": serverConfig.ReadTimeout,
204+
"server_write_timeout_s": serverConfig.WriteTimeout,
205+
"server_idle_timeout_s": serverConfig.IdleTimeout,
206+
"graceful_shutdown_timeout_s": serverConfig.GracefulShutdownTimeout,
207+
},
198208
"log": gin.H{
199209
"level": logConfig.Level,
200210
"format": logConfig.Format,

‎internal/proxy/server.go‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -53,14 +53,13 @@ func NewProxyServer(keyManager types.KeyManager, configManager types.ConfigManag
5353
openaiConfig := configManager.GetOpenAIConfig()
5454
perfConfig := configManager.GetPerformanceConfig()
5555

56-
5756
// Create high-performance HTTP client
5857
transport := &http.Transport{
5958
MaxIdleConns: 100,
6059
MaxIdleConnsPerHost: 20,
6160
MaxConnsPerHost: 0,
62-
IdleConnTimeout: 120 * time.Second,
63-
TLSHandshakeTimeout: 30 * time.Second,
61+
IdleConnTimeout: time.Duration(openaiConfig.IdleConnTimeout) * time.Second,
62+
TLSHandshakeTimeout: time.Duration(openaiConfig.ResponseTimeout) * time.Second,
6463
ExpectContinueTimeout: 1 * time.Second,
6564
DisableCompression: !perfConfig.EnableGzip,
6665
ForceAttemptHTTP2: true,
@@ -73,19 +72,19 @@ func NewProxyServer(keyManager types.KeyManager, configManager types.ConfigManag
7372
MaxIdleConns: 200,
7473
MaxIdleConnsPerHost: 40,
7574
MaxConnsPerHost: 0,
76-
IdleConnTimeout: 300 * time.Second, // Keep streaming connections longer
77-
TLSHandshakeTimeout: 30 * time.Second,
75+
IdleConnTimeout: time.Duration(openaiConfig.IdleConnTimeout) * time.Second,
76+
TLSHandshakeTimeout: time.Duration(openaiConfig.ResponseTimeout) * time.Second,
7877
ExpectContinueTimeout: 1 * time.Second,
7978
DisableCompression: true, // Always disable compression for streaming
8079
ForceAttemptHTTP2: true,
8180
WriteBufferSize: 64 * 1024,
8281
ReadBufferSize: 64 * 1024,
83-
ResponseHeaderTimeout: 30 * time.Second,
82+
ResponseHeaderTimeout: time.Duration(openaiConfig.ResponseTimeout) * time.Second,
8483
}
8584

8685
httpClient := &http.Client{
8786
Transport: transport,
88-
Timeout: time.Duration(openaiConfig.Timeout) * time.Millisecond,
87+
Timeout: time.Duration(openaiConfig.RequestTimeout) * time.Second,
8988
}
9089

9190
// Streaming client without overall timeout
@@ -229,7 +228,7 @@ func (ps *ProxyServer) executeRequestWithRetry(c *gin.Context, startTime time.Ti
229228
ctx, cancel = context.WithCancel(c.Request.Context())
230229
} else {
231230
// Non-streaming requests use configured timeout from the already fetched config
232-
timeout := time.Duration(openaiConfig.Timeout) * time.Millisecond
231+
timeout := time.Duration(openaiConfig.RequestTimeout) * time.Second
233232
ctx, cancel = context.WithTimeout(c.Request.Context(), timeout)
234233
}
235234
defer cancel()

0 commit comments

Comments
 (0)