Skip to content

Commit bdee2db

Browse files
c121914yuxqvvu
andauthored
V4.14.4 dev (#6058)
* perf: faq * index * delete dataset * delete dataset * perf: delete dataset * init * fix: outLink UID (#6048) * perf: query extension * fix: s3 configs (#6050) * fix: s3 configs * s3 --------- Co-authored-by: archer <[email protected]> * s3 valid string check * perf: completion api * fix: model test * perf: init * fix: init * fix: init shell * fix: faq --------- Co-authored-by: Roy <[email protected]>
1 parent 44f9503 commit bdee2db

File tree

21 files changed

+595
-614
lines changed

21 files changed

+595
-614
lines changed

document/content/docs/introduction/development/openapi/share.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,20 +15,20 @@ description: FastGPT 分享链接身份鉴权
1515

1616
### 接口统一响应格式
1717

18-
```json
18+
```jsonc
1919
{
2020
"success": true,
2121
"message": "错误提示",
2222
"msg": "同message, 错误提示",
2323
"data": {
24-
"uid": "用户唯一凭证"
24+
"uid": "用户唯一凭证" // 必须返回
2525
}
2626
}
2727
```
2828

2929
`FastGPT` 将会判断`success`是否为`true`决定是允许用户继续操作。`message``msg`是等同的,你可以选择返回其中一个,当`success`不为`true`时,将会提示这个错误。
3030

31-
`uid`是用户的唯一凭证,将会用于拉取对话记录以及保存对话记录。可参考下方实践案例。
31+
`uid` 是用户的唯一凭证,必须返回该 ID 且 ID 的格式为不包含 "|"、"/“、"\" 字符的、长度小于等于 255 的字符串,否则会返回 `Invalid UID` 的错误。`uid` 将会用于拉取对话记录以及保存对话记录,可参考下方实践案例。
3232

3333
### 触发流程
3434

document/content/docs/upgrading/4-14/4144.mdx

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ curl --location --request POST 'https://{{host}}/api/admin/initv4144' \
3030
4. 通过 API 上传本地文件至知识库,保存至 S3。同时将旧版 Gridfs 代码全部移除。
3131
5. 新版订阅套餐逻辑。
3232
6. 支持配置对话文件白名单。
33+
7. S3 支持 pathStyle 配置。
3334

3435
## ⚙️ 优化
3536

@@ -38,6 +39,8 @@ curl --location --request POST 'https://{{host}}/api/admin/initv4144' \
3839
3. 用户通知,支持中英文,以及优化模板。
3940
4. 删除知识库采用队列异步删除模式。
4041
5. LLM 请求时,图片无效报错提示。
42+
6. completions 接口,非 stream 模式, detail=false 时,增加返回 reason_content。
43+
7. 增加对于无效的 S3 key 检测。
4144

4245
## 🐛 修复
4346

@@ -47,6 +50,9 @@ curl --location --request POST 'https://{{host}}/api/admin/initv4144' \
4750
4. 工作台卡片在名字过长时错位。
4851
5. 分享链接中url query 中携带全局变量时,前端 UI 不会加载该值。
4952
6. window 下判断 CSV 文件异常。
53+
7. 模型测试时,如果模型未启动,会导致无法被测试。
54+
8. MCP header 中带特殊内容时,会抛错。
55+
9. 工作流引用其他 Agent 时,切换版本号后未及时更新 UI。
5056

5157
## 插件
5258

document/data/doc-last-modified.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
"document/content/docs/introduction/development/openapi/chat.mdx": "2025-11-14T13:21:17+08:00",
3535
"document/content/docs/introduction/development/openapi/dataset.mdx": "2025-09-29T11:34:11+08:00",
3636
"document/content/docs/introduction/development/openapi/intro.mdx": "2025-09-29T11:34:11+08:00",
37-
"document/content/docs/introduction/development/openapi/share.mdx": "2025-08-05T23:20:39+08:00",
37+
"document/content/docs/introduction/development/openapi/share.mdx": "2025-12-08T16:10:51+08:00",
3838
"document/content/docs/introduction/development/proxy/cloudflare.mdx": "2025-07-23T21:35:03+08:00",
3939
"document/content/docs/introduction/development/proxy/http_proxy.mdx": "2025-07-23T21:35:03+08:00",
4040
"document/content/docs/introduction/development/proxy/nginx.mdx": "2025-07-23T21:35:03+08:00",
@@ -118,7 +118,7 @@
118118
"document/content/docs/upgrading/4-14/4141.mdx": "2025-11-19T10:15:27+08:00",
119119
"document/content/docs/upgrading/4-14/4142.mdx": "2025-11-18T19:27:14+08:00",
120120
"document/content/docs/upgrading/4-14/4143.mdx": "2025-11-26T20:52:05+08:00",
121-
"document/content/docs/upgrading/4-14/4144.mdx": "2025-12-08T01:44:15+08:00",
121+
"document/content/docs/upgrading/4-14/4144.mdx": "2025-12-08T17:57:59+08:00",
122122
"document/content/docs/upgrading/4-8/40.mdx": "2025-08-02T19:38:37+08:00",
123123
"document/content/docs/upgrading/4-8/41.mdx": "2025-08-02T19:38:37+08:00",
124124
"document/content/docs/upgrading/4-8/42.mdx": "2025-08-02T19:38:37+08:00",

packages/service/common/s3/buckets/base.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ export class S3BaseBucket {
5050
port: externalPort,
5151
accessKey: options.accessKey,
5252
secretKey: options.secretKey,
53+
pathStyle: options.pathStyle,
5354
transportAgent: options.transportAgent
5455
});
5556
}

packages/service/common/s3/constants.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,3 +52,9 @@ export const getSystemMaxFileSize = () => {
5252
const config = global.feConfigs?.uploadFileMaxSize || 1024; // MB, default 1024MB
5353
return config; // bytes
5454
};
55+
56+
export const S3_KEY_PATH_INVALID_CHARS_MAP: Record<string, boolean> = {
57+
'/': true,
58+
'\\': true,
59+
'|': true
60+
};

packages/service/common/s3/controller.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import { setCron } from '../system/cron';
44
import { checkTimerLock } from '../system/timerLock/utils';
55
import { TimerIdEnum } from '../system/timerLock/constants';
66
import path from 'node:path';
7+
import { S3Error } from 'minio';
78

89
export async function clearExpiredMinioFiles() {
910
try {
@@ -56,6 +57,12 @@ export async function clearExpiredMinioFiles() {
5657
addLog.warn(`Bucket not found: ${file.bucketName}`);
5758
}
5859
} catch (error) {
60+
if (
61+
error instanceof S3Error &&
62+
error.message.includes('Object name contains unsupported characters.')
63+
) {
64+
await MongoS3TTL.deleteOne({ _id: file._id });
65+
}
5966
fail++;
6067
addLog.error(`Failed to delete minio file: ${file.minioKey}`, error);
6168
}

packages/service/core/ai/functions/queryExtension.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ import { useTextCosine } from '../hooks/useTextCosine';
1313
This module can eliminate referential ambiguity and expand queries based on context to improve retrieval.
1414
Submodular Optimization Mode: Generate multiple candidate queries, then use submodular algorithm to select the optimal query combination
1515
*/
16-
const title = global.feConfigs?.systemTitle || 'FastAI';
16+
const title = global.feConfigs?.systemTitle || 'Nginx';
1717
const defaultPrompt = `## 你的任务
1818
你作为一个向量检索助手,你的任务是结合历史记录,为"原问题"生成{{count}}个不同版本的"检索词"。这些检索词应该从不同角度探索主题,以提高向量检索的语义丰富度和精度。
1919
@@ -230,7 +230,7 @@ assistant: ${chatBg}
230230
.replace(/ /g, '');
231231

232232
try {
233-
const queries = json5.parse(jsonStr) as string[];
233+
let queries = json5.parse(jsonStr) as string[];
234234

235235
if (!Array.isArray(queries) || queries.length === 0) {
236236
return {
@@ -248,6 +248,8 @@ assistant: ${chatBg}
248248
const { lazyGreedyQuerySelection, embeddingModel: useEmbeddingModel } = useTextCosine({
249249
embeddingModel
250250
});
251+
queries = queries.map((item) => String(item));
252+
251253
const { selectedData: selectedQueries, embeddingTokens } = await lazyGreedyQuerySelection({
252254
originalText: query,
253255
candidates: queries,

packages/service/core/ai/llm/request.ts

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -81,14 +81,15 @@ export const createLLMResponse = async <T extends CompletionsBodyType>(
8181
return requestMessages;
8282
})();
8383

84-
const requestBody = await llmCompletionsBodyFormat({
84+
const { requestBody, modelData } = await llmCompletionsBodyFormat({
8585
...body,
8686
messages: rewriteMessages
8787
});
8888

8989
// console.log(JSON.stringify(requestBody, null, 2));
9090
const { response, isStreamResponse, getEmptyResponseTip } = await createChatCompletion({
9191
body: requestBody,
92+
modelData,
9293
userKey,
9394
options: {
9495
headers: {
@@ -491,10 +492,16 @@ const llmCompletionsBodyFormat = async <T extends CompletionsBodyType>({
491492
parallel_tool_calls,
492493
toolCallMode,
493494
...body
494-
}: LLMRequestBodyType<T>): Promise<InferCompletionsBody<T>> => {
495+
}: LLMRequestBodyType<T>): Promise<{
496+
requestBody: InferCompletionsBody<T>;
497+
modelData: LLMModelItemType;
498+
}> => {
495499
const modelData = getLLMModel(body.model);
496500
if (!modelData) {
497-
return body as unknown as InferCompletionsBody<T>;
501+
return {
502+
requestBody: body as unknown as InferCompletionsBody<T>,
503+
modelData
504+
};
498505
}
499506

500507
const response_format = (() => {
@@ -548,7 +555,10 @@ const llmCompletionsBodyFormat = async <T extends CompletionsBodyType>({
548555
});
549556
}
550557

551-
return requestBody as unknown as InferCompletionsBody<T>;
558+
return {
559+
requestBody: requestBody as unknown as InferCompletionsBody<T>,
560+
modelData
561+
};
552562
};
553563
const createChatCompletion = async ({
554564
modelData,
@@ -579,6 +589,7 @@ const createChatCompletion = async ({
579589
try {
580590
// Rewrite model
581591
const modelConstantsData = modelData || getLLMModel(body.model);
592+
582593
if (!modelConstantsData) {
583594
return Promise.reject(`${body.model} not found`);
584595
}

packages/service/core/dataset/controller.ts

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,25 @@ export async function delDatasetRelevantData({
115115
// Delete vector data
116116
await deleteDatasetDataVector({ teamId, datasetIds });
117117

118+
// Delete dataset_data_texts in batches by datasetId
119+
for (const datasetId of datasetIds) {
120+
await MongoDatasetDataText.deleteMany({
121+
teamId,
122+
datasetId
123+
}).maxTimeMS(300000); // Reduce timeout for single batch
124+
}
125+
// Delete dataset_datas in batches by datasetId
126+
for (const datasetId of datasetIds) {
127+
await MongoDatasetData.deleteMany({
128+
teamId,
129+
datasetId
130+
}).maxTimeMS(300000);
131+
}
132+
133+
await delCollectionRelatedSource({ collections });
134+
// Delete vector data
135+
await deleteDatasetDataVector({ teamId, datasetIds });
136+
118137
// delete collections
119138
await MongoDatasetCollection.deleteMany({
120139
teamId,

packages/service/core/dataset/data/controller.ts

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,11 @@ import { addDays } from 'date-fns';
55
import { isS3ObjectKey, jwtSignS3ObjectKey } from '../../../common/s3/utils';
66

77
export const formatDatasetDataValue = ({
8-
teamId,
9-
datasetId,
108
q,
119
a,
1210
imageId,
1311
imageDescMap
1412
}: {
15-
teamId: string;
16-
datasetId: string;
1713
q: string;
1814
a?: string;
1915
imageId?: string;
@@ -73,8 +69,6 @@ export const getFormatDatasetCiteList = (list: DatasetDataSchemaType[]) => {
7369
return list.map((item) => ({
7470
_id: item._id,
7571
...formatDatasetDataValue({
76-
teamId: item.teamId,
77-
datasetId: item.datasetId,
7872
q: item.q,
7973
a: item.a,
8074
imageId: item.imageId

0 commit comments

Comments
 (0)