Skip to content

Commit 442e561

Browse files
authored
change settings page and add ratio task detail page (#77)
* feat: Update site name to DataMate and refine text for AI data processing * feat: Refactor settings page and implement model access functionality - Created a new ModelAccess component for managing model configurations. - Removed the old Settings component and replaced it with a new SettingsPage component that integrates ModelAccess, SystemConfig, and WebhookConfig. - Added SystemConfig component for managing system settings. - Implemented WebhookConfig component for managing webhook configurations. - Updated API functions for model management in settings.apis.ts. - Adjusted routing to point to the new SettingsPage component. * feat: Implement Data Collection Page with Task Management and Execution Log - Created DataCollectionPage component to manage data collection tasks. - Added TaskManagement and ExecutionLog components for task handling and logging. - Integrated task operations including start, stop, edit, and delete functionalities. - Implemented filtering and searching capabilities in task management. - Introduced SimpleCronScheduler for scheduling tasks with cron expressions. - Updated CreateTask component to utilize new scheduling and template features. - Enhanced BasicInformation component to conditionally render fields based on visibility settings. - Refactored ImportConfiguration component to remove NAS import section. * feat: Update task creation API endpoint and enhance task creation form with new fields and validation * Refactor file upload and operator management components - Removed unnecessary console logs from file download and export functions. - Added size property to TaskItem interface for better task management. - Simplified TaskUpload component by utilizing useFileSliceUpload hook for file upload logic. - Enhanced OperatorPluginCreate component to handle file uploads and parsing more efficiently. - Updated ConfigureStep component to use Ant Design Form for better data handling and validation. - Improved PreviewStep component to navigate back to the operator market. - Added support for additional file types in UploadStep component. - Implemented delete operator functionality in OperatorMarketPage with confirmation prompts. - Cleaned up unused API functions in operator.api.ts to streamline the codebase. - Fixed number formatting utility to handle zero values correctly. * Refactor Knowledge Generation to Knowledge Base - Created new API service for Knowledge Base operations including querying, creating, updating, and deleting knowledge bases and files. - Added constants for Knowledge Base status and type mappings. - Defined models for Knowledge Base and related files. - Removed obsolete Knowledge Base creation and home components, replacing them with new implementations under the Knowledge Base structure. - Updated routing to reflect the new Knowledge Base paths. - Adjusted menu items to align with the new Knowledge Base terminology. - Modified ModelAccess interface to include modelName and type properties. * feat: Implement Knowledge Base Page with CRUD operations and data management - Added KnowledgeBasePage component for displaying and managing knowledge bases. - Integrated search and filter functionalities with SearchControls component. - Implemented CreateKnowledgeBase component for creating and editing knowledge bases. - Enhanced AddDataDialog for file uploads and dataset selections. - Introduced TableTransfer component for managing data transfers between tables. - Updated API functions for knowledge base operations, including file management. - Refactored knowledge base model to include file status and metadata. - Adjusted routing to point to the new KnowledgeBasePage. * feat: enhance OperatorPluginCreate and ConfigureStep for better upload handling and UI updates * refactor: remove unused components and clean up API logging in KnowledgeBase * feat: update icons in various components and improve styling for better UI consistency * fix: adjust upload step handling and improve error display in configuration step * feat: Add RatioTransfer component for dataset selection and configuration - Implemented RatioTransfer component to manage dataset selection and ratio configuration. - Integrated dataset fetching with search and filter capabilities. - Added RatioConfig component for displaying and updating selected datasets' configurations. - Enhanced SelectDataset component with improved UI and functionality for dataset selection. - Updated RatioTasksPage to utilize new ratio task status mapping and improved error handling for task deletion. - Refactored ratio model and constants for better type safety and clarity. - Changed Vite configuration to use local backend service for development. * feat: Add .editorconfig and enhance SystemConfig with table for settings display * feat: Enhance parameter configuration for range inputs and update default values * feat: Update site name to DataMate and refine text for AI data processing * Refactor file upload and operator management components - Removed unnecessary console logs from file download and export functions. - Added size property to TaskItem interface for better task management. - Simplified TaskUpload component by utilizing useFileSliceUpload hook for file upload logic. - Enhanced OperatorPluginCreate component to handle file uploads and parsing more efficiently. - Updated ConfigureStep component to use Ant Design Form for better data handling and validation. - Improved PreviewStep component to navigate back to the operator market. - Added support for additional file types in UploadStep component. - Implemented delete operator functionality in OperatorMarketPage with confirmation prompts. - Cleaned up unused API functions in operator.api.ts to streamline the codebase. - Fixed number formatting utility to handle zero values correctly. * Refactor Knowledge Generation to Knowledge Base - Created new API service for Knowledge Base operations including querying, creating, updating, and deleting knowledge bases and files. - Added constants for Knowledge Base status and type mappings. - Defined models for Knowledge Base and related files. - Removed obsolete Knowledge Base creation and home components, replacing them with new implementations under the Knowledge Base structure. - Updated routing to reflect the new Knowledge Base paths. - Adjusted menu items to align with the new Knowledge Base terminology. - Modified ModelAccess interface to include modelName and type properties. * feat: Implement Knowledge Base Page with CRUD operations and data management - Added KnowledgeBasePage component for displaying and managing knowledge bases. - Integrated search and filter functionalities with SearchControls component. - Implemented CreateKnowledgeBase component for creating and editing knowledge bases. - Enhanced AddDataDialog for file uploads and dataset selections. - Introduced TableTransfer component for managing data transfers between tables. - Updated API functions for knowledge base operations, including file management. - Refactored knowledge base model to include file status and metadata. - Adjusted routing to point to the new KnowledgeBasePage. * feat: enhance OperatorPluginCreate and ConfigureStep for better upload handling and UI updates * feat: update icons in various components and improve styling for better UI consistency * fix: adjust upload step handling and improve error display in configuration step * feat: Update site name to DataMate and refine text for AI data processing * Refactor file upload and operator management components - Removed unnecessary console logs from file download and export functions. - Added size property to TaskItem interface for better task management. - Simplified TaskUpload component by utilizing useFileSliceUpload hook for file upload logic. - Enhanced OperatorPluginCreate component to handle file uploads and parsing more efficiently. - Updated ConfigureStep component to use Ant Design Form for better data handling and validation. - Improved PreviewStep component to navigate back to the operator market. - Added support for additional file types in UploadStep component. - Implemented delete operator functionality in OperatorMarketPage with confirmation prompts. - Cleaned up unused API functions in operator.api.ts to streamline the codebase. - Fixed number formatting utility to handle zero values correctly. * Refactor Knowledge Generation to Knowledge Base - Created new API service for Knowledge Base operations including querying, creating, updating, and deleting knowledge bases and files. - Added constants for Knowledge Base status and type mappings. - Defined models for Knowledge Base and related files. - Removed obsolete Knowledge Base creation and home components, replacing them with new implementations under the Knowledge Base structure. - Updated routing to reflect the new Knowledge Base paths. - Adjusted menu items to align with the new Knowledge Base terminology. - Modified ModelAccess interface to include modelName and type properties. * feat: Implement Knowledge Base Page with CRUD operations and data management - Added KnowledgeBasePage component for displaying and managing knowledge bases. - Integrated search and filter functionalities with SearchControls component. - Implemented CreateKnowledgeBase component for creating and editing knowledge bases. - Enhanced AddDataDialog for file uploads and dataset selections. - Introduced TableTransfer component for managing data transfers between tables. - Updated API functions for knowledge base operations, including file management. - Refactored knowledge base model to include file status and metadata. - Adjusted routing to point to the new KnowledgeBasePage. * feat: enhance OperatorPluginCreate and ConfigureStep for better upload handling and UI updates * feat: update icons in various components and improve styling for better UI consistency * fix: adjust upload step handling and improve error display in configuration step * feat: add settings drawer and integrate SettingsPage component * feat: add ratio task management features including detail view and API integration
1 parent 2b09c7d commit 442e561

File tree

11 files changed

+707
-113
lines changed

11 files changed

+707
-113
lines changed

frontend/src/components/CardView.tsx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -209,7 +209,9 @@ function CardView<T extends BaseCardDataType>(props: CardViewProps<T>) {
209209
{item?.status && (
210210
<Tag color={item?.status?.color}>
211211
<div className="flex items-center gap-2 text-xs py-0.5">
212-
<span>{item?.status?.icon}</span>
212+
{item?.status?.icon && (
213+
<span>{item?.status?.icon}</span>
214+
)}
213215
<span>{item?.status?.label}</span>
214216
</div>
215217
</Tag>

frontend/src/mock/mock-apis.cjs

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,17 @@ const MockAPI = {
9292
deleteInstructionTemplateByIdUsingDelete: "/synthesis/templates/:templateId", // 删除指令模板
9393
instructionTuningUsingPost: "/synthesis/instruction-tuning", // 指令微调
9494
cotDistillationUsingPost: "/synthesis/cot-distillation", // Cot蒸馏
95+
96+
// 数据配比接口
97+
createRatioTaskUsingPost: "/synthesis/ratio-task", // 创建配比任务
98+
queryRatioTasksUsingGet: "/synthesis/ratio-task", // 获取配比任务列表
99+
queryRatioTaskByIdUsingGet: "/synthesis/ratio-task/:taskId", // 根据ID获取配比任务详情
100+
deleteRatioTaskByIdUsingDelete: "/synthesis/ratio-task/:taskId", // 删除配比任务
101+
updateRatioTaskByIdUsingPut: "/synthesis/ratio-task/:taskId", // 更新配比任务
102+
executeRatioTaskByIdUsingPost: "/synthesis/ratio-task/:taskId/execute", // 执行配比任务
103+
stopRatioTaskByIdUsingPost: "/synthesis/ratio-task/:taskId/stop", // 停止配比任务
104+
queryRatioJobStatusUsingGet: "/synthesis/ratio-task/:taskId/status", // 获取配比任务状态
105+
queryRatioModelsUsingGet: "/synthesis/ratio-models", // 获取配比模型列表
95106

96107
// 数据评测接口
97108
queryEvaluationTasksUsingPost: "/evaluation/tasks", // 获取评测任务列表

frontend/src/mock/mock-seed/data-cleansing.cjs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -443,7 +443,6 @@ module.exports = function (router) {
443443
const { id } = req.params;
444444
const operator = operatorList.find((op) => op.id === id);
445445
console.log("获取算子详情:", id, operator);
446-
447446
if (operator) {
448447
// 增加浏览次数模拟
449448
operator.viewCount = (operator.viewCount || 0) + 1;
Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
2+
const Mock = require("mockjs");
3+
const API = require("../mock-apis.cjs");
4+
5+
function ratioJobItem() {
6+
return {
7+
id: Mock.Random.guid().replace(/[^a-zA-Z0-9]/g, ""),
8+
name: Mock.Random.ctitle(5, 15),
9+
description: Mock.Random.csentence(10, 30),
10+
status: Mock.Random.pick(["PENDING", "RUNNING", "COMPLETED", "FAILED", "PAUSED"]),
11+
totals: Mock.Random.integer(1000, 10000),
12+
ratio_method: Mock.Random.pick(["DATASET", "TAG"]),
13+
target_dataset_id: Mock.Random.guid().replace(/[^a-zA-Z0-9]/g, ""),
14+
target_dataset_name: Mock.Random.ctitle(3, 8),
15+
config: [
16+
{
17+
datasetId: Mock.Random.guid().replace(/[^a-zA-Z0-9]/g, ""),
18+
counts: Mock.Random.integer(100, 1000).toString(),
19+
filter_conditions: "",
20+
},
21+
{
22+
datasetId: Mock.Random.guid().replace(/[^a-zA-Z0-9]/g, ""),
23+
counts: Mock.Random.integer(100, 1000).toString(),
24+
filter_conditions: "",
25+
},
26+
],
27+
created_at: Mock.Random.datetime("yyyy-MM-dd HH:mm:ss"),
28+
updated_at: Mock.Random.datetime("yyyy-MM-dd HH:mm:ss"),
29+
};
30+
}
31+
32+
const ratioJobList = new Array(20).fill(null).map(ratioJobItem);
33+
34+
35+
module.exports = function (router) {
36+
// 获取配比任务列表
37+
router.get(API.queryRatioTasksUsingGet, (req, res) => {
38+
const { page = 0, size = 10, status } = req.query;
39+
let filteredJobs = ratioJobList;
40+
if (status) {
41+
filteredJobs = ratioJobList.filter((job) => job.status === status);
42+
}
43+
const startIndex = page * size;
44+
const endIndex = startIndex + parseInt(size);
45+
const pageData = filteredJobs.slice(startIndex, endIndex);
46+
res.send({
47+
code: "0",
48+
msg: "Success",
49+
data: {
50+
content: pageData,
51+
totalElements: filteredJobs.length,
52+
totalPages: Math.ceil(filteredJobs.length / size),
53+
size: parseInt(size),
54+
number: parseInt(page),
55+
},
56+
});
57+
});
58+
59+
// 创建配比任务
60+
router.post(API.createRatioTaskUsingPost, (req, res) => {
61+
const newJob = {
62+
...ratioJobItem(),
63+
...req.body,
64+
id: Mock.Random.guid().replace(/[^a-zA-Z0-9]/g, ""),
65+
status: "PENDING",
66+
createdAt: new Date().toISOString(),
67+
};
68+
ratioJobList.push(newJob);
69+
res.status(201).send({
70+
code: "0",
71+
msg: "Ratio job created successfully",
72+
data: newJob,
73+
});
74+
});
75+
76+
// 获取配比任务详情
77+
router.get(API.queryRatioTaskByIdUsingGet, (req, res) => {
78+
const { taskId } = req.params;
79+
const job = ratioJobList.find((j) => j.id === taskId);
80+
if (job) {
81+
res.send({
82+
code: "0",
83+
msg: "Success",
84+
data: job,
85+
});
86+
} else {
87+
res.status(404).send({
88+
code: "1",
89+
msg: "Ratio job not found",
90+
data: null,
91+
});
92+
}
93+
});
94+
95+
// 删除配比任务
96+
router.delete(API.deleteRatioTaskByIdUsingDelete, (req, res) => {
97+
const { taskId } = req.params;
98+
const index = ratioJobList.findIndex((j) => j.id === taskId);
99+
if (index !== -1) {
100+
ratioJobList.splice(index, 1);
101+
res.send({
102+
code: "0",
103+
msg: "Ratio job deleted successfully",
104+
data: null,
105+
});
106+
} else {
107+
res.status(404).send({
108+
code: "1",
109+
msg: "Ratio job not found",
110+
data: null,
111+
});
112+
}
113+
});
114+
115+
// 更新配比任务
116+
router.put(API.updateRatioTaskByIdUsingPut, (req, res) => {
117+
const { taskId } = req.params;
118+
const index = ratioJobList.findIndex((j) => j.id === taskId);
119+
if (index !== -1) {
120+
ratioJobList[index] = {
121+
...ratioJobList[index],
122+
...req.body,
123+
updatedAt: new Date().toISOString(),
124+
};
125+
res.send({
126+
code: "0",
127+
msg: "Ratio job updated successfully",
128+
data: ratioJobList[index],
129+
});
130+
} else {
131+
res.status(404).send({
132+
code: "1",
133+
msg: "Ratio job not found",
134+
data: null,
135+
});
136+
}
137+
});
138+
139+
// 执行配比任务
140+
router.post(API.executeRatioTaskByIdUsingPost, (req, res) => {
141+
const { taskId } = req.params;
142+
const job = ratioJobList.find((j) => j.id === taskId);
143+
if (job) {
144+
job.status = "RUNNING";
145+
job.startedAt = new Date().toISOString();
146+
res.send({
147+
code: "0",
148+
msg: "Ratio job execution started",
149+
data: {
150+
executionId: Mock.Random.guid().replace(/[^a-zA-Z0-9]/g, ""),
151+
status: "RUNNING",
152+
message: "Job execution started successfully",
153+
},
154+
});
155+
} else {
156+
res.status(404).send({
157+
code: "1",
158+
msg: "Ratio job not found",
159+
data: null,
160+
});
161+
}
162+
});
163+
164+
// 停止配比任务
165+
router.post(API.stopRatioTaskByIdUsingPost, (req, res) => {
166+
const { taskId } = req.params;
167+
const job = ratioJobList.find((j) => j.id === taskId);
168+
if (job) {
169+
job.status = "STOPPED";
170+
job.finishedAt = new Date().toISOString();
171+
res.send({
172+
code: "0",
173+
msg: "Ratio job stopped successfully",
174+
data: null,
175+
});
176+
} else {
177+
res.status(404).send({
178+
code: "1",
179+
msg: "Ratio job not found",
180+
data: null,
181+
});
182+
}
183+
});
184+
185+
// 获取配比任务状态
186+
router.get(API.queryRatioJobStatusUsingGet, (req, res) => {
187+
const { taskId } = req.params;
188+
const job = ratioJobList.find((j) => j.id === taskId);
189+
if (job) {
190+
res.send({
191+
code: "0",
192+
msg: "Success",
193+
data: {
194+
status: job.status,
195+
progress: job.progress,
196+
},
197+
});
198+
} else {
199+
res.status(404).send({
200+
code: "1",
201+
msg: "Ratio job not found",
202+
data: null,
203+
});
204+
}
205+
});
206+
207+
// 获取配比模型列表
208+
router.get(API.queryRatioModelsUsingGet, (req, res) => {
209+
const models = [
210+
{ id: "model1", name: "均匀分配模型", description: "将目标数量均匀分配到各数据集。" },
211+
{ id: "model2", name: "标签优先模型", description: "优先满足标签配比需求。" },
212+
{ id: "model3", name: "自定义模型", description: "支持自定义分配逻辑。" },
213+
];
214+
res.send({
215+
code: "0",
216+
msg: "Success",
217+
data: models,
218+
});
219+
});
220+
};

frontend/src/pages/Layout/Sidebar.tsx

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { memo, useEffect, useState } from "react";
2-
import { Button, Menu, Popover } from "antd";
2+
import { Button, Drawer, Menu, Popover } from "antd";
33
import {
44
CloseOutlined,
55
MenuOutlined,
@@ -9,13 +9,15 @@ import { ClipboardList, Sparkles, X } from "lucide-react";
99
import { menuItems } from "@/pages/Layout/menu";
1010
import { NavLink, useLocation, useNavigate } from "react-router";
1111
import TaskUpload from "./TaskUpload";
12+
import SettingsPage from "../SettingsPage/SettingsPage";
1213

1314
const AsiderAndHeaderLayout = () => {
1415
const { pathname } = useLocation();
1516
const navigate = useNavigate();
1617
const [activeItem, setActiveItem] = useState<string>("");
1718
const [sidebarOpen, setSidebarOpen] = useState(true);
1819
const [taskCenterVisible, setTaskCenterVisible] = useState(false);
20+
const [settingVisible, setSettingVisible] = useState(false);
1921

2022
// Initialize active item based on current pathname
2123
const initActiveItem = () => {
@@ -137,11 +139,8 @@ const AsiderAndHeaderLayout = () => {
137139
</Popover>
138140
<Button
139141
block
140-
color={pathname === "/data/settings" ? "primary" : "default"}
141-
variant={pathname === "/data/settings" ? "filled" : "outlined"}
142142
onClick={() => {
143-
setActiveItem("");
144-
navigate("/data/settings");
143+
setSettingVisible(true);
145144
}}
146145
>
147146
设置
@@ -167,19 +166,26 @@ const AsiderAndHeaderLayout = () => {
167166
</div>
168167
<Button
169168
block
170-
color={pathname === "/data/settings" ? "primary" : "default"}
171-
variant={pathname === "/data/settings" ? "filled" : "outlined"}
172169
onClick={() => {
173-
setActiveItem("");
174-
navigate("/data/settings");
170+
setSettingVisible(true);
175171
}}
176172
>
177173
<SettingOutlined />
178174
</Button>
179175
</div>
180176
)}
181177
</div>
182-
178+
<Drawer
179+
title="设置"
180+
placement="bottom"
181+
width="100%"
182+
height="100%"
183+
open={settingVisible}
184+
onClose={() => setSettingVisible(false)}
185+
bodyStyle={{ padding: 0 }}
186+
>
187+
<SettingsPage></SettingsPage>
188+
</Drawer>
183189
{/* 添加遮罩层,点击外部区域时关闭 */}
184190
{taskCenterVisible && (
185191
<div

0 commit comments

Comments
 (0)