Merge branch 'master' of https://github.com/Dramwig/FlowLine

Author: hkra0

docs/fig/logo.png

@@ -1,6 +1,12 @@
-# FlowLine
+<div align="center">
+  <img src="./fig/logo.png" width="50%" alt="FlowLine" />
+
+  <!-- [![LICENSE](https://badgen.net/static/LICENSE/Apache)](LICENSE) -->
+
+  [中文](../readme.md) | English
+</div>
+
 
-[中文](./readme.md) | English
 
 FlowLine is an automated system for **GPU resource management** and **concurrent command stream scheduling**, supporting both **Command Line Interface (CLI)** and **Web Graphical User Interface (GUI)** interaction modes. It is suitable for multi-task experiments, deep learning training, or high-concurrency computing environments.
 
@@ -38,14 +44,14 @@ pip install -e <path_to_flowline_repository>
 
 > Note: Ensure you have installed basic dependencies from `requirements.txt` (`pandas`, `psutil`, `openpyxl`, etc.).
 
-#### 2. Create Task Control Sheet `todo.xlsx`
+#### 2. Create Task Control Sheet
 
-The system uses an Excel file (`.xlsx` format) to define task parameters. **This is the only input method for all tasks.** Each row represents an independent task, and each column corresponds to a parameter that will be automatically mapped to `--key value` CLI format.
+The system uses a list file (`.xlsx`、 `.csv` or `.json` format) to define task parameters. **This is the only input method for all tasks.** Each row represents an independent task, and each column corresponds to a parameter that will be automatically mapped to `--key value` CLI format.
 
 <details>
 <summary>Example and Explanation</summary>
 
-Example file: [`test/todo.xlsx`](./test/todo.xlsx)
+Example files: [`test/todo.xlsx`](./test/todo.xlsx), [`test/todo.csv`](./test/todo.csv),[`test/todo.json`](./test/todo.json), which can be constructed using the example program [`test/task_builder.py`](./test/task_builder.py).
 
 | *name*    | lr    | batch_size | *run_num* | *need_run_num* | *cmd*       |
 | --------- | ----- | ---------- | --------- | -------------- | ----------- |
@@ -59,7 +65,7 @@ Field descriptions:
 * `cmd`: Reserved field (can be empty or specify main command like `train_main`). Can be used with custom `func` logic.
 * Other fields can be freely defined and will be passed to the command constructor.
 
-> Note: If reserved fields are missing, **the system will auto-complete them during Excel loading** to ensure valid structure.
+> Note: If reserved fields are missing, **the system will auto-complete them during loading** to ensure valid structure.
 
 The flexible task sheet structure supports everything from parameter tuning to complex grid search automation.
 
@@ -112,6 +118,61 @@ Always use `-u` to ensure **real-time log writing** to these files.
 </details>
 </details>
 
+#### 4. Enter `run` to start the task flow
+
+<details>
+<summary>FlowLine CLI Command Reference Table</summary>
+
+| Command        | Parameter                 | Description                                                                                                |
+| -------------- | ------------------------- | ---------------------------------------------------------------------------------------------------------- |
+| `run`          | None                      | Toggles the task processing loop state (start/stop)                                                        |
+| `gpu <id>`     | `<id>`: GPU ID            | Toggles the availability of the specified GPU (available/unavailable)                                      |
+| `killgpu <id>` | `<id>`: GPU ID            | Terminates all processes running on the specified GPU                                                      |
+| `kill <id>`    | `<id>`: Process ID        | Terminates the process with the specified process ID                                                       |
+| `ls`           | None                      | Lists all running processes, showing process ID, PID, task ID, GPU ID, status, and command                 |
+| `gpus`         | None                      | Displays the status of all GPUs, including utilization, memory usage, temperature, power consumption, etc. |
+| `min <num>`    | `<num>`: Memory size (MB) | Sets the minimum required memory for processes                                                             |
+| `max <num>`    | `<num>`: Process count    | Sets the maximum number of concurrent processes                                                            |
+| `task`         | None                      | Lists the pending task queue, showing task ID, name, run count, etc.                                       |
+| `exit`         | None                      | Exits the program (equivalent to `Ctrl+D`)                                                                 |
+| `help` or `?`  | None                      | Displays help information                                                                                  |
+
+<details>
+<summary>Command Usage Examples</summary>
+
+```bash
+# Start the task processing loop
+> run
+
+# Check GPU status
+> gpus
+
+# View running processes
+> ls
+
+# Set the maximum number of concurrent processes to 4
+> max 4
+
+# Set the minimum memory requirement to 2048 MB
+> min 2048
+
+# Disable GPU 1
+> gpu 1
+
+# Terminate all processes on GPU 0
+> killgpu 0
+
+# View pending tasks
+> task
+
+# Exit the program
+> exit
+```
+
+</details>
+</details>
+
+
 ### 🌐 Using Web Interface (Visual Task Management)
 
 > **No extra configuration needed - Works directly in SSH environments**
@@ -150,10 +211,33 @@ This project provides **automated detection and utilization of idle GPUs** for r
 - **DO NOT misuse to monopolize shared resources or disrupt others' research.**
 
 ### 🚨 Risk Statement
-> **For learning/research purposes only. Users assume all responsibility.**
 
 Potential risks include but not limited to:
 - Resource conflicts from concurrent scheduling
 - Violation of lab/platform policies if abused
 
-Developers **shall not be liable** for any direct/indirect losses including resource conflicts, account restrictions, or data loss resulting from script usage.
+Developers **shall not be liable** for any direct/indirect losses including resource conflicts, account restrictions, or data loss resulting from script usage.
+
+
+## 💐 Contributions
+
+We welcome everyone to contribute code, fix bugs, or improve the documentation for this template!
+
+- If you have any suggestions or questions, please submit an issue.
+- Pull requests are welcome.
+  
+> [!TIP]
+> If this project is helpful to you, please give it a **Star**!
+
+**Thanks to all contributors!**
+
+[![Contributors](https://contrib.rocks/image?repo=dramwig/FlowLine)](https://github.com/dramwig/FlowLine/graphs/contributors)
+
+<a href="https://www.star-history.com/#dramwig/FlowLine&Date">
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=dramwig/FlowLine&type=Date&theme=dark" />
+   <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=dramwig/FlowLine&type=Date" />
+   
+<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=dramwig/FlowLine&type=Date" />
+ </picture>
+</a>

readme_en.md docs/readme_en.mdreadme_en.md renamed to docs/readme_en.md

@@ -1,2 +1 @@
-from .core import run_cli
-from .api import get_app
+from .api import get_app, run_cli

flowline/__init__.py

@@ -1 +1,2 @@
-from .routes import get_app
+from .routes import get_app
+from .interface import run_cli

flowline/api/__init__.py

@@ -1,2 +1,2 @@
 from .program import ProgramManager
-from .interface import run_cli
+from ..api.interface import run_cli

flowline/core/interface.py flowline/api/interface.pyflowline/core/interface.py renamed to flowline/api/interface.py

@@ -40,13 +40,13 @@ class TaskManager:
     """
     excel的保留关键字(属性): run_num, need_run_num, name, cmd
     """
-    def __init__(self, excel_path):
+    def __init__(self, task_dir):
         self._lock = threading.Lock()
-        self.excel_path = excel_path
-        logger.info(f"read excel: {excel_path}")
-        self.df = pd.read_excel(excel_path)
+        self.task_dir = task_dir
+        logger.info(f"read excel: {task_dir}")
+        self._read_df()
         self.format_tidy_df()
-        self.df.to_excel(excel_path, index=False)   
+        self._save_df()
 
         self.tasks = []
         self.task_ids = queue.PriorityQueue()
@@ -72,6 +72,26 @@ def wrapper(self, *args, **kwargs):
                 return func(self, *args, **kwargs)
         return wrapper
 
+    def _read_df(self):
+        if self.task_dir.endswith(".xlsx"):
+            self.df = pd.read_excel(self.task_dir)
+        elif self.task_dir.endswith(".csv"):
+            self.df = pd.read_csv(self.task_dir)
+        elif self.task_dir.endswith(".json"):
+            self.df = pd.read_json(self.task_dir)
+        else:
+            raise ValueError("Invalid file extension. Please use .xlsx, .csv, or .json.")
+    
+    def _save_df(self):
+        if self.task_dir.endswith(".xlsx"):
+            self.df.to_excel(self.task_dir, index=False)
+        elif self.task_dir.endswith(".csv"):
+            self.df.to_csv(self.task_dir, index=False)
+        elif self.task_dir.endswith(".json"):
+            self.df.to_json(self.task_dir, orient="records", force_ascii=False, indent=4)
+        else:
+            raise ValueError("Invalid file extension. Please use .xlsx, .csv, or .json.")
+    
     # -------------------------------
 
     def get_task_dict(self):
@@ -99,7 +119,7 @@ def put_task_ids(self, id):
     def update_task_ids(self, id):
         self.tasks[id].run_num += 1
         self.df.loc[id, 'run_num'] += 1
-        self.df.to_excel(self.excel_path, index=False)
+        self._save_df()
         logger.info(f"update task {id} run times: {self.df.loc[id, 'run_num']}")
 
     @synchronized

flowline/core/__init__.py

@@ -6,6 +6,6 @@
     def func(dict, gpu_id):
         return "CUDA_VISIBLE_DEVICES="+str(gpu_id)+" python -u test/test.py "+ " ".join([f"--{k} {v}" for k, v in dict.items()])
 
-    run_cli(func, "test/todo.xlsx")
+    run_cli(func, "test/todo.csv")
 
 # pkill -9 python

flowline/core/task.py

@@ -1,6 +1,12 @@
-# FlowLine
+<!-- # FlowLine -->
 
-中文 | [English](./readme_en.md)
+<div align="center">
+  <img src="./docs/fig/logo.png" width="50%" alt="FlowLine" />
+
+  <!-- [![LICENSE](https://badgen.net/static/LICENSE/Apache)](LICENSE) -->
+
+  中文 | [English](./docs/readme_en.md)
+</div>
 
 FlowLine 是一个用于 **GPU资源管理** 和 **并发指令流调度** 的自动化系统，支持 **命令行界面（CLI）** 和 **Web 图形界面（GUI）** 两种交互方式，适用于多任务实验、深度学习训练或高并发计算环境。
 
@@ -40,14 +46,14 @@ pip install -e <flowline库路径>
 
 > 注：请确保你已安装 `pandas`、`psutil`、`openpyxl` 等`requirements.txt`内的基本依赖。
 
-#### 2. 编写任务控制表 `todo.xlsx`
+#### 2. 编写任务控制表
 
-系统通过一个 Excel 文件（`.xlsx` 格式）来定义任务参数，**这是所有任务的唯一输入方式**。文件中的每一行代表一个独立任务，每一列对应一个参数项，将自动映射为命令行中的 `--key value` 格式。
+系统通过一个列表文件（`.xlsx`、 `.csv` 或 `.json` 格式）来定义任务参数，**这是所有任务的唯一输入方式**。文件中的每一行代表一个独立任务，每一列对应一个参数项，将自动映射为命令行中的 `--key value` 格式。
 
 <details>
 <summary>示例和说明</summary>
 
-示例文件：[`test/todo.xlsx`](./test/todo.xlsx)
+示例文件：[`test/todo.xlsx`](./test/todo.xlsx)、[`test/todo.csv`](./test/todo.csv)、[`test/todo.json`](./test/todo.json)， 可以通过运行[`test/task_builder.py`](./test/task_builder.py)示例构造程序构造。
 
 | *name*    | lr    | batch\_size |*run\_num*|*need\_run\_num*| *cmd*       |
 | --------- | ----- | ----------- | -------- | -------------- | ----------- |
@@ -62,7 +68,7 @@ pip install -e <flowline库路径>
 * `cmd`：保留字段，当前版本可预留为空或指定主命令（如 `train_main`），可结合 `func` 自定义逻辑使用。
 * 其余字段可自由定义，系统会将这些字段作为参数传入自定义命令构造函数中。
 
-> 注意：如果缺失上述保留字段，**系统会在加载 Excel 时自动补全**，确保表结构合法。
+> 注意：如果缺失上述保留字段，**系统会在加载文件时自动补全**，确保表结构合法。
 
 任务表结构灵活，可覆盖从参数微调到复杂网格搜索的自动并发调度。
 
@@ -97,8 +103,6 @@ if __name__ == "__main__":
 <details>
 <summary>关于输出和python -u</summary>
 
-💡 **关于 `python -u`：**
-
 在命令中加入 `-u` 参数（即 `python -u ...`）表示以 **非缓冲模式（unbuffered mode）** 启动 Python：
 
 * 标准输出（`stdout`）和标准错误（`stderr`）会**立即刷新**；
@@ -118,6 +122,59 @@ log/
 </details>
 </details>
 
+#### 4. 输入`run`开始运行任务流
+
+<details>
+<summary>FlowLine CLI 命令参考表</summary>
+
+| 语法 | 参数 | 功能说明 |
+|------|------|----------|
+| `run` | 无 | 切换任务处理循环的运行状态（启动/停止） |
+| `gpu <id>` | `<id>`: GPU编号 | 切换指定GPU的可用状态（可用/不可用） |
+| `killgpu <id>` | `<id>`: GPU编号 | 终止指定GPU上的所有进程 |
+| `kill <id>` | `<id>`: 进程ID | 终止指定进程ID的进程 |
+| `ls` | 无 | 列出所有正在运行的进程，显示进程ID、PID、任务ID、GPU ID、状态和命令 |
+| `gpus` | 无 | 显示所有GPU的状态信息，包括利用率、内存使用、温度、功耗等 |
+| `min <num>` | `<num>`: 内存大小(MB) | 设置进程所需的最小内存限制 |
+| `max <num>` | `<num>`: 进程数量 | 设置最大并发进程数 |
+| `task` | 无 | 列出待处理任务队列，显示任务ID、名称、运行次数等信息 |
+| `exit` | 无 | 退出程序（等效`Ctrl+D`） |
+| `help` 或 `?` | 无 | 显示帮助信息 |
+
+
+<details>
+<summary>命令使用示例</summary>
+
+```bash
+# 启动任务处理循环
+> run
+
+# 查看GPU状态
+> gpus
+
+# 查看运行中的进程
+> ls
+
+# 设置最大进程数为4
+> max 4
+
+# 设置最小内存要求为2048MB
+> min 2048
+
+# 禁用GPU 1
+> gpu 1
+
+# 终止GPU 0上的所有进程
+> killgpu 0
+
+# 查看待处理任务
+> task
+
+# 退出程序
+> exit
+```
+</details>
+</details>
 
 ### 🌐 使用 Web 界面（可视化任务管理）
 
@@ -164,13 +221,30 @@ python -m http.server 8000
 
 ### 🚨 风险声明
 
-> **本工具仅供学习与研究使用，使用者须自行承担一切后果。**
-
 使用本脚本可能带来的风险包括但不限于：
 
 - 与他人并发调度产生冲突，影响公平使用；
 - 若滥用，可能违反实验室/平台管理规定；
 
 开发者对因使用本脚本而导致的**资源冲突、账号受限、数据丢失或任何直接间接损失**概不负责。
 
+## 💐 贡献
+
+欢迎大家为本模板贡献代码、修正bug或完善文档！
+- 如有建议或问题，请提交Issue。
+- 欢迎提交Pull Request。
+
+> [!TIP] 
+> 若对您有帮助，请给这个项目点上 **Star**!
+
+**感谢所有贡献者！**
+
+[![贡献者](https://contrib.rocks/image?repo=dramwig/FlowLine)](https://github.com/dramwig/FlowLine/graphs/contributors)
 
+<a href="https://www.star-history.com/#dramwig/FlowLine&Date">
+ <picture>
+   <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=dramwig/FlowLine&type=Date&theme=dark" />
+   <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=dramwig/FlowLine&type=Date" />
+   <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=dramwig/FlowLine&type=Date" />
+ </picture>
+</a>

main_cli.py

@@ -9,7 +9,7 @@
 else:
     version = base_version
 
-with open("readme_en.md", "r", encoding="utf-8") as fh:
+with open("docs/readme_en.md", "r", encoding="utf-8") as fh:
     long_description = fh.read()
 
 setup(