工具池 反序列化 ,工具池 tools13 ,同上一个 issue 数据集 0 ，创建文件成功，日志打印有问题

[zanguixuan3]

2025-12-31 02:04:41 | INFO | data_engine.utils.logger_utils:144 - Create logger ID 3 with loglevel: INFO, export to /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/output/log/tool_deserialize_meta_postprocess_internal_time_20251231020441.txt
2025-12-31 02:04:42 | INFO | data_engine.core.executor_tools:52 - Preparing tool...
2025-12-31 02:04:42 | INFO | data_engine.tools.base_tool:44 - Setting up data ingester...
2025-12-31 02:04:42 | INFO | data_engine.ingester.csghub_ingester:30 - Using dataset_path: /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/input, repo:longrui/tools13, branch:main
2025-12-31 02:04:42 | INFO | data_engine.tools.base_tool:55 - Preparing exporter...
2025-12-31 02:04:42 | INFO | data_engine.core.executor_tools:59 - Launching tool...
2025-12-31 02:04:42 | INFO | data_engine.ingester.csghub_ingester:41 - model_id:longrui/tools13
2025-12-31 02:04:42 | INFO | data_engine.ingester.csghub_ingester:43 - endpoint:http://modelhub.cmr-co.com
2025-12-31 02:04:42 | INFO | data_engine.ingester.csghub_ingester:44 - 入参:repo_id:longrui/tools13, repo_type:dataset, revision:main, cache_dir:/data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/input, endpoint:http://modelhub.cmr-co.com, token:b2bc8452d426461d8e4aac51b82fdebc

Downloading .gitattributes: 0%| | 0.00/2.34k [00:00<?, ?B/s]
Downloading .gitattributes: 100%|##########| 2.34k/2.34k [00:00<00:00, 2.79MB/s]

Downloading README.md: 0%| | 0.00/25.0 [00:00<?, ?B/s]
Downloading README.md: 100%|##########| 25.0/25.0 [00:00<00:00, 34.8kB/s]

Downloading data.jsonl: 0%| | 0.00/238 [00:00<?, ?B/s]
Downloading data.jsonl: 100%|##########| 238/238 [00:00<00:00, 306kB/s]
2025-12-31 02:04:42 | INFO | data_engine.ingester.csghub_ingester:54 - result: /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/input, _src_path: /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/input
2025-12-31 02:04:42 | INFO | data_engine.tools.base_tool:95 - Data ingested from /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/input
_accelerator 5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555
2025-12-31 02:04:42 | DEBUG | data_engine.tools.base_tool:137 - Op [deserialize_meta_postprocess_internal] running with number of procs:3
2025-12-31 02:04:42 | INFO | data_engine.tools.base_tool:109 - Processing tool...
/data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/input/data.jsonl
_accelerator -5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5-5
2025-12-31 02:04:42 | INFO | data_engine.tools.base_tool:114 - Tool are done in 0.204s.
2025-12-31 02:04:42 | INFO | data_engine.tools.base_tool:121 - Exporting dataset to somewhere...
2025-12-31 02:04:42 | INFO | data_engine.exporter.csghub_exporter:97 - Start to upload /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/output/_df_dataset.jsonl/_data to repo: longrui/tools13 with branch: main
2025-12-31 02:04:42 | INFO | data_engine.exporter.csghub_exporter:200 - repo longrui/tools13 all branches: ['main', 'refs-convert-parquet']
2025-12-31 02:04:42 | INFO | data_engine.exporter.csghub_exporter:153 - Start to push /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/output/_df_dataset.jsonl/_data to repo: longrui/tools13 with branch: v1,user_name: longrui, token: b2bc8452d426461d8e4aac51b82fdebc
2025-12-31 02:04:43 | INFO | data_engine.exporter.csghub_exporter:166 - Done push /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/output/_df_dataset.jsonl/_data to repo: longrui/tools13 with branch: v1
2025-12-31 02:04:43 | INFO | data_engine.exporter.csghub_exporter:169 - Remove /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/output/_git
2025-12-31 02:04:43 | INFO | data_engine.exporter.csghub_exporter:172 - Remove /data/dataflow/元数据反序列化后处理tools13source_info_a40dcf0d-403a-4577-99f1-fbaea9ec657d/output/_df_dataset.jsonl/_data
2025-12-31 02:04:43 | WARNING | data_server.job.JobExecutor:127 - Job 116 still in PROCESSING state in finally block, marking as FAILE

[zanguixuan3]

导入数据集 C:\Users\AIERXUAN\Desktop\yanshishujuji\tools7>:: 0 进入目录
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools7>cd /d C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>:: 1 初始化 LFS 并跟踪你要的大文件（按需改后缀）
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git lfs install
Updated Git hooks.
Git LFS initialized.
#你的大文件类型填全
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git lfs track ".gz" ".zip" ".tar" ".bin" ".model" "*.exe"

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>:: 2 如有旧.git先干掉
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>if exist .git rmdir /s /q .git

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git init
Initialized empty Git repository in C:/Users/AIERXUAN/Desktop/yanshishujuji/tools6/.git/

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git config --global --add safe.directory "%CD%"

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>:: 3 加远程
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git remote add origin http://账号: TOKEN@ 域名/datasets/longrui/tools6.git

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>:: 4 提交
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git add .

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git commit -m "Initial commit"
[master (root-commit) ab5ce62] Initial commit
3 files changed, 67 insertions(+)
create mode 100644 .gitattributes
create mode 100644 README.md
create mode 100644 demo.csv

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>:: 5 本地分支改名→main
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git branch -m master main

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>

C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>:: 6 强制推送
C:\Users\AIERXUAN\Desktop\yanshishujuji\tools6>git push origin main --force

yanshishujuji(1).zip