update web ui and swift cli (#126)

Author: Jintao-Huang

README.md

@@ -12,7 +12,7 @@
         <a href="README_CN.md">中文</a>&nbsp ｜ &nbspEnglish
 </p>
 
-# Introduction
+# 📖 Introduction
 
 SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) is an extensible framwork designed to faciliate lightweight model fine-tuning and inference. It integrates implementations for various efficient fine-tuning methods,  by embracing approaches that is parameter-efficient, memory-efficient, and time-efficient. SWIFT integrates seamlessly into ModelScope ecosystem and offers the capabilities to finetune various models, with a primary emphasis on LLMs and vision models. Additionally, SWIFT is fully compatible with [PEFT](https://github.com/huggingface/peft), enabling users to  leverage the familiar Peft interface to finetune ModelScope models.
 
@@ -36,7 +36,7 @@ Key features:
 
 Users can check the [documentation of Swift](docs/source/GetStarted/Introduction.md) to get detail tutorials.
 
-### 🎉News
+### 🎉 News
 
 - 🔥 2023.10.30: Support QA-LoRA and LongLoRA to decrease memory usage in training.
 - 🔥 2023.10.30: Support ROME(Rank One Model Editing) to add/modify knowledges, training is not needed!
@@ -48,10 +48,12 @@ Users can check the [documentation of Swift](docs/source/GetStarted/Introduction
 - 🔥 2023.9.25: Supported qwen-14b model series: qwen-14b, qwen-14b-chat. The corresponding shell script can be found at `scripts/qwen_14b`, `scripts/qwen_14b_chat`.
 - 2023.9.12: Supported training with MP+DDP to accelerate full-parameter fine-tuning speed. The corresponding shell script can be found at `scripts/qwen_7b_chat/full_mp_ddp/sft.sh`.
 
-## LLM SFT Example
+## ✨ LLM SFT Example
 Press [this link](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm) to view the detail documentation of these examples.
 
 ### Basic Usage
+Quickly fine-tune, infer with LLM, and build a Web-UI.
+#### Run using Python
 ```bash
 git clone https://github.com/modelscope/swift.git
 cd swift
@@ -66,8 +68,10 @@ os.environ['CUDA_VISIBLE_DEVICES'] = '0'
 
 import torch
 
-from swift.llm import DatasetName, InferArguments, ModelType, SftArguments
-from swift.llm.run import infer_main, sft_main
+from swift.llm import (
+    DatasetName, InferArguments, ModelType, SftArguments
+)
+from swift.llm.run import infer_main, sft_main, web_ui_main
 
 model_type = ModelType.qwen_7b_chat_int4
 sft_args = SftArguments(
@@ -81,14 +85,44 @@ best_ckpt_dir = sft_main(sft_args)
 print(f'best_ckpt_dir: {best_ckpt_dir}')
 torch.cuda.empty_cache()
 infer_args = InferArguments(
-    model_type=sft_args.model_type,
     ckpt_dir=best_ckpt_dir,
-    dataset=sft_args.dataset,
+    load_args_from_ckpt_dir=True,
     stream=True,
     show_dataset_sample=5)
 infer_main(infer_args)
+torch.cuda.empty_cache()
+web_ui_main(infer_args)
+```
+
+#### Run using Swift CLI
+**SFT**:
+```bash
+# Experimental environment: A10, 3090, A100, ...
+# 10GB GPU memory
+CUDA_VISIBLE_DEVICES=0 swift sft --model_id_or_path qwen/Qwen-7B-Chat-Int4 --dataset blossom-math-zh
+
+# Using DDP
+# Experimental environment: 2 * 3090
+# 2 * 10GB GPU memory
+CUDA_VISIBLE_DEVICES=0,1 \
+NPROC_PER_NODE=2 \
+swift sft \
+    --model_id_or_path qwen/Qwen-7B-Chat-Int4 \
+    --dataset blossom-math-zh \
+
+# Using custom dataset
+CUDA_VISIBLE_DEVICES=0 swift sft --model_id_or_path qwen/Qwen-7B-Chat-Int4 --custom_train_dataset_path chatml.jsonl
 ```
 
+**Inference**:
+```bash
+CUDA_VISIBLE_DEVICES=0 swift infer --ckpt_dir 'xxx/vx_xxx/checkpoint-xxx'
+```
+
+**Web-UI**:
+```bash
+CUDA_VISIBLE_DEVICES=0 swift web-ui --ckpt_dir 'xxx/vx_xxx/checkpoint-xxx'
+```
 
 ### Features
 - Supported SFT Methods: [lora](https://arxiv.org/abs/2106.09685), [qlora](https://arxiv.org/abs/2305.14314), full(full parameter fine-tuning)
@@ -125,7 +159,7 @@ infer_main(infer_args)
   - Chat: chatml(qwen), baichuan, chatglm2, chatglm3, llama, openbuddy-llama, default, internlm, xverse, skywork
 
 
-# Installation
+# 🛠️ Installation
 
 SWIFT is running in Python environment. Please make sure your python version is higher than 3.8.
 
@@ -151,7 +185,7 @@ SWIFT requires torch>=1.13.
 docker pull registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.8.0-py38-torch2.0.1-tf2.13.0-1.9.1
 ```
 
-# Getting Started
+# 🚀 Getting Started
 
 SWIFT supports multiple tuners, as well as tuners provided by [PEFT](https://github.com/huggingface/peft). To use these tuners, simply call:
 
@@ -289,7 +323,7 @@ output
 The config/weights stored in the output dir is the config of `extra_state_keys` and the weights of it. This is different from PEFT, which stores the weights and config of the `default` tuner.
 
 
-# Learn More
+# 🔍 Learn More
 
 - [ModelScope library](https://github.com/modelscope/modelscope/)
 

README_CN.md

@@ -12,7 +12,7 @@
         中文&nbsp ｜ &nbsp<a href="README.md">English</a>
 </p>
 
-# 简介
+# 📖 简介
 SWIFT（Scalable lightWeight Infrastructure for Fine-Tuning）是一个可扩展的轻量级一站式训练、推理深度学习框架。它集成了各种高效的微调方法，如LoRA、QLoRA、阿里云自研的ResTuning-Bypass等，以及开箱即用的训练推理脚本，使开发者可以在单张商业级显卡上微调推理LLM&AIGC模型。此外，SWIFT与[PEFT](https://github.com/huggingface/peft)完全兼容，使开发者可以在ModelScope模型体系中使用PEFT的能力。
 
 目前支持的方法：
@@ -34,7 +34,7 @@ SWIFT（Scalable lightWeight Infrastructure for Fine-Tuning）是一个可扩展
 
 用户可以查看 [Swift官方文档](docs/source/GetStarted/Introduction.md) 来了解详细信息。
 
-## 新闻
+## 🎉 新闻
 
 - 🔥 2023.10.30: 支持 QA-LoRA 和 LongLoRA两种新的tuners
 - 🔥 2023.10.30: 支持使用ROME(Rank One Model Editing)来编辑模型，在无需训练的情况下即可给模型灌注新知识！
@@ -46,10 +46,12 @@ SWIFT（Scalable lightWeight Infrastructure for Fine-Tuning）是一个可扩展
 - 🔥 2023.9.25: 支持**qwen-14b**系列模型: qwen-14b, qwen-14b-chat. 对应的sh脚本可以查看`scripts/qwen_14b`, `scripts/qwen_14b_chat`.
 - 2023.9.12: 支持MP+DDP的方式训练, 加快全参数微调的速度, 对应的sh脚本可以查看`scripts/qwen_7b_chat/full_mp_ddp/sft.sh`.
 
-## 大模型微调的例子
+## ✨ 大模型微调的例子
 可以[在这里](https://github.com/modelscope/swift/tree/main/examples/pytorch/llm) 查看LLM微调的使用文档。
 
 ### 简单使用
+快速对LLM进行微调, 推理并搭建Web-UI.
+#### 使用python运行
 ```bash
 git clone https://github.com/modelscope/swift.git
 cd swift
@@ -64,8 +66,10 @@ os.environ['CUDA_VISIBLE_DEVICES'] = '0'
 
 import torch
 
-from swift.llm import DatasetName, InferArguments, ModelType, SftArguments
-from swift.llm.run import infer_main, sft_main
+from swift.llm import (
+    DatasetName, InferArguments, ModelType, SftArguments
+)
+from swift.llm.run import infer_main, sft_main, web_ui_main
 
 model_type = ModelType.qwen_7b_chat_int4
 sft_args = SftArguments(
@@ -79,12 +83,43 @@ best_ckpt_dir = sft_main(sft_args)
 print(f'best_ckpt_dir: {best_ckpt_dir}')
 torch.cuda.empty_cache()
 infer_args = InferArguments(
-    model_type=sft_args.model_type,
     ckpt_dir=best_ckpt_dir,
-    dataset=sft_args.dataset,
+    load_args_from_ckpt_dir=True,
     stream=True,
     show_dataset_sample=5)
 infer_main(infer_args)
+torch.cuda.empty_cache()
+web_ui_main(infer_args)
+```
+
+#### 使用Swift CLI运行
+**微调**:
+```bash
+# Experimental environment: A10, 3090, A100, ...
+# 10GB GPU memory
+CUDA_VISIBLE_DEVICES=0 swift sft --model_id_or_path qwen/Qwen-7B-Chat-Int4 --dataset blossom-math-zh
+
+# 使用DDP
+# Experimental environment: 2 * 3090
+# 2 * 10GB GPU memory
+CUDA_VISIBLE_DEVICES=0,1 \
+NPROC_PER_NODE=2 \
+swift sft \
+    --model_id_or_path qwen/Qwen-7B-Chat-Int4 \
+    --dataset blossom-math-zh \
+
+# 使用自己的数据集
+CUDA_VISIBLE_DEVICES=0 swift sft --model_id_or_path qwen/Qwen-7B-Chat-Int4 --custom_train_dataset_path chatml.jsonl
+```
+
+**推理**:
+```bash
+CUDA_VISIBLE_DEVICES=0 swift infer --ckpt_dir 'xxx/vx_xxx/checkpoint-xxx'
+```
+
+**Web-UI**
+```bash
+CUDA_VISIBLE_DEVICES=0 swift web-ui --ckpt_dir 'xxx/vx_xxx/checkpoint-xxx'
 ```
 
 
@@ -123,7 +158,7 @@ infer_main(infer_args)
   - 对话: chatml(qwen), baichuan, chatglm2, chatglm3, llama, openbuddy-llama, default, internlm, xverse, skywork
 
 
-# 安装
+# 🛠️ 安装
 
 SWIFT在Python环境中运行。请确保您的Python版本高于3.8。
 
@@ -149,7 +184,7 @@ SWIFT依赖torch>=1.13。
 docker pull registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.8.0-py38-torch2.0.1-tf2.13.0-1.9.1
 ```
 
-# 快速开始
+# 🚀 快速开始
 SWIFT支持多个tuners，包括由[PEFT](https://github.com/huggingface/peft)提供的tuners。要使用这些tuners，只需调用:
 ```python
 from swift import Swift, LoRAConfig
@@ -281,7 +316,7 @@ output
 存储在output目录中的config/weights是extra_state_keys的配置和权重。这与Peft不同，Peft存储了`default` tuner的config/weights。
 
 
-# Learn More
+# 🔍 Learn More
 
 - [ModelScope库](https://github.com/modelscope/modelscope/)