Skip to content

Commit b808c49

Browse files
yangjianfengo1yangjianfengo1
authored
[Doc] 增加中英文切换 (#3318)
* 增加中英文切换 * 增加中英文切换 * 修改readme
1 parent b21272d commit b808c49

19 files changed

+213
-15
lines changed

‎.github/workflows/gh-pages.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ jobs:
1515
- uses: actions/setup-python@v5
1616
with:
1717
python-version: 3.x
18-
- run: pip install mkdocs-material mkdocs-get-deps mkdocs-material-extensions mkdocs-multilang
18+
- run: pip install mkdocs-material mkdocs-get-deps mkdocs-material-extensions mkdocs-multilang mkdocs-static-i18n
1919
- name: Deploy to GitHub Pages
2020
env:
2121
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
English | [简体中文](README_CN.md)
12
<p align="center">
23
<a href="https://github.com/PaddlePaddle/FastDeploy/releases"><img src="https://github.com/user-attachments/assets/42b0039f-39e3-4279-afda-6d1865dfbffb" width="500"></a>
34
</p>
@@ -68,7 +69,7 @@ Learn how to use FastDeploy through our documentation:
6869
- [Offline Inference Development](./docs/offline_inference.md)
6970
- [Online Service Deployment](./docs/online_serving/README.md)
7071
- [Full Supported Models List](./docs/supported_models.md)
71-
- [Optimal Deployment](./docs/optimal_deployment/README.md)
72+
- [Best Practices](./docs/best_practices/README.md)
7273

7374
## Supported Models
7475

‎README_CN.md

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
[English](README.md) | 简体中文
2+
<p align="center">
3+
<a href="https://github.com/PaddlePaddle/FastDeploy/releases"><img src="https://github.com/user-attachments/assets/42b0039f-39e3-4279-afda-6d1865dfbffb" width="500"></a>
4+
</p>
5+
<p align="center">
6+
<a href=""><img src="https://img.shields.io/badge/python-3.10-aff.svg"></a>
7+
<a href=""><img src="https://img.shields.io/badge/os-linux-pink.svg"></a>
8+
<a href="https://github.com/PaddlePaddle/FastDeploy/graphs/contributors"><img src="https://img.shields.io/github/contributors/PaddlePaddle/FastDeploy?color=9ea"></a>
9+
<a href="https://github.com/PaddlePaddle/FastDeploy/commits"><img src="https://img.shields.io/github/commit-activity/m/PaddlePaddle/FastDeploy?color=3af"></a>
10+
<a href="https://github.com/PaddlePaddle/FastDeploy/issues"><img src="https://img.shields.io/github/issues/PaddlePaddle/FastDeploy?color=9cc"></a>
11+
<a href="https://github.com/PaddlePaddle/FastDeploy/stargazers"><img src="https://img.shields.io/github/stars/PaddlePaddle/FastDeploy?color=ccf"></a>
12+
13+
</p>
14+
15+
<p align="center">
16+
<a href="https://trendshift.io/repositories/4046" target="_blank"><img src="https://trendshift.io/api/badge/repositories/4046" alt="PaddlePaddle%2FFastDeploy | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a></br>
17+
<a href="https://paddlepaddle.github.io/FastDeploy/zh/get_started/installation/nvidia_gpu/"><b> 安装指导 </b></a>
18+
|
19+
<a href="https://paddlepaddle.github.io/FastDeploy/zh/get_started/quick_start"><b> 快速入门 </b></a>
20+
|
21+
<a href="https://paddlepaddle.github.io/FastDeploy/zh/supported_models/"><b> 支持模型列表 </b></a>
22+
23+
</p>
24+
25+
--------------------------------------------------------------------------------
26+
# FastDeploy 2.0：基于飞桨的大语言模型与视觉语言模型推理部署工具包
27+
28+
## 最新活动
29+
30+
**[2025-07] 《FastDeploy2.0推理部署实测》专题活动已上线!** 完成文心4.5系列开源模型的推理部署等任务，即可获得骨瓷马克杯等FastDeploy2.0官方周边及丰富奖金！🎁 欢迎大家体验反馈～ 📌[报名地址](https://www.wjx.top/vm/meSsp3L.aspx#) 📌[活动详情](https://github.com/PaddlePaddle/FastDeploy/discussions/2728)
31+
32+
## 关于
33+
34+
**FastDeploy** 是基于飞桨（PaddlePaddle）的大语言模型（LLM）与视觉语言模型（VLM）推理部署工具包，提供**开箱即用的生产级部署方案**，核心技术特性包括：
35+
36+
- 🚀 **负载均衡式PD分解**：工业级解决方案，支持上下文缓存与动态实例角色切换，在保障SLO达标和吞吐量的同时优化资源利用率
37+
- 🔄 **统一KV缓存传输**：轻量级高性能传输库，支持智能NVLink/RDMA选择
38+
- 🤝 **OpenAI API服务与vLLM兼容**：单命令部署，兼容[vLLM](https://github.com/vllm-project/vllm/)接口
39+
- 🧮 **全量化格式支持**：W8A16、W8A8、W4A16、W4A8、W2A16、FP8等
40+
- ⏩ **高级加速技术**：推测解码、多令牌预测（MTP）及分块预填充
41+
- 🖥️ **多硬件支持**：NVIDIA GPU、昆仑芯XPU、海光DCU、昇腾NPU、天数智芯GPU、燧原GCU、沐曦GPU等
42+
43+
44+
## 要求
45+
46+
- 操作系统: Linux
47+
- Python: 3.10 ~ 3.12
48+
49+
## 安装
50+
51+
FastDeploy 支持在**英伟达（NVIDIA）GPU**、**昆仑芯（Kunlunxin）XPU**、**天数（Iluvatar）GPU**、**燧原（Enflame）GCU** 以及其他硬件上进行推理部署。详细安装说明如下：
52+
53+
- [英伟达 GPU](./docs/zh/get_started/installation/nvidia_gpu.md)
54+
- [昆仑芯 XPU](./docs/zh/get_started/installation/kunlunxin_xpu.md)
55+
- [天数 CoreX](./docs/zh/get_started/installation/iluvatar_gpu.md)
56+
- [燧原 S60](./docs/zh/get_started/installation/Enflame_gcu.md)
57+
58+
**注意:** 我们正在积极拓展硬件支持范围。目前，包括昇腾（Ascend）NPU、海光（Hygon）DCU 和摩尔线程（MetaX）GPU 在内的其他硬件平台正在开发测试中。敬请关注更新！
59+
60+
## 入门指南
61+
62+
通过我们的文档了解如何使用 FastDeploy：
63+
- [10分钟快速部署](./docs/zh/get_started/quick_start.md)
64+
- [ERNIE-4.5 部署](./docs/zh/get_started/ernie-4.5.md)
65+
- [ERNIE-4.5-VL 部署](./docs/zh/get_started/ernie-4.5-vl.md)
66+
- [离线推理](./docs/zh/offline_inference.md)
67+
- [在线服务](./docs/zh/online_serving/README.md)
68+
- [模型支持列表](./docs/zh/supported_models.md)
69+
- [最佳实践](./docs/zh/best_practices/README.md)
70+
71+
## 支持模型列表
72+
73+
| Model | Data Type | PD Disaggregation | Chunked Prefill | Prefix Caching | MTP | CUDA Graph | Maximum Context Length |
74+
|:--- | :------- | :---------- | :-------- | :-------- | :----- | :----- | :----- |
75+
|ERNIE-4.5-300B-A47B | BF16/WINT4/WINT8/W4A8C8/WINT2/FP8 | ✅| ✅ | ✅|✅(WINT4)| WIP |128K |
76+
|ERNIE-4.5-300B-A47B-Base| BF16/WINT4/WINT8 | ✅| ✅ | ✅|✅(WINT4)| WIP | 128K |
77+
|ERNIE-4.5-VL-424B-A47B | BF16/WINT4/WINT8 | WIP | ✅ | WIP | ❌ | WIP |128K |
78+
|ERNIE-4.5-VL-28B-A3B | BF16/WINT4/WINT8 | ❌ | ✅ | WIP | ❌ | WIP |128K |
79+
|ERNIE-4.5-21B-A3B | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | WIP | ✅|128K |
80+
|ERNIE-4.5-21B-A3B-Base | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | WIP | ✅|128K |
81+
|ERNIE-4.5-0.3B | BF16/WINT8/FP8 | ❌ | ✅ | ✅ | ❌ | ✅| 128K |
82+
83+
## 进阶用法
84+
85+
- [量化](./docs/zh/quantization/README.md)
86+
- [分离式部署](./docs/zh/features/disaggregated.md)
87+
- [投机解码](./docs/zh/features/speculative_decoding.md)
88+
- [前缀缓存](./docs/zh/features/prefix_caching.md)
89+
- [分块预填充](./docs/zh/features/chunked_prefill.md)
90+
91+
## 致谢
92+
93+
FastDeploy 依据 [Apache-2.0 开源许可证](./LICENSE). 进行授权。在开发过程中，我们参考并借鉴了 [vLLM](https://github.com/vllm-project/vllm) 的部分代码，以保持接口兼容性，在此表示衷心感谢。

‎docs/optimal_deployment/ERNIE-4.5-0.3B-Paddle.md renamed to ‎docs/best_practices/ERNIE-4.5-0.3B-Paddle.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
## Environmental Preparation
33
### 1.1 Hardware requirements
44
The minimum number of GPUs required to deploy `ERNIE-4.5-0.3B` on the following hardware for each quantization is as follows:
5-
| | WINT8 | WINT4 | FP8 |
5+
6+
| | WINT8 | WINT4 | FP8 |
67
|-----|-----|-----|-----|
78
|H800 80GB| 1 | 1 | 1 |
89
|A800 80GB| 1 | 1 | / |

‎docs/optimal_deployment/ERNIE-4.5-21B-A3B-Paddle.md renamed to ‎docs/best_practices/ERNIE-4.5-21B-A3B-Paddle.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
## Environmental Preparation
33
### 1.1 Hardware requirements
44
The minimum number of GPUs required to deploy `ERNIE-4.5-21B-A3B` on the following hardware for each quantization is as follows:
5-
| | WINT8 | WINT4 | FP8 |
5+
6+
| | WINT8 | WINT4 | FP8 |
67
|-----|-----|-----|-----|
78
|H800 80GB| 1 | 1 | 1 |
89
|A800 80GB| 1 | 1 | / |

‎docs/optimal_deployment/ERNIE-4.5-300B-A47B-Paddle.md renamed to ‎docs/best_practices/ERNIE-4.5-300B-A47B-Paddle.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
## Environmental Preparation
33
### 1.1 Hardware requirements
44
The minimum number of GPUs required to deploy `ERNIE-4.5-300B-A47B` on the following hardware for each quantization is as follows:
5-
| | WINT8 | WINT4 | FP8 | WINT2 | W4A8 |
5+
6+
| | WINT8 | WINT4 | FP8 | WINT2 | W4A8 |
67
|-----|-----|-----|-----|-----|-----|
78
|H800 80GB| 8 | 4 | 8 | 2 | 4 |
89
|A800 80GB| 8 | 4 | / | 2 | 4 |

‎docs/optimal_deployment/ERNIE-4.5-VL-28B-A3B-Paddle.md renamed to ‎docs/best_practices/ERNIE-4.5-VL-28B-A3B-Paddle.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
### 1.1 Support Status
66

77
The minimum number of cards required for deployment on the following hardware is as follows:
8+
89
| Device [GPU Mem] | WINT4 | WINT8 | BFLOAT16 |
910
|:----------:|:----------:|:------:| :------:|
1011
| A30 [24G] | 2 | 2 | 4 |

‎docs/optimal_deployment/ERNIE-4.5-VL-424B-A47B-Paddle.md renamed to ‎docs/best_practices/ERNIE-4.5-VL-424B-A47B-Paddle.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
## 1. Environment Preparation
55
### 1.1 Support Status
66
The minimum number of cards required for deployment on the following hardware is as follows:
7+
78
| Device [GPU Mem] | WINT4 | WINT8 | BFLOAT16 |
89
|:----------:|:----------:|:------:| :------:|
910
| H20 [144G] | 8 | 8 | 8 |

‎docs/optimal_deployment/FAQ.md renamed to ‎docs/best_practices/FAQ.md

File renamed without changes.

‎docs/optimal_deployment/README.md renamed to ‎docs/best_practices/README.md

File renamed without changes.

0 commit comments

Comments
 (0)