File tree Expand file tree Collapse file tree 2 files changed +9
-8
lines changed Expand file tree Collapse file tree 2 files changed +9
-8
lines changed Original file line number Diff line number Diff line change @@ -75,12 +75,12 @@ Learn how to use FastDeploy through our documentation:
75
75
76
76
| Model | Data Type | PD Disaggregation | Chunked Prefill | Prefix Caching | MTP | CUDA Graph | Maximum Context Length |
77
77
| :--- | :------- | :---------- | :-------- | :-------- | :----- | :----- | :----- |
78
- | ERNIE-4.5-300B-A47B | BF16/WINT4/WINT8/W4A8C8/WINT2/FP8 | ✅| ✅ | ✅| ✅(WINT4) | WIP | 128K |
79
- | ERNIE-4.5-300B-A47B-Base| BF16/WINT4/WINT8 | ✅| ✅ | ✅| ✅(WINT4) | WIP | 128K |
78
+ | ERNIE-4.5-300B-A47B | BF16/WINT4/WINT8/W4A8C8/WINT2/FP8 | ✅| ✅ | ✅| ✅| WIP | 128K |
79
+ | ERNIE-4.5-300B-A47B-Base| BF16/WINT4/WINT8 | ✅| ✅ | ✅| ❌ | WIP | 128K |
80
80
| ERNIE-4.5-VL-424B-A47B | BF16/WINT4/WINT8 | WIP | ✅ | WIP | ❌ | WIP | 128K |
81
81
| ERNIE-4.5-VL-28B-A3B | BF16/WINT4/WINT8 | ❌ | ✅ | WIP | ❌ | WIP | 128K |
82
- | ERNIE-4.5-21B-A3B | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | WIP | ✅| 128K |
83
- | ERNIE-4.5-21B-A3B-Base | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | WIP | ✅| 128K |
82
+ | ERNIE-4.5-21B-A3B | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | ✅ | ✅| 128K |
83
+ | ERNIE-4.5-21B-A3B-Base | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | ❌ | ✅| 128K |
84
84
| ERNIE-4.5-0.3B | BF16/WINT8/FP8 | ❌ | ✅ | ✅ | ❌ | ✅| 128K |
85
85
86
86
## Advanced Usage
Original file line number Diff line number Diff line change 1
1
[ English] ( README.md ) | 简体中文
2
+ [ English] ( README.md ) | 简体中文
2
3
<p align =" center " >
3
4
<a href =" https://github.com/PaddlePaddle/FastDeploy/releases " ><img src =" https://github.com/user-attachments/assets/42b0039f-39e3-4279-afda-6d1865dfbffb " width =" 500 " ></a >
4
5
</p >
@@ -72,12 +73,12 @@ FastDeploy 支持在**英伟达(NVIDIA)GPU**、**昆仑芯(Kunlunxin)XPU
72
73
73
74
| Model | Data Type | PD Disaggregation | Chunked Prefill | Prefix Caching | MTP | CUDA Graph | Maximum Context Length |
74
75
| :--- | :------- | :---------- | :-------- | :-------- | :----- | :----- | :----- |
75
- | ERNIE-4.5-300B-A47B | BF16/WINT4/WINT8/W4A8C8/WINT2/FP8 | ✅| ✅ | ✅| ✅(WINT4) | WIP | 128K |
76
- | ERNIE-4.5-300B-A47B-Base| BF16/WINT4/WINT8 | ✅| ✅ | ✅| ✅(WINT4) | WIP | 128K |
76
+ | ERNIE-4.5-300B-A47B | BF16/WINT4/WINT8/W4A8C8/WINT2/FP8 | ✅| ✅ | ✅| ✅| WIP | 128K |
77
+ | ERNIE-4.5-300B-A47B-Base| BF16/WINT4/WINT8 | ✅| ✅ | ✅| ❌ | WIP | 128K |
77
78
| ERNIE-4.5-VL-424B-A47B | BF16/WINT4/WINT8 | WIP | ✅ | WIP | ❌ | WIP | 128K |
78
79
| ERNIE-4.5-VL-28B-A3B | BF16/WINT4/WINT8 | ❌ | ✅ | WIP | ❌ | WIP | 128K |
79
- | ERNIE-4.5-21B-A3B | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | WIP | ✅| 128K |
80
- | ERNIE-4.5-21B-A3B-Base | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | WIP | ✅| 128K |
80
+ | ERNIE-4.5-21B-A3B | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | ✅ | ✅| 128K |
81
+ | ERNIE-4.5-21B-A3B-Base | BF16/WINT4/WINT8/FP8 | ❌ | ✅ | ✅ | ❌ | ✅| 128K |
81
82
| ERNIE-4.5-0.3B | BF16/WINT8/FP8 | ❌ | ✅ | ✅ | ❌ | ✅| 128K |
82
83
83
84
## 进阶用法
You can’t perform that action at this time.
0 commit comments