Merge remote-tracking branch 'paddlenlp/develop' into dev_20241029_add_memory_count

Author: DrownFish19

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,4 +1,18 @@
 <!-- Demo: https://github.com/PaddlePaddle/PaddleNLP/pull/26 -->
+#### Before submitting
+
+- [ ] Lint code. If there are lint issues, please format the code first.
+
+```shell
+# Install and register `pre-commit` in the project folder
+pip install pre-commit && pre-commit install
+
+# Process previous code files separately
+pre-commit run --file XXXX.py
+```
+
+- [ ] Add test cases into `tests` folder. If there are codecov issues, please add tests cases first.
+
 ### PR types
 <!-- One of [ New features | Bug fixes | Function optimization | Performance optimization | Breaking changes | Others ] -->
 

.gitignore

@@ -129,3 +129,9 @@ FETCH_HEAD
 csrc/third_party/
 dataset/
 output/
+
+# gen codes
+autogen/
+
+# cutlass kernel
+!csrc/gpu/cutlass_kernels/gemm/collective/builders

.gitmodules

@@ -0,0 +1,6 @@
+[submodule "csrc/third_party/cutlass"]
+	path = csrc/third_party/cutlass
+	url = https://github.com/NVIDIA/cutlass.git
+[submodule "csrc/third_party/nlohmann_json"]
+	path = csrc/third_party/nlohmann_json
+	url = https://github.com/nlohmann/json.git

.pre-commit-config.yaml

@@ -1,4 +1,4 @@
-exclude: 'slm/model_zoo/gpt-3'
+exclude: 'slm/model_zoo/gpt-3;csrc/third_party'
 repos:
 # For Python files
 -   repo: https://github.com/psf/black.git
@@ -61,4 +61,4 @@ repos:
         entry: python scripts/codestyle/check_dead_links.py
         language: python
         files: \.(md|markdown|rst)$
-        pass_filenames: true
+        pass_filenames: true

README.md

@@ -10,32 +10,15 @@ pip install -r requirements.txt
 
 ## 编译 Cuda 算子
 
-生成 FP8的 cutlass 算子
-```shell
-python utils/auto_gen_fp8_fp8_gemm_fused_kernels.py
-
-python utils/auto_gen_fp8_fp8_dual_gemm_fused_kernels.py
-```
-
-编译
-```shell
-python setup_cuda.py install
-```
-
-### 手动安装 Cutlass 库
-1. 访问 Cutlass 仓库: [NVIDIA/cutlass](https://github.com/NVIDIA/cutlass)
-
-2. 拉取代码:
-    git clone -b v3.5.0 --single-branch https://github.com/NVIDIA/cutlass.git
-
-3. 将下载的 `cutlass` 目录放在 `csrc/third_party/cutlass`下
-
-4. 重新编译 Cuda 算子
 ```shell
 python setup_cuda.py install
 ```
 
 ### FP8 GEMM 自动调优
+
+确保 `cutlass` 库已经安装，然后执行以下命令进行自动调优。
+- 对于89架构的 GPU，CUDA 版本至少为12.4
+- 对于90架构的 GPU，CUDA 版本至少为12.0
 ```shell
 sh tune_fp8_gemm.sh
 ```

csrc/README.md

@@ -15,20 +15,9 @@
 #include <stdlib.h>
 #include <string.h>
 
-#include "paddle/extension.h"
+#include "helper.h"
 #include <stdio.h>
 
-
-bool is_in_end(const int64_t id, const int64_t* end_ids, int length) {
-  bool flag = false;
-  for (int i = 0; i < length; i++) {
-    if (id == end_ids[i]) {
-      return true;
-    }
-  }
-  return flag;
-}
-
 void set_value_by_flags(const bool* stop_flags,
                         const int64_t* end_ids,
                         int64_t* topk_ids,