Merge branch 'main' into pipe

Author: MayDomine

.github/workflows/build.yml

@@ -1,7 +1,7 @@
 name: Build
 
 on:
-  pull_request:
+  pull_request_target:
     types: [opened, reopened, synchronize]
     branches:
       - 'dev'

.vim-arsync

@@ -0,0 +1,11 @@
+remote_host qiyuan
+remote_user hanxv 
+remote_port '0'
+remote_path  /home/hanxv/workspace/BMTrain/
+local_path    /Users/tachicoma/projects/BMTrain/
+ignore_dotfiles 0
+ignore_path ["*.pt", "*.nsys-rep"]
+auto_sync_up    1
+remote_or_local remote
+sleep_before_sync 0
+rsync_flags ["--max-size=10m"]

README.md

@@ -28,6 +28,8 @@
 </div>
 
 ## What's New
+- 2024/02/26 **BMTrain** [1.0.0](https://github.com/OpenBMB/BMTrain/releases/tag/v1.0.0) released. Code refactoring and Tensor parallel support. See the detail in [update log](docs/UPDATE_1.0.0.md)
+- 2023/08/17 **BMTrain** [0.2.3](https://github.com/OpenBMB/BMTrain/releases/tag/v0.2.3) released. See the [update log](docs/UPDATE_0.2.3.md).
 - 2022/12/15 **BMTrain** [0.2.0](https://github.com/OpenBMB/BMTrain/releases/tag/0.2.0) released. See the [update log](docs/UPDATE_0.2.0.md).
 - 2022/06/14 **BMTrain** [0.1.7](https://github.com/OpenBMB/BMTrain/releases/tag/0.1.7) released. ZeRO-2 optimization is supported!
 - 2022/03/30 **BMTrain** [0.1.2](https://github.com/OpenBMB/BMTrain/releases/tag/0.1.2) released. Adapted to [OpenPrompt](https://github.com/thunlp/OpenPrompt)and [OpenDelta](https://github.com/thunlp/OpenDelta).
@@ -51,7 +53,7 @@ Our [documentation](https://bmtrain.readthedocs.io/en/latest/index.html) provide
 
 - From pip （recommend） : ``pip install bmtrain``
 
-- From source code: download the package and run ``python setup.py install``
+- From source code: download the package and run ``pip install .``
 
 Installing BMTrain may take a few to ten minutes, as it requires compiling the c/cuda source code at the time of installation.
 We recommend compiling BMTrain directly in the training environment to avoid potential problems caused by the different environments.
@@ -68,7 +70,6 @@ Before you can use BMTrain, you need to initialize it at the beginning of your c
 import bmtrain as bmt
 bmt.init_distributed(
     seed=0,
-    zero_level=3,   # support 2 and 3 now
     # ...
 )
 ```
@@ -118,9 +119,9 @@ class MyModule(bmt.DistributedModule): # changed here
         super().__init__()
         self.param = bmt.DistributedParameter(torch.empty(1024)) # changed here
         self.module_list = torch.nn.ModuleList([
-            bmt.Block(SomeTransformerBlock()), # changed here
-            bmt.Block(SomeTransformerBlock()), # changed here
-            bmt.Block(SomeTransformerBlock())  # changed here
+            bmt.Block(SomeTransformerBlock(), zero_level=3), # changed here, support 2 and 3 now
+            bmt.Block(SomeTransformerBlock(), zero_level=3), # changed here, support 2 and 3 now
+            bmt.Block(SomeTransformerBlock(), zero_level=3)  # changed here, support 2 and 3 now
         ])
 
     def forward(self):
@@ -181,7 +182,8 @@ class MyModule(bmt.DistributedModule):
 
     def forward(self):
         x = self.param
-        x = self.module_list(x, 1, 2, 3) # changed here
+        for module in self.module_list:
+            x = module(x, 1, 2, 3)
         return x
 
 ```