You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+27-42Lines changed: 27 additions & 42 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,7 +14,7 @@
14
14
15
15
# Introduction
16
16
17
-
SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) is an extensible framwork designed to faciliate lightweight model fine-tuning. It integrates implementations for various efficient fine-tuning methods, by embracing approaches that is parameter-efficient, memory-efficient, and time-efficient. SWIFT integrates seamlessly into ModelScope ecosystem and offers the capabilities to finetune various modles, with a primary emphasis on LLMs and vision models. Additionally, SWIFT is fully compatible with [Peft](https://github.com/huggingface/peft), enabling users to leverage the familiar Peft interface to finetune ModelScope models.
17
+
SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) is an extensible framwork designed to faciliate lightweight model fine-tuning and inference. It integrates implementations for various efficient fine-tuning methods, by embracing approaches that is parameter-efficient, memory-efficient, and time-efficient. SWIFT integrates seamlessly into ModelScope ecosystem and offers the capabilities to finetune various models, with a primary emphasis on LLMs and vision models. Additionally, SWIFT is fully compatible with [PEFT](https://github.com/huggingface/peft), enabling users to leverage the familiar Peft interface to finetune ModelScope models.
18
18
19
19
Currently supported approches (and counting):
20
20
@@ -23,77 +23,74 @@ Currently supported approches (and counting):
4. Side: [Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks](https://arxiv.org/abs/1912.13503)
25
25
5. ResTuning-Bypass
26
-
7. All tuners offered on [Peft](https://github.com/huggingface/peft)
26
+
7. All tuners offered on [PEFT](https://github.com/huggingface/peft)
27
27
28
28
Key features:
29
29
30
30
1. By integrating the ModelScope library, models can be readily obatined via a model-id.
31
-
2. Tuners provided by SWIFT be combined together to allow exploration of multiple tuners on a model for best result.
32
-
3. Support calling `activate_adapter`或`deactivate_adapter` to activate/deactivate a single tuner. User can use one model with multiple tuners in different threads.
31
+
2. Tuners provided by SWIFT can be combined together to allow exploration of multiple tuners on a model for best result.
32
+
3. Support calling `activate_adapter` or `deactivate_adapter`or `set_active_adapters`to activate/deactivate tuners. User can inference with one model and multiple tuners in different threads independently.
33
33
34
-
Users can check the [documentation of Swift](./docs/Get Started/1.Introduction.md) to get detail tutorials.
34
+
Users can check the [documentation of Swift](docs/source/GetStarted/Introduction.md) to get detail tutorials.
SWIFT supports multiple tuners, as well as tuners provided by [Peft](https://github.com/huggingface/peft). To use the these tuners, simply call:
85
+
SWIFT supports multiple tuners, as well as tuners provided by [PEFT](https://github.com/huggingface/peft). To use these tuners, simply call:
90
86
91
87
```python
92
-
from swift import Swift
88
+
from swift import Swift, LoRAConfig
89
+
config = LoRAConfig(...)
93
90
model = Swift.prepare_model(model, config, extra_state_keys=['...'])
94
91
```
95
92
96
-
The code snippet above initialized the tuner randomly. The input model is an instance of `torch.nn.Module`, config is a subclass instance of `SwiftConfig` or `PeftConfig`. extra_state_keys is
93
+
The code snippet above initialized the tuner randomly. The input model is an instance of `torch.nn.Module`, the config is a subclass instance of `SwiftConfig` or `PeftConfig`. extra_state_keys is
97
94
the extra module weights(like the linear head) to be trained and stored in the output dir.
98
95
99
96
You may combine multiple tuners by:
@@ -103,7 +100,7 @@ from swift import Swift, LoRAConfig, PromptConfig
103
100
model = Swift.prepare_model(model, {'lora': LoRAConfig(...), 'prompt': PromptConfig(...)})
104
101
```
105
102
106
-
You can all`save_pretrained` and `push_to_hub` after finetuning:
103
+
Call`save_pretrained` and `push_to_hub` after finetuning:
The saving strategy between Swift tuners and Peft tuners are slightly different. You can name a tuner of a SWIFT by:
200
+
The saving strategy between Swift tuners and Peft tuners are slightly different. You can name a tuner by:
216
201
217
202
```python
218
203
model = Swift.prepare_model(model, {'default': LoRAConfig(...)})
@@ -230,7 +215,7 @@ output
230
215
|-- adapter_model.bin
231
216
```
232
217
233
-
The config/weights stored in the output dir is the config of `extra_state_keys` and the weights of it. This is different from Peft, which stores the weights and config of the `default` tuner.
218
+
The config/weights stored in the output dir is the config of `extra_state_keys` and the weights of it. This is different from PEFT, which stores the weights and config of the `default` tuner.
0 commit comments