ResTuning readme (#129)

Co-authored-by: zeyinzi.jzyz <[email protected]>

Author: jiangzeyinzi

README.md

@@ -24,7 +24,7 @@ Currently supported approches (and counting):
 4. Adapter: [Parameter-Efficient Transfer Learning for NLP](http://arxiv.org/abs/1902.00751)
 5. Prompt Tuning: [Visual Prompt Tuning](https://arxiv.org/abs/2203.12119)
 6. Side: [Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks](https://arxiv.org/abs/1912.13503)
-7. ResTuning-Bypass
+7. Res-Tuning: [Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone](https://arxiv.org/abs/2310.19859)  < [arXiv](https://arxiv.org/abs/2310.19859)  |  [Project Page](https://res-tuning.github.io/)  |  [Usage](docs/source/GetStarted/ResTuning.md) >
 8. ROME: [Rank-One Editing of Encoder-Decoder Models](https://arxiv.org/abs/2211.13317)
 9. All tuners offered on [PEFT](https://github.com/huggingface/peft)
 

README_CN.md

@@ -23,7 +23,7 @@ SWIFT（Scalable lightWeight Infrastructure for Fine-Tuning）是一个可扩展
 4. Adapter：[Parameter-Efficient Transfer Learning for NLP](http://arxiv.org/abs/1902.00751)
 5. Prompt: [Visual Prompt Tuning](https://arxiv.org/abs/2203.12119)
 6. Side: [Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks](https://arxiv.org/abs/1912.13503)
-7. ResTuning-Bypass
+7. Res-Tuning: [Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone](https://arxiv.org/abs/2310.19859)  < [arXiv](https://arxiv.org/abs/2310.19859)  |  [Project Page](https://res-tuning.github.io/)  |  [Usage](docs/source/GetStarted/ResTuning.md) >
 8. ROME: [Rank-One Editing of Encoder-Decoder Models](https://arxiv.org/abs/2211.13317)
 9. 所有在[PEFT](https://github.com/huggingface/peft)上提供的tuners
 

docs/source/GetStarted/ResTuning.md

@@ -0,0 +1,63 @@
+<div align="center">
+
+## [NeurIPS 2023] Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone
+
+### [arXiv](https://arxiv.org/abs/2310.19859)  |  [Project Page](https://res-tuning.github.io/)
+
+</div>
+
+Res-Tuning is a flexible and efficient tuning paradigm. We manage to free the design of tuners from the network architecture, facilitating flexible combination of various tuning strategies and further extend a memory-efficient bypass variant, which significantly reduces the memory consumption and multi-task inference cost.
+
+The implementation is a pluggable tuner component for [SWIFT](https://github.com/modelscope/swift), designed to be user-friendly.
+
+### Catalog
+
+- [x] Res-Adapter
+- [x] Res-Tuning-Bypass
+- [ ] Res-Prefix
+- [ ] Res-Prompt
+
+### Usage
+
+#### Demo
+- Run our interactive demo using [vision_example](https://github.com/modelscope/swift/blob/main/examples/pytorch/cv/notebook/swift_vision.ipynb).
+
+#### Init Tuner
+
+```Python
+from swift import ResTuningConfig
+config = ResTuningConfig(
+    dims=768,
+    root_modules=r'.*blocks.0$',
+    stem_modules=r'.*blocks\.\d+$',
+    target_modules=r'norm',
+    tuner_cfg='res_adapter'
+)
+```
+- dims: The dimensions of the hidden states.
+- root_modules: The root module to be replaced.
+- stem_modules: The stem modules to be replaced.
+- target_modules: The target module to be replaced.
+- tuner_cfg: The configuration of the tuning module.
+
+#### Load Model
+
+```Python
+from swift import Swift
+import timm, torch
+model = timm.create_model("vit_base_patch16_224", pretrained=False, num_classes=100)
+model_tune = Swift.prepare_model(model, config)
+print(model_tune.get_trainable_parameters())
+print(model(torch.ones(1, 3, 224, 224)).shape)
+```
+
+
+### Citation
+```
+@inproceedings{jiang2023restuning,
+  title={Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone},
+  author={Jiang, Zeyinzi and Mao, Chaojie and Huang, Ziyuan and Ma, Ao and Lv, Yiliang and Shen, Yujun and Zhao, Deli and Zhou, Jingren},
+  booktitle={Advances in Neural Information Processing Systems},
+  year={2023}
+}
+```