fix use_cache (#3487)

Author: Jintao-Huang

docs/source/Instruction/ReleaseNote3.0.md

@@ -78,9 +78,3 @@
 4. merge_lora的存储目录可以通过`--output_dir`指定了，且merge_lora和量化不能在一个命令中执行，需要最少两个命令
 5. 使用`swift app --model xxx`开启app-ui界面，支持了多模态界面推理
 6. 移除了AIGC的依赖以及对应的examples和训练代码
-
-## 待完成
-
-1. 自定义数据集评测3.0版本尚不支持，请使用2.6.1版本
-2. Megatron预训练能力3.0版本尚不支持，请使用2.6.1版本
-3. 文档和README暂时未更新完整

docs/source_en/Instruction/ReleaseNote3.0.md

@@ -91,9 +91,3 @@ The parameters marked as compatible in version 2.0 have been entirely removed.
 5. Use `swift app --model xxx` to launch the app-ui interface, which supports multimodal interface inference.
 
 6. Removed dependencies for AIGC along with corresponding examples and training code.
-
-## Pending Tasks
-
-1. Custom dataset evaluation is not supported in version 3.0. Please use version 2.6.1.
-2. Megatron pre-training capabilities are not supported in version 3.0. Please use version 2.6.1.
-3. Documentation and README are temporarily incomplete and will be updated.

swift/llm/train/sft.py

@@ -34,11 +34,10 @@ def __init__(self, args: Union[List[str], TrainArguments, None] = None) -> None:
 
     def _prepare_gradient_checkpointing(self):
         args = self.args
-
+        self.model.config.use_cache = False
         if args.gradient_checkpointing:
             self.model.supports_gradient_checkpointing = True
             dynamic_gradient_checkpointing(self.model)
-            self.model.config.use_cache = False  # fix transformers==4.36
             self.model.enable_input_require_grads()
         model_meta = self.model.model_meta
         model_arch = get_model_arch(model_meta.model_arch)

swift/trainers/mixin.py

@@ -224,7 +224,7 @@ def _save(self, output_dir: Optional[str] = None, state_dict=None):
             from swift.llm import save_checkpoint
             additional_saved_files = self.model_meta.additional_saved_files
             save_checkpoint(None, self.template.processor, output_dir, additional_saved_files=additional_saved_files)
-            if hasattr(self.model, 'origin_generation_config'):
+            if getattr(self.model, 'origin_generation_config', None):
                 self.model.origin_generation_config.save_pretrained(output_dir)
 
     def _fix_zero3_gather_all_parameters(self) -> None: