Update inference.py

codelion · codelion · commit 820dfe81576f · 2025-08-25T11:31:54.000+08:00
diff --git a/optillm/inference.py b/optillm/inference.py
@@ -1277,8 +1277,7 @@ def get_optimized_generation_config(self, generation_params: Optional[Dict[str,
             "eos_token_id": self.tokenizer.eos_token_id,
             "return_dict_in_generate": True,
             "output_scores": generation_params.get("logprobs", False),
-            "use_cache": True,
-            "return_legacy_cache": True,  # To avoid warning
+            "use_cache": True
         }
         return config
     

Original file line number	Diff line number	Diff line change
`@@ -1277,8 +1277,7 @@ def get_optimized_generation_config(self, generation_params: Optional[Dict[str,`
`1277`	`1277`	`"eos_token_id": self.tokenizer.eos_token_id,`
`1278`	`1278`	`"return_dict_in_generate": True,`
`1279`	`1279`	`"output_scores": generation_params.get("logprobs", False),`
`1280`		`- "use_cache": True,`
`1281`		`- "return_legacy_cache": True, # To avoid warning`
	`1280`	`+ "use_cache": True`
`1282`	`1281`	`}`
`1283`	`1282`	`return config`
`1284`	`1283`