[FT] Support setting the multiple random seeds and decoupling it from model deployment? #1110
Description
Issue encountered
When I want to run the results of multiple seeds for a task, the existing solution only allows specifying the seed during the initialization of the Pipeline (when initializing VLLMModelConfig). However, for evaluating multiple seeds, it is often only necessary to specify the seed during inference. Currently, this solution requires redeployment every time a seed is specified, and the model deployment of VLLM takes a lot of extra time.
Solution/Feature
One solution is to decouple the setting of the seed from model deployment, allowing for a single deployment to perform multiple evaluations with multiple seeds and calculate the average. This is similar to a multi-task approach.
Possible alternatives
No other solutions have been thought of.