Fix vllm issues: https://github.com/Project-HAMi/HAMi/issues/1381, https://github.com/Project-HAMi/HAMi/issues/1461

Fix vllm issues: https://github.com/Project-HAMi/HAMi/issues/1381, https://github.com/Project-HAMi/HAMi/issues/1461 #45