[1/N][port from deepseek085] add custom allreduce from AITER#629
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
5ee9cab to
d785596
Compare
d785596 to
555aff3
Compare
control it by the env flag VLLM_ROCM_USE_AITER_CUSTOM_ALL_REDUCE (default: True) Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
555aff3 to
37e91e2
Compare
|
LGTM @zejunchen-zejun Thank you for the feature. |


Sync deepseek085 optimization to rocm/vllm llama branch.
Will upstream same code changes to public vllm.
The custom allreduce is controlled by VLLM_ROCM_USE_AITER_CUSTOM_ALL_REDUCE(default: True).
If AITER is imported, the custom allreduce will be default used.