feat(platforms): add MetaX MACA support and vllm-upstream extra#2596
Open
Dayuxiaoshui wants to merge 6 commits intovllm-project:mainfrom
Open
feat(platforms): add MetaX MACA support and vllm-upstream extra#2596Dayuxiaoshui wants to merge 6 commits intovllm-project:mainfrom
Dayuxiaoshui wants to merge 6 commits intovllm-project:mainfrom
Conversation
- Add MacaOmniPlatform on vllm_metax with maca workers (CUDA-compatible path). - Register maca platform plugin; setup.py detects maca before generic CUDA; add requirements/maca.txt. - Add OmniPlatformEnum.MACA, forward_maca / is_maca branches for diffusion and int8. - Add optional dependency extra vllm-upstream (vllm==0.19.0) for PyPI CUDA stacks; do not mix with vllm-metax in the same env.
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Remove vllm-upstream dependency and update notes. Signed-off-by: Dayuxiaoshui <158081477+Dayuxiaoshui@users.noreply.github.com>
Signed-off-by: Dayuxiaoshui <158081477+Dayuxiaoshui@users.noreply.github.com>
Signed-off-by: Dayuxiaoshui <158081477+Dayuxiaoshui@users.noreply.github.com>
…detection (drop undefined _nvidia_nvml_gpu_count_for_setup; maca = vllm_metax + torch.cuda only; use VLLM_OMNI_TARGET_DEVICE if multiple plugins match)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add MacaOmniPlatform on vllm_metax with maca workers (CUDA-compatible path).
Add OmniPlatformEnum.MACA, forward_maca / is_maca branches for diffusion and int8.
Purpose
MacaOmniPlatformbuilt onvllm_metax.platform.MacaPlatform, dedicated worker entrypoints, and platform auto-detection that avoids colliding with NVIDIA CUDA when NVML reports discrete NVIDIA GPUs.maca(requirements/maca.txt,setup.py/VLLM_OMNI_TARGET_DEVICE=maca, and maca-before-cuda heuristic whenvllm_metaxis importable).is_maca()andforward_maca()(aligned with other platforms instead of folding maca intois_cuda()).vllm-upstreamextra (vllm==0.19.0) for environments that install upstream vLLM from PyPI; document that it must not be mixed withvllm-metaxin the same environment.Test Plan
python -m compileall -q vllm_omni(andpython -m buildwithVLLM_OMNI_TARGET_DEVICE=cpuif validating wheel metadata).pytest tests/distributed/omni_connectors/test_kv_flow.py -q— exercises omni connector logic without requiring MACA hardware.pip install -e .withVLLM_OMNI_TARGET_DEVICE=maca(or rely on auto-detection), and smoke-runvllm serve <model> --omnior a minimal omni offline script from docs.Reason we do not add MACA-specific CI in this PR: full correctness depends on vendor images and vLLM-metax version alignment, which cannot be reproduced in generic upstream CI without dedicated runners.
Test Result
compileallOK;pytest tests/distributed/omni_connectors/test_kv_flow.py18 passed (CPU, nolibcuda).vllm_metax+ vLLM.Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model. Please runmkdocs serveto sync the documentation editions to./docs. (Not done in this PR; MACA install path is vendor-documented. Follow-up: add a short GPU install tab for MACA if maintainers want parity with MUSA.)BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)