Skip to content

Amd blocking test collection + test dispatching to the specific agent pool (v3) #172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
+10 −6
Open

Amd blocking test collection + test dispatching to the specific agent pool (v3) #172

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b0e60e5
Update test-template-ci.j2
Alexei-V-Ivanov-AMD Sep 24, 2025
04bc1df
Update test-template-fastcheck.j2
Alexei-V-Ivanov-AMD Sep 24, 2025
3763f02
Update test-template-ci.j2
Alexei-V-Ivanov-AMD Sep 24, 2025
93d6d6f
Update test-template-ci.j2
Alexei-V-Ivanov-AMD Sep 24, 2025
53c3ded
Update test-template-ci.j2
Alexei-V-Ivanov-AMD Oct 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions buildkite/test-template-ci.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -495,14 +495,14 @@ steps:

{% for step in steps %}
{% if step.mirror_hardwares and mirror_hw in step.mirror_hardwares %}
- label: "AMD MI300: {{ step.label }}"
- label: "AMD MI325: {{ step.label }}"
depends_on: amd-build
agents:
{% if step.label and step.label=="Benchmarks" or step.label=="Kernels Attention Test %N" or step.label=="LoRA Test %N" or step.label=="Kernels Quantization Test %N" %}
{% if step.label=="LoRA Test %N" or step.label=="Kernels Attention Test %N" or step.label=="Kernels Quantization Test %N" or step.label=="Kernels MoE Test %N" or step.label and step.label=="Benchmarks" or step.label=="Benchmarks CLI Test" or step.label=="Basic Models Tests (Extra Initialization) %N" or step.label=="Language Models Tests (Hybrid) %N" %}
queue: amd_mi325_8
{% elif step.label=="Distributed Tests (4 GPUs)" or step.label=="2 Node Tests (4 GPUs in total)" or step.label=="Multi-step Tests (4 GPUs)" or step.label=="Pipeline Parallelism Test" or step.label=="LoRA TP Test (Distributed)" %}
{% elif step.label=="Distributed Tests (4 GPUs)" or step.label=="EPLB Execution Test" or step.label=="2 Node Tests (4 GPUs in total)" or step.label=="Multi-step Tests (4 GPUs)" or step.label=="Pipeline Parallelism Test" or step.label=="LoRA TP Test (Distributed)" or step.label=="Pipeline + Context Parallelism Test" or step.label=="LoRA TP Test (Distributed)" %}
queue: amd_mi325_4
{% elif step.label=="Distributed Comm Ops Test" or step.label=="Distributed Tests (2 GPUs)" or step.label=="Plugin Tests (2 GPUs)" or step.label=="Weight Loading Multiple GPU Test" or step.label=="Weight Loading Multiple GPU Test - Large Models" %}
{% elif step.label=="Metrics, Tracing Test" or step.label=="Distributed Tests (2 GPUs)" or step.label=="Plugin Tests (2 GPUs)" or step.label=="Weight Loading Multiple GPU Test" or step.label=="Weight Loading Multiple GPU Test - Large Models" %}
queue: amd_mi325_2
{% else %}
queue: amd_mi325_1
Expand All @@ -511,7 +511,11 @@ steps:
env:
DOCKER_BUILDKIT: "1"
priority: 100
{% if step.label and step.label=="Regresson Test" %}
soft_fail: false
{% else %}
soft_fail: true
{% endif %}
{% endif %}
{% endfor %}
{% for step in steps %}
Expand Down
4 changes: 2 additions & 2 deletions buildkite/test-template-fastcheck.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -346,14 +346,14 @@ steps:
{% for step in steps %}
{% if step.mirror_hardwares and mirror_hw in step.mirror_hardwares %}
{% if step.label and step.label=="Basic Correctness Test" %}
- block: "Run AMD MI300: {{ step.label }} with {{mirror_hw}}"
- block: "Run AMD MI325: {{ step.label }} with {{mirror_hw}}"
key: block-amd-{{ step.label | replace(" ", "-") | lower | replace("(", "") | replace(")", "") | replace("%", "") | replace(",", "-") | replace("+", "-") }}
depends_on: amd-build

- label: "AMD MI300: {{ step.label }} with {{mirror_hw}}"
depends_on: block-amd-{{ step.label | replace(" ", "-") | lower | replace("(", "") | replace(")", "") | replace("%", "") | replace(",", "-") | replace("+", "-") }}
agents:
queue: amd_mi300_1
queue: amd_mi325_1
command: bash .buildkite/scripts/hardware_ci/run-amd-test.sh "(command rocm-smi || true) && export VLLM_ALLOW_DEPRECATED_BEAM_SEARCH=1 && cd {{ (step.working_dir or default_working_dir) | safe }} ; {{ step.command or (step.commands | join(" && ")) | safe }}"
env:
DOCKER_BUILDKIT: "1"
Expand Down