Support b300 Instance by hgreebe · Pull Request #7210 · aws/aws-parallelcluster

hgreebe · 2026-01-27T13:07:07Z

Description of changes

Support b300 Instance
Have network interface configuration default for b300 where primary interface use an ENA network interface and all others use efa-only (Configuration is use case 1 from this documentation)
Add support for running test_efa with b300 instance
Upgrade NCCL version to 2.28.9
Upgrade OFI NCCL plugin version to 1.18.0
Upgrade Fabtest version to 2.4.0

Tests

Ran test_efa with ubuntu2404, alinux2023, and rocky9
Ran gpu health checks
Compared NCCL benchmark results with OFI plugin team

Please review the guidelines for contributing and Pull Request Instructions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

… for test_efa

gmarciani · 2026-01-27T14:36:29Z

cli/src/pcluster/templates/queues_stack.py

+    is_b300 = instance_family == P6_B300
    efa_enabled = compute_resource.efa and compute_resource.efa.enabled
-    interface_type = "efa" if efa_enabled and not is_gb200 else None
+    interface_type = "efa" if efa_enabled and not is_gb200 and not is_b300 else None


[minor] Documentation: I suggest to make this logic more talkative by either:

add a comment here specifying that those instance families require the first interface to be ENA even if EFA is enabled.

define a constant INSTANCE_TYPES_WITH_FIRST_INTERFACE_ENA = [P6E_GB200,P6_B300] and reference the constant in this logic

gmarciani · 2026-01-27T14:42:51Z

tests/integration-tests/tests/efa/test_efa/test_efa/pcluster.config.yaml

      Networking:
        PlacementGroup:
-          Enabled: {% if instance not in ["p4d.24xlarge", "p6-b200.48xlarge"] %}true{% else %}false{% endif %}
+          Enabled: {% if instance not in ["p4d.24xlarge", "p6-b200.48xlarge", "p6-b300.48xlarge"] %}true{% else %}false{% endif %}


[minor] In the test code we have the logic to determine the capacity reservation and placement group. We could improve config readability by putting here the variable placement_group_enabled and set this variable in the test code accordingly.

I know that this approach was here before your PR, so not a blocker for this PR.

gmarciani · 2026-01-27T14:47:20Z

cli/src/pcluster/constants.py

 PCLUSTER_BUILD_IMAGE_CLEANUP_ROLE_BOOTSTRAP_TAG_KEY = "parallelcluster:build-image-cleanup-role-bootstrapped"

 P6E_GB200 = "p6e-gb200"
+P6_B300 = "p6-b300"


when we introduced GB200 we did for a limited set of OSs. Are we sure we do not have any OS limitation for B300? For instance, according to documentation for NVIDIA 580, AL2 is not mentioned as a supported OS: https://docs.nvidia.com/datacenter/tesla/tesla-release-notes-580-126-09/index.html

I addded a validator to prevent al2 and b300 from being used together.

himani2411 · 2026-01-27T14:49:03Z

tests/integration-tests/tests/efa/test_efa/test_efa/pcluster.config.yaml

      Networking:
        PlacementGroup:
-          Enabled: {% if instance not in ["p4d.24xlarge", "p6-b200.48xlarge"] %}true{% else %}false{% endif %}
+          Enabled: {% if instance not in ["p4d.24xlarge", "p6-b200.48xlarge", "p6-b300.48xlarge"] %}true{% else %}false{% endif %}


[Minor-NON Blocking] Lets change this to check for p family instance instead of the whole instance

gmarciani · 2026-01-27T14:49:43Z

cli/src/pcluster/templates/queues_stack.py

+    is_b300 = instance_family == P6_B300
    efa_enabled = compute_resource.efa and compute_resource.efa.enabled
-    interface_type = "efa" if efa_enabled and not is_gb200 else None
+    interface_type = "efa" if efa_enabled and not is_gb200 and not is_b300 else None


[Test] I suggest to cover the logic that determines the type of the interfaces with unit tests. To make it better I suggest to capture the logic into dedicated helper function and cover such function with unit test rather than the resulting template.

gmarciani · 2026-01-27T21:55:13Z

CHANGELOG.md

 - Add validator that warns against the downsides of disabling in-place updates on compute and login nodes through DevSettings.
 - Upgrade jmespath to ~=1.0 (from ~=0.10).
 - Upgrade tabulate to <=0.9.0 (from <=0.8.10).
+- Add support for p6-b300 instances by having a default network configuration for those instances.


We should surface in the changelog that the support is for all OS except for AL2

gmarciani · 2026-01-27T21:56:27Z

cli/src/pcluster/validators/ec2_validators.py

                    " Please use one of the following OS: {2}".format(instance_type, os, SUPPORTED_OSES_FOR_P6E_GB200),
                    FailureLevel.ERROR,
                )
+        if instance_type.startswith("p6-b300") and os in UNSUPPORTED_OSES_FOR_P6_B300:


[Test] We should capture this change with unit tests.

gmarciani · 2026-01-28T17:21:23Z

cli/src/pcluster/validators/ec2_validators.py

                    " Please use one of the following OS: {2}".format(instance_type, os, SUPPORTED_OSES_FOR_P6E_GB200),
                    FailureLevel.ERROR,
                )
+        if instance_type.startswith("p6-b300") and os in UNSUPPORTED_OSES_FOR_P6_B300:


[minor] We can refer to the constant P6_B300 rather than redefining the string.

gmarciani · 2026-01-28T17:23:43Z

cli/src/pcluster/validators/ec2_validators.py

                )
+        if instance_type.startswith("p6-b300") and os in UNSUPPORTED_OSES_FOR_P6_B300:
+            self._add_failure(
+                "The instance type {0} is not officially supported with OS {1}."


I know we use the term "officially" elsewhere. However, I think we should avoid it to prevent misunderstanding. Let;'s simply say that it is not supported.

gmarciani · 2026-01-28T17:25:14Z

cli/tests/pcluster/validators/test_ec2_validators.py

            None,
        ),
+        (
+            "p6-b300.48xlarge",


[minor] If you use p6-b300.WHATEVER_SIZE rather than p6-b300.48xlarge it will be clearer that the behavior applies whatever the size is. Otherwiose there could be a doubt that it is something specific to 48xlarge.

You applied this testing best practice in the unit test above. I suggest to apply it here as well.

codecov · 2026-01-28T18:40:30Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.27%. Comparing base (6bd8e1d) to head (dd86c39).
⚠️ Report is 27 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #7210      +/-   ##
===========================================
+ Coverage    90.21%   90.27%   +0.05%     
===========================================
  Files          183      183              
  Lines        16566    16645      +79     
===========================================
+ Hits         14945    15026      +81     
+ Misses        1621     1619       -2

Flag	Coverage Δ
unittests	`90.27% <100.00%> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

hgreebe added 3 commits January 23, 2026 13:11

Support b300 network interface conf

88d9363

Upgrade fabtest and NCCL version and support using b300 instance type…

6a876c5

… for test_efa

Update CHANGELOG

c68b7c4

hgreebe force-pushed the develop-test branch from f85e4a8 to c68b7c4 Compare January 27, 2026 14:05

hgreebe marked this pull request as ready for review January 27, 2026 14:15

hgreebe requested review from a team as code owners January 27, 2026 14:15

gmarciani reviewed Jan 27, 2026

View reviewed changes

himani2411 reviewed Jan 27, 2026

View reviewed changes

gmarciani reviewed Jan 27, 2026

View reviewed changes

Add unit test to cover b300 network interfaces

2e71357

hgreebe force-pushed the develop-test branch from 3bc83a8 to 2e71357 Compare January 27, 2026 16:36

Add validator that prevents using alinux2 with b300 instance type

ea4082c

hgreebe force-pushed the develop-test branch from 9ac67e5 to ea4082c Compare January 27, 2026 17:59

gmarciani reviewed Jan 27, 2026

View reviewed changes

gmarciani reviewed Jan 28, 2026

View reviewed changes

Add unit test for validator

dd86c39

hgreebe force-pushed the develop-test branch from 9f5f672 to dd86c39 Compare January 28, 2026 18:33

Fix CHANGELOG

d68047d

hgreebe force-pushed the develop-test branch from 0afce69 to d68047d Compare January 29, 2026 16:13

Merge branch 'develop' into develop-test

d209d57

gmarciani approved these changes Jan 29, 2026

View reviewed changes

hgreebe merged commit 12d911f into aws:develop Jan 29, 2026
24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support b300 Instance#7210

Support b300 Instance#7210
hgreebe merged 8 commits intoaws:developfrom
hgreebe:develop-test

hgreebe commented Jan 27, 2026

Uh oh!

gmarciani Jan 27, 2026

Uh oh!

gmarciani Jan 27, 2026

Uh oh!

gmarciani Jan 27, 2026 •

edited

Loading

Uh oh!

hgreebe Jan 27, 2026

Uh oh!

himani2411 Jan 27, 2026

Uh oh!

gmarciani Jan 27, 2026 •

edited

Loading

Uh oh!

gmarciani Jan 27, 2026

Uh oh!

gmarciani Jan 27, 2026

Uh oh!

gmarciani Jan 28, 2026

Uh oh!

gmarciani Jan 28, 2026

Uh oh!

gmarciani Jan 28, 2026

Uh oh!

codecov bot commented Jan 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

hgreebe commented Jan 27, 2026

Description of changes

Tests

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gmarciani Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gmarciani Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jan 28, 2026

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gmarciani Jan 27, 2026 •

edited

Loading

gmarciani Jan 27, 2026 •

edited

Loading