Merge remote-tracking branch 'upstream/main'

Author: Mohamed Zeidan

.github/pull_request_template.md

@@ -14,22 +14,17 @@
 <!-- Describe your testing approach -->
 
 
-## Unit test coverage
-<!-- Check unit test coverage for your changes -->
-- [ ] All new/modified code has unit tests
-- [ ] Coverage verified for changed code
-- [ ] N/A - no testable code changes
-
-## Do we need integration tests?
-<!-- Consider if integration tests are needed -->
-- [ ] Yes - integration tests added
-- [ ] No - unit tests sufficient
-- [ ] No - infrastructure/config change only
-- [ ] Unsure - please advise
-
----
-
-## Checklist
-- [ ] PR title clearly describes the change
-- [ ] No sensitive information exposed and security is maintained
-- [ ] Ready for review
+## Are unit tests added?
+
+
+## Are integration tests added?
+
+
+## Reviewer Guidelines
+
+‼️ **Merge Requirements**: PRs with failing integration tests cannot be merged without justification. 
+
+One of the following must be true:
+- [ ] All automated PR checks pass
+- [ ] Failed tests include local run results/screenshots proving they work
+- [ ] Changes are documentation-only

.gitignore

@@ -22,7 +22,11 @@ doc/_build/
 /sagemaker-hyperpod/build
 /sagemaker-hyperpod/.coverage
 /sagemaker-hyperpod/.coverage.*
+
 /hyperpod-cluster-stack-template/build
+/hyperpod-pytorch-job-template/build
+/hyperpod-custom-inference-template/build
+/hyperpod-jumpstart-inference-template/build
 
 # Ignore all contents of result and results directories
 /result/

README.md

@@ -106,7 +106,7 @@ hyp list-cluster
 |--------|------|-------------|
 | `--region <region>` | Optional | The region that the SageMaker HyperPod and EKS clusters are located. If not specified, it will be set to the region from the current AWS account credentials. |
 | `--namespace <namespace>` | Optional | The namespace that users want to check the quota with. Only the SageMaker managed namespaces are supported. |
-| `--output <json|table>` | Optional | The output format. Available values are `table` and `json`. The default value is `json`. |
+| `--output <json\|table>` | Optional | The output format. Available values are `table` and `json`. The default value is `json`. |
 | `--debug` | Optional | Enable debug mode for detailed logging. |
 
 ### Connecting to a Cluster

helm_chart/HyperPodHelmChart/charts/health-monitoring-agent/templates/_helpers.tpl

@@ -55,7 +55,7 @@ Generate the health monitoring agent image URI based on AWS region
 */}}
 {{- define "health-monitoring-agent.imageUri" -}}
 {{- $region := "" -}}
-{{- $imageTag := .Values.imageTag | default "1.0.790.0_1.0.266.0" -}}
+{{- $imageTag := .Values.imageTag | default "1.0.819.0_1.0.267.0" -}}
 
 {{/* Debug: Show image tag selection if debug is enabled */}}
 {{- if .Values.debug -}}

helm_chart/HyperPodHelmChart/charts/health-monitoring-agent/templates/health-monitoring-agent.yaml

@@ -85,12 +85,6 @@ spec:
                       - ml.g5.16xlarge
                       - ml.g5.24xlarge
                       - ml.g5.48xlarge
-                      - ml.inf2.xlarge
-                      - ml.inf2.8xlarge
-                      - ml.inf2.24xlarge
-                      - ml.inf2.48xlarge
-                      - ml.trn1.32xlarge
-                      - ml.trn1n.32xlarge
                       - ml.g6.xlarge
                       - ml.g6.2xlarge
                       - ml.g6.4xlarge
@@ -109,7 +103,6 @@ spec:
                       - ml.g6e.12xlarge
                       - ml.g6e.24xlarge
                       - ml.g6e.48xlarge
-                      - ml.trn2.48xlarge
                       - ml.p6-b200.48xlarge
                       - ml.p6e-gb200.36xlarge
       containers:
@@ -166,3 +159,93 @@ spec:
           operator: Exists
         - effect: NoExecute
           operator: Exists
+---
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+  name: health-monitoring-agent-non-nvidia
+  namespace: {{ .Values.namespace }}
+  labels:
+    app: health-monitoring-agent-non-nvidia
+spec:
+  selector:
+    matchLabels:
+      app: health-monitoring-agent-non-nvidia
+  template:
+    metadata:
+      labels:
+        app: health-monitoring-agent-non-nvidia
+    spec:
+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+              - matchExpressions:
+                  - key: node.kubernetes.io/instance-type
+                    operator: In
+                    values:
+                      - ml.inf2.xlarge
+                      - ml.inf2.8xlarge
+                      - ml.inf2.24xlarge
+                      - ml.inf2.48xlarge
+                      - ml.trn1.32xlarge
+                      - ml.trn1n.32xlarge
+                      - ml.trn2.48xlarge
+      containers:
+        - name: health-monitoring-agent-non-nvidia
+          args:
+            - --enable-k8s-exporter=false
+            - --config.system-log-monitor=/config/system-message-monitor.json
+          image: {{ include "health-monitoring-agent.imageUri" . }}
+          resources:
+            limits:
+              cpu: 500m
+              memory: 512Mi
+            requests:
+              cpu: 500m
+              memory: 512Mi
+          imagePullPolicy: IfNotPresent
+          securityContext:
+            runAsUser: 1000
+            runAsGroup: 2000
+          env:
+            - name: NODE_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: spec.nodeName
+            - name: NODE_IP
+              valueFrom:
+                fieldRef:
+                  fieldPath: status.hostIP
+            - name: NVIDIA_VISIBLE_DEVICES
+              value: "void"
+            - name: NVIDIA_DRIVER_CAPABILITIES
+              value: ""
+          volumeMounts:
+            - name: log
+              mountPath: /var/log
+            - name: kmsg
+              mountPath: /dev/kmsg
+              readOnly: true
+            # Make sure node problem detector is in the same timezone
+            # with the host.
+            - name: localtime
+              mountPath: /etc/localtime
+              readOnly: true
+      serviceAccountName: health-monitoring-agent
+      volumes:
+        - name: log
+          # Config `log` to your system log directory
+          hostPath:
+            path: /var/log/
+        - name: kmsg
+          hostPath:
+            path: /dev/kmsg
+        - name: localtime
+          hostPath:
+            path: /etc/localtime
+      tolerations:
+        - effect: NoSchedule
+          operator: Exists
+        - effect: NoExecute
+          operator: Exists

helm_chart/HyperPodHelmChart/charts/health-monitoring-agent/values.yaml

@@ -25,7 +25,7 @@ imageTag: ""
 
 # Override the health monitoring agent image URI
 # If specified, this will override the automatic region-based URI selection
-# Example: "905418368575.dkr.ecr.us-west-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0"
+# Example: "905418368575.dkr.ecr.us-west-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0"
 hmaimage: ""
 
 # Enable debug output for region selection process

helm_chart/readme.md

@@ -234,19 +234,19 @@ helm upgrade dependencies helm_chart/HyperPodHelmChart --namespace kube-system
 
 - **Supported Regions and their ECR URIs**:
   ```
-  us-east-1 (US East (N. Virginia)):      767398015722.dkr.ecr.us-east-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  us-west-2 (US West (Oregon)):           905418368575.dkr.ecr.us-west-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  us-east-2 (US East (Ohio)):             851725546812.dkr.ecr.us-east-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  us-west-1 (US West (N. California)):    011528288828.dkr.ecr.us-west-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  eu-central-1 (Europe (Frankfurt)):      211125453373.dkr.ecr.eu-central-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  eu-north-1 (Europe (Stockholm)):        654654141839.dkr.ecr.eu-north-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  eu-west-1 (Europe (Ireland)):           533267293120.dkr.ecr.eu-west-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  eu-west-2 (Europe (London)):            011528288831.dkr.ecr.eu-west-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  ap-northeast-1 (Asia Pacific (Tokyo)):  533267052152.dkr.ecr.ap-northeast-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  ap-south-1 (Asia Pacific (Mumbai)):     011528288864.dkr.ecr.ap-south-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  ap-southeast-1 (Asia Pacific (Singapore)): 905418428165.dkr.ecr.ap-southeast-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  ap-southeast-2 (Asia Pacific (Sydney)):    851725636348.dkr.ecr.ap-southeast-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
-  sa-east-1 (South America (São Paulo)):     025066253954.dkr.ecr.sa-east-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.790.0_1.0.266.0
+  us-east-1 (US East (N. Virginia)):      767398015722.dkr.ecr.us-east-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  us-west-2 (US West (Oregon)):           905418368575.dkr.ecr.us-west-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  us-east-2 (US East (Ohio)):             851725546812.dkr.ecr.us-east-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  us-west-1 (US West (N. California)):    011528288828.dkr.ecr.us-west-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  eu-central-1 (Europe (Frankfurt)):      211125453373.dkr.ecr.eu-central-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  eu-north-1 (Europe (Stockholm)):        654654141839.dkr.ecr.eu-north-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  eu-west-1 (Europe (Ireland)):           533267293120.dkr.ecr.eu-west-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  eu-west-2 (Europe (London)):            011528288831.dkr.ecr.eu-west-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  ap-northeast-1 (Asia Pacific (Tokyo)):  533267052152.dkr.ecr.ap-northeast-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  ap-south-1 (Asia Pacific (Mumbai)):     011528288864.dkr.ecr.ap-south-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  ap-southeast-1 (Asia Pacific (Singapore)): 905418428165.dkr.ecr.ap-southeast-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  ap-southeast-2 (Asia Pacific (Sydney)):    851725636348.dkr.ecr.ap-southeast-2.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
+  sa-east-1 (South America (São Paulo)):     025066253954.dkr.ecr.sa-east-1.amazonaws.com/hyperpod-health-monitoring-agent:1.0.819.0_1.0.267.0
   ```
 
 ## 7. Troubleshooting

setup.cfg

@@ -50,7 +50,7 @@ xfail_strict = true
 addopts =
     --verbose
     --ignore=build/private
-    --cov hyperpod_cli
+    --cov sagemaker.hyperpod
     --cov-config setup.cfg
     --cov-report term-missing
     --cov-report html:build/hyperpod-documentation/coverage
@@ -59,8 +59,8 @@ addopts =
     --durations=5
     # Default to colorful output
     --color=yes
-    # Uncomment to enforce a minimum code coverage threshold.
-    # --cov-fail-under 50
+    # Enforce a minimum code coverage threshold
+    --cov-fail-under 50
 testpaths = test
 looponfailroots = src test