You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -163,7 +164,7 @@ Items marked with (R) are required *prior to targeting to a milestone / release*
163
164
To achieve efficient management of fabric devices, we propose adding the following features to the Kubernetes scheduler's DRA plugin.
164
165
Fabric devices are those that are not directly connected to the server and require attachment to the server for use.
165
166
166
-
In the DRA current implementation, fabric devices are attached after the scheduling decision, which leads to the following issue:
167
+
In the current DRA implementation, fabric devices are attached after the scheduling decision, which leads to the following issue:
167
168
168
169
Fabric devices may be contested by other clusters.
169
170
In scenarios where attachment occurs after scheduling, there is a risk that the resource cannot be attached at the time of attachment, causing the container to remain in the "Container Creating" state.
@@ -180,7 +181,7 @@ Recently, a new server architecture called Composable Disaggregated Infrastructu
180
181
181
182
In a traditional server, hardware resources such as CPUs, memory, and GPUs reside within the server.
182
183
Composable Disaggregated Infrastructure decomposes these hardware resources and makes them available as resource pools.
183
-
We can combine these resource by software definition so that we can create custom-made servers.
184
+
We can combine these resources by software definition so that we can create custom-made servers.
184
185
185
186
Composable system is composed of resource pool and Composable Manager software.
186
187
In Resource pool all components are connected to PCIe or CXL switches.
@@ -224,25 +225,26 @@ and make progress.
224
225
225
226
The basic idea is the following:
226
227
227
-
1.**Add a Ready Flag to ResourceClaim**:
228
-
- Add a flag to `ResourceClaim` that indicates the readiness state of the device.
229
-
The `PreBind` phase will be held until this flag is set to "Ready".
228
+
1.**Adding Attributes to ResourceSlice**:
229
+
- Add an attribute to `ResourceSlice` to indicate fabric devices. This key is predefined as part of the attributes.
230
230
231
-
2.**Wait for Device Attachment Completion in the PreBind() Process**:
232
-
The overall flow of the PreBind() process is as follows:
231
+
2.**Waiting for Device Attachment in PreBind**:
232
+
- For fabric devices, the scheduler waits for the device attachment to complete during the `PreBind` phase.
233
233
234
-
-**Update ResourceClaim**:
235
-
- The scheduler updates the `ResourceClaim` to notify the vendor's driver that the device needs to be prepared.
236
-
This process is the same as the existing `PreBind`.
237
-
- After updating the `ResourceClaim`, if the flag is set to "Preparing", the completion of the `PreBind` phase will be held until the flag is set to "Ready".
234
+
3.**PreBind Process**:
235
+
The overall flow of the `PreBind` process is as follows:
236
+
237
+
-**Updating ResourceClaim**:
238
+
- The scheduler DRA plugin updates the `ResourceClaim` to notify the Composable DRA Controllers that device attachment is needed.
239
+
This is the same as the existing `PreBind` process.
240
+
- In addition to the existing operations, the update to the `ResourceClaim` includes setting the necessary values in the `AllocatedDeviceStatus` conditions.
238
241
239
242
-**Monitoring and Preparation by Composable DRA Controllers**:
240
-
- Composable DRA Controllers monitor the `ResourceClaim`.
241
-
If a device that requires preparation is associated with the `ResourceClaim`, they perform the necessary preparations.
242
-
Once the preparation is complete, they set the flag to "Ready".
243
+
- Composable DRA Controllers monitor the `ResourceClaim`. If a device that requires preparation is associated with the `ResourceClaim`, they perform the necessary preparations.
244
+
- Once the preparation is complete, they set the conditions to `true`.
243
245
244
246
-**Completion of the PreBind Phase**:
245
-
- Once the flag is set to "Ready", the `PreBind` phase is completed, and the scheduler proceeds to the next step.
247
+
- Once all conditions are met, the `PreBind` phase is completed, and the scheduler proceeds to the next step.
246
248
247
249
### User Stories (Optional)
248
250
@@ -288,8 +290,7 @@ This document outlines the design of the DRA Scheduler Plugin, focusing on the h
288
290
Key additions include new attributes for device identification, enhancements to `AllocatedDeviceStatus`, and the process for handling `ResourceSlices` upon attachment failure.
289
291
The composable controller design is also discussed, emphasizing efficient utilization of fabric devices.
290
292
291
-
//TODO update figure
292
-

293
+

293
294
294
295
#### Device Attribute Additions
295
296
@@ -348,13 +349,15 @@ type AllocatedDeviceStatus struct {
When `kubernetes.io/needs-attaching: true` is set, the scheduler DRA plugin is expected to do the following:
359
362
360
363
1. Set `AllocatedDeviceStatus.NodeName`.
@@ -368,7 +371,7 @@ This issue will be addressed separately as outlined in kubernetes/kubernetes#129
368
371
#### PreBind Phase Timeout
369
372
370
373
If the device attachment is successful, we expect it to take no longer than 5 minutes.
371
-
Therefore, if we set a fixed timeout for the scheduler, we would like to set it to 10 minutes.
374
+
However, to account for potential update lags, we would like to set a fixed timeout for the scheduler to 10 minutes.
372
375
373
376
Even if the conditions `Type: kubernetes.io/is-attached` or `Type: kubernetes.io/attach-failed` are not updated, setting a timeout will prevent the scheduler from waiting indefinitely in the PreBind phase.
0 commit comments