You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: keps/sig-node/4205-psi-metric/README.md
+31-28Lines changed: 31 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -121,7 +121,7 @@ PSI metric will be available for users in the Kubernetes metrics API.
121
121
122
122
#### Story 2
123
123
124
-
Kubernetes users want to prevent new pods to be scheduled on the nodes that have resource starvation. By using PSI metric, the kubelet will set Node Condition to avoid pods being scheduled on nodes under high resource pressure. The node controller could then set a (taint on the node based on these new Node Conditions)[https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/#taint-nodes-by-condition].
124
+
Kubernetes users want to prevent new pods to be scheduled on the nodes that have resource starvation. By using PSI metric, the kubelet will set Node Condition to avoid pods being scheduled on nodes under high resource pressure. The node controller could then set a [taint on the node based on these new Node Conditions](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/#taint-nodes-by-condition).
125
125
126
126
### Risks and Mitigations
127
127
@@ -137,20 +137,23 @@ default threshold to be used for reporting the nodes under heavy resource pressu
137
137
138
138
#### Phase 1
139
139
1. Add new Data structures PSIData and PSIStats corresponding to the PSI metric output format as following:
140
+
141
+
```
140
142
some avg10=0.00 avg60=0.00 avg300=0.00 total=0
141
143
full avg10=0.00 avg60=0.00 avg300=0.00 total=0
144
+
```
142
145
143
146
```go
144
147
typePSIDatastruct {
145
-
Avg10 *float64`json:”avg10”`
146
-
Avg60 *float64`json:”avg60”`
147
-
Avg300 *float64`json:”avg300”`
148
-
Total *float64`json:”total”`
148
+
Avg10*float64`json:"avg10"`
149
+
Avg60*float64`json:"avg60"`
150
+
Avg300 *float64`json:"avg300"`
151
+
Total*float64`json:"total"`
149
152
}
150
153
151
154
typePSIStatsstruct {
152
-
Some *PSIData `json:”some,omitempty”`
153
-
Full *PSIData `json:”full,omitempty”`
155
+
Some *PSIData `json:"some,omitempty"`
156
+
Full *PSIData `json:"full,omitempty"`
154
157
}
155
158
```
156
159
@@ -161,16 +164,16 @@ metric data will be available through CRI instead.
161
164
##### CPU
162
165
```go
163
166
typeCPUStatsstruct {
164
-
// PSI stats of the overall node
165
-
PSI cadvisorapi.PSIStats`json:”psi,omitempty”`
167
+
// PSI stats of the overall node
168
+
PSI cadvisorapi.PSIStats`json:"psi,omitempty"`
166
169
}
167
170
```
168
171
169
172
##### Memory
170
173
```go
171
174
typeMemoryStatsstruct {
172
175
// PSI stats of the overall node
173
-
PSI cadvisorapi.PSIStats`json:”psi,omitempty”`
176
+
PSI cadvisorapi.PSIStats`json:"psi,omitempty"`
174
177
}
175
178
```
176
179
@@ -179,23 +182,22 @@ type MemoryStats struct {
179
182
// IOStats contains data about IO usage.
180
183
typeIOStatsstruct {
181
184
// The time at which these stats were updated.
182
-
Time metav1.Time`json:”time”`
185
+
Time metav1.Time`json:"time"`
183
186
184
-
// PSI stats of the overall node
185
-
PSI cadvisorapi.PSIStats`json:”psi,omitempty”`
187
+
// PSI stats of the overall node
188
+
PSI cadvisorapi.PSIStats`json:"psi,omitempty"`
186
189
}
187
190
188
191
typeNodeStatsstruct {
189
192
// Stats about the IO pressure of the node
190
-
IO *IOStats `json:”io,omitempty”`
191
-
193
+
IO *IOStats `json:"io,omitempty"`
192
194
}
193
195
```
194
196
195
197
#### Phase 2 to add PSI based actions.
196
198
**Note:** These actions are tentative, and will depend on different the outcome from testing and discussions with sig-node members, users, and other folks.
197
199
198
-
1. Introduce a new kubelet config parameter, pressure threshold to let users specify the pressure percentage beyond which the kubelet would report the node condition to disallow workloads to be scheduled on it.
200
+
1. Introduce a new kubelet config parameter, pressure threshold, to let users specify the pressure percentage beyond which the kubelet would report the node condition to disallow workloads to be scheduled on it.
199
201
200
202
2. Add new node conditions corresponding to high PSI (beyond threshold levels) on CPU, Memory and IO.
201
203
@@ -205,14 +207,14 @@ type NodeStats struct {
205
207
const (
206
208
…
207
209
// Conditions based on pressure at system level cgroup.
@@ -226,13 +228,14 @@ In theory, 10s interval might be rapid to taint a node with NoSchedule effect. T
226
228
* If avg60 < threshold for a node tainted with NoSchedule effect, remove the NodeCondition.
227
229
228
230
4. Collaborate with sig-scheduling to modify TaintNodesByCondition feature to integrate new taints for the new Node Conditions introduced in this enhancement.
5. Perform experiments to finalize the default optimal pressure threshold value.
234
237
235
-
6. Add a new feature gate PSINodeCondition, and guard the node condition related logic behind the feature gate. Set --feature-gates=PSINodeCondition=true to enable the feature.
238
+
6. Add a new feature gate PSINodeCondition, and guard the node condition related logic behind the feature gate. Set `--feature-gates=PSINodeCondition=true` to enable the feature.
236
239
237
240
### Test Plan
238
241
@@ -511,7 +514,7 @@ checking if there are objects with field X set) may be a last resort. Avoid
511
514
logs or events for this purpose.
512
515
-->
513
516
For Phase 1:
514
-
Use `kubectl get --raw "/api/v1/nodes/{$nodeName}/proxy/stats/summary"`` to call Summary API. If the PSIStats field is seen in the API response,
517
+
Use `kubectl get --raw "/api/v1/nodes/{$nodeName}/proxy/stats/summary"` to call Summary API. If the PSIStats field is seen in the API response,
0 commit comments