Skip to content

Commit f7e9a10

Browse files
author
Sherif Akoush
committed
add note about logic
1 parent e591db9 commit f7e9a10

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

scheduler/pkg/scheduler/scheduler.go

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,12 @@ func (s *SimpleScheduler) scheduleToServer(modelName string) error {
169169
WithField("desired_replicas", desiredReplicas).
170170
Debug("Identified candidate servers for model")
171171

172+
// The main logic of trying to find a server for the model is as follows:
173+
// 1. If there are enough replicas on a server, schedule the model
174+
// 2. If there are not enough replicas on a server, try to schedule with min replicas. In this case we actually should get
175+
// the models loaded on all the replicas of the servers (assuming min replicas is less than the number of replicas on the server)
176+
// we also mark the model in this case as failed to schedule so that if the infra changes in the future we can try to reschedule
177+
172178
// For each server filter and sort replicas and attempt schedule if enough replicas
173179
ok := s.findAndUpdateToServers(filteredServers, latestModel, desiredReplicas, desiredReplicas)
174180
// Try to scheduler with min replicas if not enough replicas

0 commit comments

Comments
 (0)