removed datastore dependency from saturation detector

[nirrozenbaum]

This PR aims to improve the request handling flow.
Before this PR the flow is:

• director gets an incoming request. could be with subset filtering (only subset of the endpoints are candidates to serve the request)
• saturation detector runs for all pods always, even if subset filtering was specified. as part of this code, saturation detection gets all pods from datastore.
• director goes again to datastore to get candidate pods for scheduling while considering the subset filter header.
• schedule
• ....
  in this flow pods are read from the datastore twice in a row, and saturation check is done for all the pods even if subset filter was specified.

After this PR:

• director gets an incoming request. could be with subset filtering (only subset of the endpoints are candidates to serve the request).
• director goes to datastore to get candidate pods for scheduling while considering the subset filter header.
• saturation detector runs only for the candidate pods for scheduling. it gets as input the candidate pods. no need to have datastore as a dependency in saturation detector.
• if candidate pods are not saturated, continue to schedule
• ...
  in this flow pods are read from the datastore once for the request flow and saturation check is done only for the relevant pods for the request.

tests updated accordingly.

Additionally, this PR removes the extensive documentation in some places. IMO we don't want to document every function internal step. this is not maintainable. the code should be self explainable (which it is!) in a way that doesn't require this extensive documentation. for example, in HandleRequestion function (inside director), there is no need to explain in the godoc what are the steps. one can just read the function and easily understand the steps.

cc: @kfswain @LukeAVanDrie

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nirrozenbaum

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [nirrozenbaum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	728fe9a
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/6895069353b5cb000832a7c0
😎 Deploy Preview	https://deploy-preview-1293--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[nayihz]

Maybe we could move admiRequest logic into getCandidatePodsForScheduling. Intuitively, the admiRequest function should be called before the getCandidatePodsForScheduling function.

[nirrozenbaum]

the main logic currently in admitReqeust is to check saturation of the system.
it's not relevant to check saturation of all the pods but only of the candidate pods (If we use subset filtering to specify list of pods and all of them are saturated, it's not helpful that other pods are available).

therefore, getCandidatePods is done before the admitRequest function.

think about it this way -
getCandidatePods func returns the set of pods that are relevant for current request.
admitRequest func check if any of the pods from the above set can actually serve the request, or if all of them are saturated.

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.