✨Add flags to allow customization of CPU shares and reservations

[tommasopozzetti]

What this PR does / why we need it:

This PR adds flags to optionally customize CPU shares and reservations for cloned VMs as part of the vSphereMachine spec

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign enxebre for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Welcome @tommasopozzetti!

It looks like this is your first PR to kubernetes-sigs/cluster-api-provider-vsphere 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api-provider-vsphere has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @tommasopozzetti. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[smcallister-bc]

This seems incredibly useful, thank you for doing this! Any chance you could do something similar for memory reservations as well?

[tommasopozzetti]

I’d be happy to add to the PR similar logic for memory reservation and shares as well as potentially memory pinning

I was hoping to first get a glance from a maintainer to see if this approach is reasonable given it’s my first contribution to this project!

[chrischdi]

Does it make sense to have something like:

resources:
  reservation:
    cpu: ...
    memory: ...
  shares:
    ...

Or maybe this should be modelled in the k8s wordings which has limits and requests? (might not match the things this PR currently sets).

This is e.g. done by the vm-operator APIs. However, vm-operator does not use shares.

Could someone research what the benefits are of setting shares? And should we also consider CPU Limit to be set?

[tommasopozzetti]

@chrischdi thanks for the review!

In terms of shares vs reservations, shares are a relative measure of prioritization while reservations are an absolute one. A VM with 2Ghz set for reservation will be guaranteed that even under host contention, it will always have 2Ghz of cpu power available to it. The vSphere admission controller will prevent a VM to power on if the total sum of reservations for VMs on a given host exceeds the total available cpu power of that host (no overprovisioning).
Shares on the other hand are just meaningful relative to shares of other VMs on that host. The host can be over provisioned and if it comes under cpu contention, VMs will be prioritized for cpu time relative to each other depending on their shares. So if a host has 3 VMs, one with 6000 shares, one with 3000 and one with 1000, if the host comes under cpu contention, the first will get 60% of cpu time, the second 30% and the third 10%. Each VM normally gets shares assigned by default proportional to its number of vCPUs but it is very useful to be able to tune that at will.

In terms of using cpu/memory limits, I have never had to implement these but they essentially would artificially cause the same effects as if the underlying host was under resource contention even if it is not, when the VM reaches said limit. More detailed info here. I'd be happy to add to this PR the limit as well as an optional configurable if desired.

Finally, in terms of the syntax, I'm open to suggestions! I personally feel like using the same syntax as standard Kubernetes containers might be misleading since the practical implementation of using reservations, shares and limits on VMs is very different than memory and cpu requests and limits for k8s pods.
I was going for a more flat mapping similar to the other properties that matches with the VM options and would look like

cpuReservationMhz: xxx
cpuShares: xxx
cpuLimitMhz: xxx
memoryReservationMB: xxx
memoryShares: xxx
memoryLimitMB: xxx
reserveAllMemory: false

but, if preferred, we could also go for something nested like

resourceManagement:
  cpu:
    reservationMhz:
    shares:
    limitMhz:
  memory:
    reservationMB:
    shares:
    limitMB:
    reserveAll:

or similar

[tommasopozzetti]

Hi @chrischdi, following up on your comment, do you have any thoughts on my reply?
Thanks!