nutanix AHV support

[tchernomax]

add nutanix AHV support

ignition support nutanix AHV, the only thing to do is to add flatcar.oem.id=nutanix in the kernel cmdline.
However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.

I managed to create a nutanix AHV flatcar image (without nutanix guest tools... but that doesn't forbid the VM to run) by following this path:

wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_image.bin.bz2
bunzip2 flatcar_production_image.bin.bz2
PART=6
LOOP=$(sudo losetup --partscan --find --show flatcar_production_image.bin)
TARGET=$(sudo mktemp -d -p /mnt --suffix -flatcar)
sudo mount "${LOOP}p${PART}" "$TARGET"
echo 'set oem_id="nutanix"' | sudo tee "$TARGET"/grub.cfg
sudo umount "${TARGET}"
sudo rmdir "${TARGET}"
sudo losetup -d "${LOOP}"
qemu-img convert -f raw -O qcow2 flatcar_production_image.bin flatcar_production_image_nutanix.qcow2
rm flatcar_production_image.bin

(it tested it on a Nutanix AHV infra, also tried a flatcar update (via nebraska) -> it works)

And creating the VM with the ignition config provided as "cloud_init" "guest_customization":

...
        "guest_customization": {
          "cloud_init": {
            "user_data": "<base64 ignition conf>"
          }
          "is_overridable": false
        }
...

https://www.nutanix.dev/api_reference/apis/prism_v3.html#tag/vms/paths/~1vms/post

I tried to reproduce this path in this PR, but I can't try the resulting image myself.
It build on my desktop, but I can't upload it to a nutanix AHV platform due to my poor home network connection.
And I failed to build the image on the remote host (where I updated the flatcar_production_image.bin.bz2 official image).

How to use

If what I changed in the code is in fact the equivalent of the adding oem_id="nutanix" in the grub.cfg of the OEM partition.
This PR should be ok.

Otherwise, please guide me (I am a noob in flatcar dev).

Testing done

I created an image :

wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_image.bin.bz2
bunzip2 flatcar_production_image.bin.bz2
PART=6
LOOP=$(sudo losetup --partscan --find --show flatcar_production_image.bin)
TARGET=$(sudo mktemp -d -p /mnt --suffix -flatcar)
sudo mount "${LOOP}p${PART}" "$TARGET"
echo 'set oem_id="nutanix"' | sudo tee "$TARGET"/grub.cfg
sudo umount "${TARGET}"
sudo rmdir "${TARGET}"
sudo losetup -d "${LOOP}"
qemu-img convert -f raw -O qcow2 flatcar_production_image.bin flatcar_production_image_nutanix.qcow2
rm flatcar_production_image.bin

• upload it to a nutanix ahv cluster
• created multiple hosts using this image via terraform restapi provider (the nutanix official provider is currently unable to create VM with ignition configuration as "cloudinit" guest info).
• created a k8s cluster on thoses hosts via rke
• tried an update via nebraska (flatcar downloaded the new image, I reboot the hosts, it booted on the new version).

I tried this ignition config:

{"ignition":{"version":"3.3.0"},"storage":{"files":[{"path":"/toto","contents":{"compression":"","source":"data:,toto"},"mode":292}]}}

and checked that /toto was actually created at boot.
(I also tried some more advanced ignition config).

[tormath1]

Hi @tchernomax and thanks for your contribution, two questions:

• About the naming, could we use only nutanix averywhere to match the OEM id? Or _ahv is required because multiple versions of Nutanix images might exist?

• However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.

  Do you know if it's open-source? If yes, we can investigate to package this to a Gentoo format.

[github-actions]

Build action triggered: https://github.com/flatcar/scripts/actions/runs/18348277688

[tchernomax]

* About the naming, could we use only `nutanix` averywhere to match the OEM id? Or `_ahv` is required because multiple versions of Nutanix images might exist?

I don't really know.
I know that nutanix provide an interface which can be plugged on top off VMWare ESX hypervisors (but in that case we use the vmware flatcar image).
There is also nutanix AHV which use a nutanix branded hypervisor (what this MR is for).
While googling I also found nutanix cloud platform and nutanix kubernetes platform but I don't really what it is.

I think the best curse of action would be to ask nutanix themself.
Tomorrow I will ask a colleague (I don't have access myself) to create an issue on nutanix support to ask them to reply on this PR.

(if there is no need for _ahv I will update this PR)

* > However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.
  Do you know if it's open-source? If yes, we can investigate to package this to a Gentoo format.

I didn't find any source online.
And the content of LICENSE file on hosts where the software is already install suggest it's not.

[aurelienlm]

I’ve asked Nutanix Support to take a look at this PR and provide feedback, just in case it helps

[aurelienlm]

the answer :
If i understand correctly, you are trying to integrate nutanix guest tools in flatcar ?

1 - Nutanix guest tools isn't opensource.
2 - I will check internally whether we can raise an internal request to our engineering for this

I will provide an update tomorrow or Monday on this.

[tormath1]

the answer : If i understand correctly, you are trying to integrate nutanix guest tools in flatcar ?

1 - Nutanix guest tools isn't opensource. 2 - I will check internally whether we can raise an internal request to our engineering for this

I will provide an update tomorrow or Monday on this.

Great thanks for asking. That'd be nice to have those guest tools to get the full support. Do you know what they add? I guess it's some helpers for network, system information, etc. ?

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

[tchernomax]

Do you know what they add? I guess it's some helpers for network, system information, etc. ?

From what I understand (@nutanix please correct me if I am wrong) it's for sending system information from the VM to the host (like ip and such).

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

We plan on using flatcar without nutanix tools, from the nutanix support told us it's not ideal but the system/flatcar should be fine.
(though we only tests flatcar on nutanix (without tools) on a simple POC)

[tormath1]

Do you know what they add? I guess it's some helpers for network, system information, etc. ?

From what I understand (@nutanix please correct me if I am wrong) it's for sending system information from the VM to the host (like ip and such).

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

We plan on using flatcar without nutanix tools, from the nutanix support told us it's not ideal but the system/flatcar should be fine. (though we only tests flatcar on nutanix (without tools) on a simple POC)

Ok then, we can start with an image without guest tools and revisit this later.

To get back on the first point (nutanix_ahv vs nutanix) that'd be nice to know which one we can use, to have some consistency with the OEM ID. Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

[aurelienlm]

sorry the Nutanix support team can't help us on this. They have close the case.

I have reach out to our account team

[tchernomax]

Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

I changed this PR to remove "AHV" reference and only use nutanix.

[tormath1]

Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

I changed this PR to remove "AHV" reference and only use nutanix.

Thanks! I'll setup our CI system to build nutanix images then. Once the image will be built, will you be available for running test for it?

[tchernomax]

Once the image will be built, will you be available for running test for it?

yes (I might not be very responsive but I will make time to tests the images you provide)