NoCloud seedfrom http/s will not be processed before other datasources

[kuwv]

Bug report

NoCloud using seedfrom is skipped even when prioritized first.

Steps to reproduce the problem

VMware (or other providers) work with the following user-data:

#cloud-config
datasource:
  NoCloud: {}

^^^ Will correctly load cloud-config from VMware customization specification dashboard.

datasource_list: [NoCloud, VMware, None]
datasource:
  NoCloud:
    seedfrom: https://example.com/cloud-init/

^^^ Will skip NoCloud ignoring valid cloud-config and kick VMware without any cloud-config.

Environment details

• Cloud-init version:
• Operating System Distribution:
• Cloud provider, platform or installer type:

cloud-init logs

2025-12-04 21:58:55,802 - sources[DEBUG]: Searching for local data source in: ['DataSourceNoCloud', 'DataSourceVMware']
2025-12-04 21:58:55,802 - handlers.py[DEBUG]: start: init-local/search-NoCloud: searching for local data from DataSourceNoCloud
2025-12-04 21:58:55,802 - sources[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'>
2025-12-04 21:58:55,802 - sources[DEBUG]: Update datasource metadata and network config due to events: boot-new-instance
2025-12-04 21:58:55,803 - sources[DEBUG]: Detected DataSourceNoCloud

2025-12-04 21:58:55,916 - DataSourceNoCloud.py[INFO]: DataSourceNoCloud  only uses seeds starting with ('/', 'file://') - will try to use https://example.com/cloud-init/ in the network stage.
2025-12-04 21:58:55,916 - performance.py[DEBUG]: Getting metadata took 0.114 seconds
2025-12-04 21:58:55,916 - sources[DEBUG]: Datasource DataSourceNoCloud  not updated for events: boot-new-instance
2025-12-04 21:58:55,916 - handlers.py[DEBUG]: finish: init-local/search-NoCloud: SUCCESS: no local data found from DataSourceNoCloud
2025-12-04 21:58:55,916 - handlers.py[DEBUG]: start: init-local/search-VMware: searching for local data from DataSourceVMware
2025-12-04 21:58:55,916 - sources[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceVMware.DataSourceVMware'>
2025-12-04 21:58:55,917 - sources[DEBUG]: Update datasource metadata and network config due to events: boot-new-instance
2025-12-04 21:58:55,917 - sources[DEBUG]: Detected DataSourceVMware [seed=None]
2025-12-04 21:58:55,917 - dmi.py[DEBUG]: querying dmi data /sys/class/dmi/id/product_name
2025-12-04 21:58:55,918 - DataSourceVMware.py[DEBUG]: discovered vmware-rpctool: /usr/bin/vmware-rpctool
2025-12-04 21:58:55,918 - DataSourceVMware.py[DEBUG]: discovered vmtoolsd: /usr/bin/vmtoolsd
2025-12-04 21:58:55,918 - DataSourceVMware.py[INFO]: query guestinfo with /usr/bin/vmware-rpctool
2025-12-04 21:58:55,918 - DataSourceVMware.py[DEBUG]: Getting guestinfo value for key metadata
2025-12-04 21:58:55,918 - subp.py[DEBUG]: Running command ['/usr/bin/vmware-rpctool', 'info-get guestinfo.metadata'] with allowed return codes [0] (shell=False, capture=True)
2025-12-04 21:58:55,921 - DataSourceVMware.py[DEBUG]: No value found for key metadata
2025-12-04 21:58:55,922 - DataSourceVMware.py[DEBUG]: Getting guestinfo value for key userdata
2025-12-04 21:58:55,922 - subp.py[DEBUG]: Running command ['/usr/bin/vmware-rpctool', 'info-get guestinfo.userdata'] with allowed return codes [0] (shell=False, capture=True)
2025-12-04 21:58:55,925 - DataSourceVMware.py[DEBUG]: No value found for key userdata
2025-12-04 21:58:55,926 - DataSourceVMware.py[DEBUG]: Getting guestinfo value for key vendordata
2025-12-04 21:58:55,926 - subp.py[DEBUG]: Running command ['/usr/bin/vmware-rpctool', 'info-get guestinfo.vendordata'] with allowed return codes [0] (shell=False, capture=True)
2025-12-04 21:58:55,929 - DataSourceVMware.py[DEBUG]: No value found for key vendordata

[holmanb]

NoCloud using seedfrom is skipped even when prioritized first.

Runtime detection logic is used when multiple items are in the list. Force a single datasource by limiting the list to a single item.

[kuwv]

NoCloud using seedfrom is skipped even when prioritized first.

Runtime detection logic is used when multiple items are in the list. Force a single datasource by limiting the list to a single item.

Howdy @holmanb, I currently have two separate images that I'm attempting to combine into one.

[holmanb]

NoCloud using seedfrom is skipped even when prioritized first.

Runtime detection logic is used when multiple items are in the list. Force a single datasource by limiting the list to a single item.

Howdy @holmanb, I currently have two separate images that I'm attempting to combine into one.

I don't follow.

[kuwv]

NoCloud using seedfrom is skipped even when prioritized first.

Runtime detection logic is used when multiple items are in the list. Force a single datasource by limiting the list to a single item.

Howdy @holmanb, I currently have two separate images that I'm attempting to combine into one.

I don't follow.

In my understanding datasource can be configured from user-data but datasource_list cannot. The only way I can use a single datasource so far is through two separate images.

I want to standardize around one image for multiple teams.

[holmanb]

In my understanding datasource can be configured from user-data

Can it? What do you expect to happen?

[kuwv]

In my understanding datasource can be configured from user-data

Can it? What do you expect to happen?

Apologies, but I can't tell how much experience you have with cloud-init: https://cloudinit.readthedocs.io/en/latest/reference/examples.html#configure-data-sources

I have already tested this, reviewed the cloud-init source, and validated datasource_list / datasource availablility.

Previously, cloud-init supported NoCloud-Net to pull from a http/s sources instead of disk. This was configurable via user-data. Now it is not. IMO this is a regression unless there is a consistent approach to the previous logic used.

NOTE: VMware doesn't use metadata_urls as it uses vmtoolsd instead. Whereas the below should work when using two valid datasources:

#cloud-config
datasource:
  Ec2: {}
  NoCloud:
    seedfrom: https://example.com/cloud-init

This is inconsistent behavior within cloud-init.

[holmanb]

This was configurable via user-data. Now it is not.

What is the most recent version that this worked with? And which distro are you using?

[kuwv]

This was configurable via user-data. Now it is not.

What is the most recent version that this worked with? And which distro are you using?

I'm on RHEL9 and Ubuntu 22.

This was available in 21.1: https://cloudinit.readthedocs.io/en/21.1/topics/datasources/nocloud.html

cloud-init/cloudinit/importer.py

Line 38 in d7bc162

# nocloud-net is the only datasource that requires matching on a name that

But, I haven't tested it. Very well could wait until the network step to run just as seen above.

Regardless of how NoCloud is functioning, the way the VMware works in regards to other modules are two different behaviors. You can zero out some of the other modules but not VMware.

If I understand correctly (not sure about all the modules behavior though)

datasource:
  Ec2: {}  # <- will zero out
  VMware: {}  # <- will still pull from `vmtoolsd` anyway

[holmanb]

Your goal is to have the IMDS provide user-data which tells cloud-init a URL where it can find user-data at a different location, right?

#include-file accomplishes this goal without messing with datasource detection - have you tried that?

[kuwv]

@holmanb I looked into this and while this is convenient for consolidating a few pieces this won't work unfortunately. This is mainly because we have two approaches for provisioning but both dynamically generate their content.

[holmanb]

@holmanb I looked into this and while this is convenient for consolidating a few pieces this won't work unfortunately. This is mainly because we have two approaches for provisioning but both dynamically generate their content.

How does dynamic generation prevent using #include?