wrong sources.list is used if additional sources are added in user-data

[kwitsch]

If additional sources are added in the user-data file cloud-init will overwrite the sources.list file(which is the expected behavior). Unfortunately the new file is for debian an not raspbian.
This results in some errors regarding the sources in cloud-init.

[timebertt]

Hi @kwitsch, can you provide some examples to elaborate on what you mean (e.g. user-data, resulting sources.list file and so on)?

Also, have you checked https://cloudinit.readthedocs.io/en/latest/topics/modules.html#apt-configure?
cloud-config has settings for dealing with the apt sources: preserve_sources_list and sources_list. Maybe that already helps you?

[kwitsch]

Example from my test-swarm which kind of works but is messy:

package_update: true
package_upgrade: true
package_reboot_if_required: true

apt_preserve_sources_list: true
apt:
  sources:
    docker.list:
      source: deb [arch=armhf] https://download.docker.com/linux/raspbian $RELEASE stable
      keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88

packages:
  - docker-ce
  - docker-ce-cli

It errors during packages addition but recovers through package_update&upgrade.
The problem is that cloud-init uses its own sources.list and adds additional sources to it. The currently used sources.list is a debian one which doesn't work on raspbian.
If you remove "apt_preserve_sources_list: true" from my example you could look at the currently used sources.list after cloud-init has finished. sources.list

[timebertt]

The currently used sources.list is a debian one which doesn't work on raspbian.

I'm not sure, I can follow you.

Have you tried switching to https://download.docker.com/linux/debian? raspiOS is pretty much Debian and as far as I can tell, the Debian packages should work there as well. Though, I'm definitely not an expert here.

I tried the following user-data on my PI with the arm64 image and it worked fine:

package_update: true
apt:
  sources:
    docker.list:
      source: deb [arch=arm64] https://download.docker.com/linux/debian $RELEASE stable
      keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88

packages:
  - docker-ce
  - docker-ce-cli

$ sudo docker run hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.
...

[kwitsch]

Ok i will try that next weekend and provide an answer.
All my tests are done with the armhf image.
If I let cloud-init overwrite the default sources.list with the debian one i get GPG errors.
Maybe all of this is caused by the mixup between debian and raspbian repos.

I would be glad if its just some configuration flaw on my part, I really like the idea of spinnung up a swarm without all the after-flash configuration 😅
@timebertt big thanks to you for providing this image

[kwitsch]

done some testing, here the results:

apt:
  sources:
    docker.list:
      source: deb [arch=arm64,armhf] https://download.docker.com/linux/debian $RELEASE stable
      keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88

packages:
  - docker-ce
  - docker-ce-cli

works fine in the arm64 image but creates problems during the install on armhf

apt:
  primary:
    - arches: [default]
      uri: http://raspbian.raspberrypi.org/raspbian/
  sources:
    docker.list:
      source: deb [arch=armhf] https://download.docker.com/linux/raspbian $RELEASE stable
      keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88

packages:
  - docker-ce
  - docker-ce-cli

works fine on armhf

as result i create different vendor-data files for the different images

[timebertt]

Ah, good to know, that this behaves differently on armhf.
Let me test this on my Pis and see if we can/should change the default mirror for the armhf images.