Skip to content

thar-be-registries: BTreeMap key sorting breaks endpoint order in hosts.toml #898

@Riqardos

Description

@Riqardos

Package I'm using:

thar-be-registries in bottlerocket-core-kit

What I expected to happen:

When configuring settings.container-registry.mirrors with multiple endpoints, the generated /etc/containerd/certs.d/<registry>/hosts.toml should preserve the endpoint order as specified. Since containerd uses section order as priority (first = primary, rest = fallback), endpoints should appear in hosts.toml in the same order they were configured.

What actually happened:

The endpoint order in hosts.toml does not match the configured order — endpoints are sorted alphabetically instead. This causes containerd to use an unintended endpoint as primary.

Verified on a live node:

  • /etc/containerd/thar-be-registries.toml — endpoint order is correct (matches configuration)
  • /etc/containerd/certs.d/docker.io/hosts.toml — endpoint order is wrong (sorted alphabetically)
# thar-be-registries.toml (CORRECT order)
[[mirrors]]
registry = "docker.io"
endpoint = [
  "https://mirror-a.example.com",
  "https://mirror-b.example.com"

# /etc/containerd/certs.d/docker.io/hosts.toml (WRONG order — sorted alphabetically)
server = "https://registry-1.docker.io"

[host."https://mirror-b.example.com"]
capabilities = ["pull", "resolve"]

[host."https://mirror-a.example.com"]
capabilities = ["pull", "resolve"]

How to reproduce the problem:

Configure two or more mirror endpoints whose URLs sort in a different alphabetical order than the intended priority order. Inspect the generated hosts.toml — endpoints will appear in alphabetical order regardless of the configured order.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions