src/konflux-rpm-lockfile: update baseurl dynamically #4381
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request introduces a temporary solution to a DNF5 issue by dynamically updating repository base URLs for ppc64le and s390x architectures. This is achieved by creating temporary .repo files with modified fedora-secondary URLs. The approach directly addresses the problem and is integrated into the package resolution logic. However, there are a couple of areas related to resource management and regex specificity that could be improved for robustness and maintainability.
joelcapitao
force-pushed
the
fix-konflux-rpm-lockfile
branch
2 times, most recently
from
67fbc04 to
dc1ec48
Compare
Until now, we tackled the DNF5 issue [1] by prioritizing the URL location of the repo 'coreos-pool' when multiple repo were found for the say locked NEVRA. But, if that locked NEVRA has only one URL location coming from the fedora repos i.e: fedora, fedora-updates, then the repoquery command returns a broken URL location for ppc64le and s390x arches because of the DNF5 issue. It returns the URL from the first item defined in baseurl, whereas the resolution worked with the second item i.e fedora-secondary repo where the RPMs for ppc64le and s390x are available. This patch fixes this until the issue [1] is fixed upstream. It's designed to work only with the main fedora baseurl FQDN i.e dl.fedoraproject.org and for ppc64le/s390x arches. It basically updates the repos to use the fedora-secondary URL as the main one as we know the RPMs for those arches lives there. [1] rpm-software-management/dnf5#1673
joelcapitao
force-pushed
the
fix-konflux-rpm-lockfile
branch
from
dc1ec48 to
f85d2da
Compare
reading the history in that issue it seems to me like they never really got a good reproducer for the issue. It seems like we have a reliable reproducer? |
hmm. did you mean to link rpm-software-management/dnf5#2466 ? |
| # Prioritize the url from fedora-coreos-pool | ||
| # there is a bug in dnf here where the url returned is incorrect when the | ||
| # repofile have more than one baseurl, which causes ppc64le and s390x | ||
| # urls comming from fedora and fedora-updates to be invalid | ||
| # See https://github.com/rpm-software-management/dnf5/issues/2466 | ||
| existing_url = processed_urls.get(name, None) | ||
| if 'coreos-pool' in url or not existing_url: | ||
| if not existing_url: | ||
| processed_urls[name] = url |
There was a problem hiding this comment.
I'll argue here that we still want to prioritize coreos-pool URLs.
The reason is that URLs for RPMs from the fedora-updates repos can become stale (i.e. if a newer version of a package is created then only that newer version will be in the updates repo).
Basically IMO fedora-updates repo URLs can't be relied on. If you immediately try to do a build then everything should work, but if you delay doing a build OR somehow want to go back later (2 days, 7 days, 30 days) and try to do a build it will fail.
There was a problem hiding this comment.
I did a retrospective to understand better what happened with the glib2 URL c.f coreos/fedora-coreos-config#3723 (comment) that lead to this PR.
- 29 Nov 2025 22:25:06 UTC
glib2-2.86.2-1.fc43was built https://koji.fedoraproject.org/koji/buildinfo?buildID=2871528 - 1 Dec 11:11 PM UTC, it was resolved in
bump-lockfilejob and push coreos/fedora-coreos-config@c0abc9c. The URL was wrong: https://dl.fedoraproject.org/pub/fedora/linux/updates/43/Everything/ppc64le/Packages/g/glib2-2.86.2-1.fc43.ppc64le.rpm - 01 Dec 2025 11:11:34 UTC, coreos-koji-tagger tagged it in
coreos-poolhttps://koji.fedoraproject.org/koji/taskinfo?taskID=139580263 because of the bump-lockfile push commit. - in the next bump-lockfile commit, the coreos-pool URL was used coreos/fedora-coreos-config@3f8148a because it's prioritized.
We are fixing this cornercase, where a NEVRA is not yet tagged in coreos-pool when pushing a bump-lockfile commit. The URL from fedora repo is used in N bump-lockfile commit, and coreos-pool URL is used in N+1.
I agree we still need to prioritize coreos-pool URLs to not break our pipeline if things move very fast in fedora repos.
But still we have to rely on Fedora URL between bump-lockfile commits. If fedora repos content changes in the meantime, then this still might break our pipeline. So maybe we should go a step further by relying on coreos-pool URLs ONLY.
This can be achieved by constructing the coreos-pool URL as we know the prefix and the NEVRA i.e https://kojipkgs.fedoraproject.org/repos-dist/coreos-pool/latest/aarch64/Packages/<first_letter_of_nvr>/<nvr>. We are doing something similar for the overrides c.f coreos/fedora-coreos-config#3905. That would be more robust and we won't have to merge this PR content which is fragile IMO, WDYT ?
2 participants
Until now, we tackled the DNF5 issue [1] by prioritizing the URL location of the repo 'coreos-pool' when multiple repo were found for the say locked NEVRA.
But, if that locked NEVRA has only one URL location coming from the fedora repos i.e: fedora, fedora-updates, then the repoquery command returns a broken URL location for ppc64le and s390x arches because of the DNF5 issue. It returns the URL from the first item defined in baseurl, whereas the resolution worked with the second item i.e fedora-secondary repo where the RPMs for ppc64le and s390x are available.
This patch fixes this until the issue [1] is fixed upstream. It's designed to work only with the main fedora baseurl FQDN i.e dl.fedoraproject.org and for ppc64le/s390x arches. It basically updates the repos to use the fedora-secondary URL as the main one as we know the RPMs for those arches lives there.
[1] rpm-software-management/dnf5#1673