packages/repostitories: add pacman support

[ocelotsloth]

Adds support for package and repository reporting on pacman-based distributions.

As implemented--currently omits security update information, these rolling distributions only support complete updates and do not support security-only update options.

PatchMon/PatchMon#373

Testing

Host Level Reporting

Repository Reporting (with Include statements traversed)

Outdated Package Reporting

Complete Package Reporting

[ocelotsloth]

This should be good for testing. @B-DUB99 (PatchMon/PatchMon#373) are you able to independently test?

git clone https://github.com/ocelotsloth/PatchMon-agent.git PatchMon-agent-ocelotsloth
cd PatchMon-agent-ocelotsloth
sed -i 's|GO_CMD=/usr/local/go/bin/go|GO_CMD=go|' Makefile
make

At that point you can replace your local copy (maybe keep a backup so you can revert) and test:

sudo mv build/patchmon-agent /usr/local/bin/patchmon-agent
sudo systemctl restart patchmon-agent
sudo tail -f /etc/patchmon-agent/logs/patchmon-agent.log

[ocelotsloth]

I did find this issue while testing but it seems to be a regression in the front and/or backend--I was able to see all the packages with the mitigation outlined in the issue: PatchMon/PatchMon#386

[raspitoaster]

I just tested your branch and it is amazing! Because I am using Debian and Arch (maybe 40:60), I upvoted the issue on Github and a few days later, there is an already working version available.

Two findings so far:

1. the patchmon-agent log is in (etc/patchmon/logs - so the command above should read tail -f /etc/patchmon/logs/patchmon-agent.log instead of tail -f /etc/patchmon-agent/logs/patchmon-agent.log

2. The Arch repositories are reported by source and not by repo - I use reflector which generates a list of sources (the mirrorlist) based on rules defined in /etc/xdg/reflector/reflector.conf

Instead of having all the mirrors listed in /etc/pacman.d/mirrorlist reported as repositories in PatchMon,

we should have only the enabled repositories defined (=uncommented) in /etc/pacman.conf like with Debian.

[ocelotsloth]

1. the patchmon-agent log is in (etc/patchmon/logs - so the command above should read tail -f /etc/patchmon/logs/patchmon-agent.log instead of tail -f /etc/patchmon-agent/logs/patchmon-agent.log

Definitely agree, I noticed that too. Let's address it in a separate PR, I believe there's a PR for it already (#13 ).

Sort of in the same vein as doing things the "normal" way I have some thoughts about packaging the agent into a dev, rpm, etc.

Instead of having all the mirrors listed in /etc/pacman.d/mirrorlist reported as repositories in PatchMon,

Hmm I'll test this—I must not have any commented mirrors on my machine. Should be an easy fix thank you!

Might be tomorrow before I get to this.

[raspitoaster]

Hmm I'll test this—I must not have any commented mirrors on my machine. Should be an easy fix thank you!

Might be tomorrow before I get to this.

Looking at your screenshot (Repository Reporting) above, you have at least 9 active mirrors for the "core" repo.
I highly appreciate your work as it is already very useful - would be perfect if you could fix that issue.

[ocelotsloth]

Looking at your screenshot (Repository Reporting) above, you have at least 9 active mirrors for the "core" repo. I highly appreciate your work as it is already very useful - would be perfect if you could fix that issue.

I think I might be misunderstanding you--I'm not sure how to reproduce what you're describing:

I went ahead and added a couple test cases:

https://github.com/ocelotsloth/PatchMon-agent/blob/71fdf6ab02fef64fe24673258a8f474f3c23b445/internal/repositories/pacman_test.go#L50-L56

Along with the actual check:

https://github.com/ocelotsloth/PatchMon-agent/blob/main/internal/repositories/pacman_test.go#L91-L98

It looks like it's correctly ignoring commented out lines from the configuration file.

It's expected that multiple entries will show up for each repository if you define multiple Server mirror blocks. Perhaps we instead need the data format to allow for multiple sources in the same Repository struct?

// pkg/models.go

// Repository represents a software repository
type Repository struct {
	Name         string `json:"name"`
	URL          string `json:"url"`
	Distribution string `json:"distribution"`
	Components   string `json:"components"`
	RepoType     string `json:"repoType"`
	IsEnabled    bool   `json:"isEnabled"`
	IsSecure     bool   `json:"isSecure"`
}

I went with reporting individual Repository objects for each configured mirror because it seemed to fit best with the UI's reporting of "secure" and "insecure" repositories.

[ocelotsloth]

https://wiki.debian.org/SourcesList#Repository_URI

{i} despite being called URIs, the new format only supports one URI per entry

https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/5/html/deployment_guide/s1-yum-yumconf-repository

baseurl=url, file or ftp://path to repository

It does seem like Arch is a little unique in this way but I've never actually tried to find a way to configure fallback mirrors on other distributions before. I may experiment with it later.

[raspitoaster]

yes - ok, the way Arch addresses the package sources is different from Debian.

Debian:

## Repo and RepoSource definitions in /etc/apt/sources.list
##
deb http://ftp.debian.org/debian bookworm main contrib
deb http://ftp.debian.org/debian bookworm-updates main contrib
deb http://security.debian.org bookworm-security main contrib

Arch:

## Repo definitions in /etc/pacman.conf
##
[endeavouros]
SigLevel = PackageRequired
Include = /etc/pacman.d/endeavouros-mirrorlist
#[core-testing]
#Include = /etc/pacman.d/mirrorlist
[core]
Include = /etc/pacman.d/mirrorlist
#[extra-testing]
#Include = /etc/pacman.d/mirrorlist
[extra]
Include = /etc/pacman.d/mirrorlist

## then the RepoSource definitions in /etc/pacman.d/mirrorlist
## Germany
Server = https://de.arch.niranjan.co/$repo/os/$arch
Server = https://mirror.moson.org/arch/$repo/os/$arch
Server = https://mirror.kumi.systems/archlinux/$repo/os/$arch

## and in /etc/pacman.d/endeavouros-mirrorlist
## /etc/pacman.d/endeavouros-mirrorlist
## Germany
Server = https://mirror.alpix.eu/endeavouros/repo/$repo/$arch
Server = https://mirror.moson.org/endeavouros/repo/$repo/$arch
Server = https://ftp.rz.tu-bs.de/pub/mirror/endeavouros/repo/$repo/$arch
Server = https://de.repo.c48.uk/endeavouros/repo/$repo/$arch

The RepoSource definitions in /etc/pacman.d/mirrorlist and /etc/pacman.d/endeavouros-mirrorlist

• one mirrorlist file per Repo
• multiple lines per repo recommended, sorted by quality)
• placeholder possible, one RepoSource (server line) matches multiple Repos

--> the line "Server = https://mirror.moson.org/arch/$repo/os/$arch" in /etc/pacman.d/mirrorlist in combination with /etc/pacman.conf expands to:

https://mirror.moson.org/arch/core/os/x86_64/
https://mirror.moson.org/arch/extra/os/x86_64/

--> the line "Server = https://mirror.moson.org/endeavouros/repo/$repo/$arch" in /etc/pacman.d/endeavouros-mirrorlist in combination with /etc/pacman.conf expands to:

https://mirror.moson.org/endeavouros/repo/endeavouros/x86_64/

The logic of the conf file (/etc/pacman.conf) and the conf directories (/etc/pacman.d/) is the same as in Debian.
You can define includes that point to files and directories containing files.

From my point of view, the patchmon-agent should not care about mirrorlists or even mirrors.
It is sufficient to parse the pacman conf file (usually /etc/pacman.conf) and optional included files/directories (defined by "Include =" in the file /etc/pacman.conf) for patterns like [core] or [extra].
In the above example, we should end up with 3 repositories:

[core]
[extra]
[endeavouros]

I hope that helps.

[9technologygroup]

Hi team.

I hope all are well :)
Thank you so much for all of your hard work on this.

What's your opinion @ocelotsloth in relation to some of the comments @raspitoaster has mentioned about repolists?

Do you feel that the agent as you've configured it is enough to push into release or some more work needed.

Thanks
iby

[raspitoaster]

I am happy with the implementation because I am in a special situation using a central package cache, so I have only one "server="-line per repository.
Almost anyone else, using (the standard) pacman method where mirrorlists are managed by tools like reflector/manjaro-mirrors/eos-rankmirrors, will have tens or even hundreds of mirrors listed as repos in PatchMon. This could be avoided if PatchMon-agent on Arch systems would consider pacman repos as repos (as proposed above) and not mirrors (as currently implemented by @ocelotsloth's commit).