export more information from apt_info.py: package count, obsolete, broken packages

[anarcat]

Hi,

We're migrating away from icinga here and we have an in-house check that provides us with quite useful metrics.

apt_info.py has this:

# /usr/share/prometheus-node-exporter-collectors/apt_info.py
# HELP apt_upgrades_pending Apt packages pending updates by origin.
# TYPE apt_upgrades_pending gauge
apt_upgrades_pending{origin="",arch=""} 0
# HELP apt_upgrades_held Apt packages pending updates but held back.
# TYPE apt_upgrades_held gauge
apt_upgrades_held{origin="",arch=""} 0
# HELP apt_autoremove_pending Apt packages pending autoremoval.
# TYPE apt_autoremove_pending gauge
apt_autoremove_pending 21
# HELP node_reboot_required Node reboot is required for software updates.
# TYPE node_reboot_required gauge
node_reboot_required 0

our dsa-check-packages has this:

# dsa-check-packages 
OK: 618 ok, 4 rc
618 packages current.
4 packages removed but not purged: linux-image-6.1.0-18-amd64, linux-image-6.1.0-17-amd64, linux-image-6.1.0-21-amd64, linux-image-6.1.0-20-amd64
|obs_loc=0;1;5;0 outdated=0;1;5;0 current=618;;;0 obs_ign=0;;;0 rm_unprg=4;;;0 hold=0;;;0 prg_conf=0;1;;0

that is:

• held packages (apt_upgrades_held)
• obsolete packages, including a way to exclude packages (not covered!)
• outdated packages (apt_upgrades_pending)
• current packages (not covered!)
• removed packages with leftover configuration ("removed but not purged", not covered)
• purged packages with leftover configuration (!?) (not covered)
• pending upgrades but that apt upgrade won't upgrade (see for example Ubuntu phased upgrades or pinned packages?)
• add an ignore list

the "obsolete packages" is the most important, as those are packages that might not be covered by security upgrade and a manual configuration...

i plan on adding those into apt_info.py through a merge request unless someone objects in a coming weeks. please let me know.

This issue is tracked in https://gitlab.torproject.org/tpo/tpa/team/-/issues/41712 on our side.

[Sda79]

I don't know if it's the right place, but could you also not include phasing updates ?
Because right now the apt_info script will report pending update even when these packages are being deferred due to phasing.

For example when you want to upgrade with apt upgrade command :

The following upgrades have been deferred due to phasing:
  cloud-init python3-distupgrade ubuntu-release-upgrader-core
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

The apt_upgrades_pending metric will stay a 3 in this case.
Phasing updates are used in Ubuntu.

[anarcat]

not part of my use case, but sure, that would be nice.

how do you tell those apart? how does apt know those shouldn't be updated?

in this case, for example, could you show the output of apt-cache policy cloud-init?

[dswarbrick]

The script would need to test for this property of the packages returned by the cache: https://apt-team.pages.debian.net/python-apt/library/apt.package.html#apt.package.Package.phasing_applied

For example:

>>> cache = apt.cache.Cache()
>>> [p for p in cache if p.phasing_applied]
[<Package: name:'distro-info-data' architecture='amd64' id:1833>, <Package: name:'python3-distupgrade' architecture='amd64' id:10292>, <Package: name:'ubuntu-release-upgrader-core' architecture='amd64' id:10793>, <Package: name:'ubuntu-release-upgrader-qt' architecture='amd64' id:41862>]

[anarcat]

nice!

also:

i plan on adding those into apt_info.py through a merge request unless someone objects in a coming weeks. please let me know.

i'm rather busy right now, so if someone else wants to work on this, go for it, just ping me here so i don't start duplicating work (and i'll do the same if i start working on it).