Skip to content

Conversation

@ecourtois
Copy link
Contributor

Context & Motivation: We currently face a version mismatch preventing the use of Raspberry Pi cameras:

  • Current OS: Raspberry Pi OS Bookworm (Debian 12) provides Python 3.11.
  • Current Stack: cogip-tools runs on Python 3.12.
  • The Blocker: The python3-libcamera package (required for libcamera support) is a system package compiled against the OS Python version (3.11). It is incompatible with our venv running Python 3.12.

Goal: To resolve this, we must align the cogip-tools Python version with the OS version. We will upgrade the base image to Debian 13 (Trixie), which provides Python 3.13.

Required Changes & Impact

  1. Python 3.13 Upgrade & Dependencies
  • Upgrade cogip-tools to Python 3.13.
  • Update all third-party wheel dependencies to versions supporting Python 3.13.
  1. Monitor Application Rewrite (Major Refactoring) The upgrade forces a rewrite of the Monitor application
  • Issue: The Monitor relies on an older version of PySide6 (constrained by strict dependencies) which does not support Python 3.13.
  • Deprecation: The module Qt3D is deprecated in newer Qt versions.
  • Solution: The Monitor will be completely rewritten using modern QtQuick / QtQuick3D.
  1. System & Boot Process Debian Trixie introduces changes to the boot process
  • Adapt system configuration and systemd service orchestration.
  • Optimize dependency management during boot.
  1. Camera Tools
  • Update camera tools to support both USB and Raspberry Pi native cameras.

If the Socket.IO namespace connection is refused, the server must raise
a ConnectionRefusedError for the socketio library, not from stdlib.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
…obot

Adding the robot to the Monitor requires that Planner and Detector are
connected to know if they are virtual or not.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
@ecourtois ecourtois self-assigned this Dec 4, 2025
@ecourtois ecourtois added the dependencies Pull requests that update a dependency file label Dec 4, 2025
@ecourtois ecourtois linked an issue Dec 4, 2025 that may be closed by this pull request
@ecourtois ecourtois force-pushed the 185-raspios-upgrade-base-image-to-debian-13-trixie-and-python-313 branch 5 times, most recently from 1429264 to 350fd09 Compare December 13, 2025 15:17
@ecourtois ecourtois force-pushed the 185-raspios-upgrade-base-image-to-debian-13-trixie-and-python-313 branch 2 times, most recently from bba7fa3 to 83e4f19 Compare December 15, 2025 20:45
Copy link
Contributor

@gdoffe gdoffe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few things.
And please remove markdown from at least one commit message (raspios: migration to Trixie and overhaul systemd orchestration )

Migrate the Raspberry Pi OS base image from Bookworm to Trixie (Debian 13).

This migration required a major overhaul of the system configuration,
Debian packages, and systemd service orchestration to fix new dependencies
and optimize the boot process.

- Updated obsolete `apt` package names:
    - `chromium-browser` -> `chromium`
    - `libegl1-mesa`/`libgles2-mesa` -> `libegl1`/`libgles2`

- Removed obsolete `apt` packages (`cpufreqd`, `cpufrequtils`).

- Updated `sysctl` configuration (ip_forward) to use the `/etc/sysctl.d/`
  directory instead of `/etc/sysctl.conf`.

- userconfig: Neutralized the `userconfig` service (first-boot wizard)
  which was blocking network startup. A systemd override
  (`/etc/systemd/system/userconfig.service.d/override.conf`) now forces
  the service to execute `/bin/true`.

- rfkill: Masked the `systemd-rfkill.socket` and `.service`
  to prevent the Wi-Fi from being blocked on boot.

- cloud-init: Completely disabled all `cloud-init` services
  to massively reduce boot time.

- weston:
    - Migrated Weston launch from the `.bash_profile` autologin to a
      dedicated systemd service (`weston.service`).
    - Changed the default boot target from `multi-user.target`
      to `graphical.target`.
    - Fixed the `XDG_RUNTIME_DIR` not set error by using
      the `RuntimeDirectory=` and `Environment=` directives in the service.
    - Removed file-based logging (`--log`) in favor of the systemd
      journal (capturing stdout/stderr).

- Build Order: Fixed a major bug where services (e.g., `ModemManager`,
  `avahi`) were being re-enabled by `cogip-*` dependencies.
  The `disable`/`mask` commands are now executed last in the Dockerfile's
  `robot` and `beacon` stages.

- Network (systemd-networkd):
    - Fixed `wlan0` being stuck in the "configuring" state by keeping
      the `wpa_supplicant@wlan0.service` but adding `LinkLocalAddressing=no`
      to the `.network` file.
    - Enabled `systemd-networkd-wait-online.service@wlan0` to synchronize
      `cogip-*` service startup with the Wi-Fi connection.

- Service Notification (`Type=notify`):
    - Modified `cogip-server` and `cogip-dashboard` to use `Type=notify`
      to ensure dependencies start in the correct order.
    - Added `systemd.daemon.notify("READY=1")` to the Python application
      code (via `lifespan`/`on_startup`).
    - Changed `ExecStart=` to call `/opt/.venv/bin/python -m ...` directly,
      bypassing a `uv run` bug that was not passing the `$NOTIFY_SOCKET`
      variable.

- Service Dependencies (Lidar):
    - Made `cogip-detector` (Lidar) an optional dependency for
      `cogip-robotcam` by using `Wants=` instead of `Requires=`.
    - Added a `check-lidar.py` script (via `ExecStartPre=`) that actively
      probes the Lidar (YDLidar G2, `0xA5 0x92` command) to prevent the
      service from starting if the Lidar is absent (Robot 1 only).

Breaking change: the base image is now Trixie. The former autologin
mechanism is replaced by `graphical.target` and the `weston.service`.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
v4l2py has been deprecated in favor of linuxpy.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Add support for Raspberry Pi cameras using picamera2.
Replace SharedMemory with multiprocessing.Queue for frame transfer.
Use two different streams for processing (high res, gray) and web streaming
(low res, colour).

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Monitor is developed using Qt3D which not officially maintained anymore.
Last compatible version of PySide6 does not support Python 3.13,
so Monitor needs to be modernized by being completely rewritten
using QtQuick and QtQuick3D.
This commit removes old Monitor.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Add new Monitor developed using PySide6 and QtQuick/QtQuick3D.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Upgrade all mkdocs related packages to latest version.
Fix remaining warnings.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Nanobind version was not pinned in build-system section.
Version 2.10 has introduced a regression, so pin the version
to the previous one.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
Fix issue with mixed root/user permissions.

Summary of new/working state:
- source directory is copied in the Dockerfile with root permissions
to allow synchronisation using watch feature of docker compose.
- each robot service has its own build dir (/src/build/${ROBOT_ID}) to
prevent override to parallel startup.
- "uv sync" command is executed with root permission because it creates
bindings (__init__.pyi) in the source directory.
- "uv run" commands are executed with user permissions because tools are
using shared memory files in "/dev/shm" mounted from the host.
- to avoid permission denied errors due to recompilation, "uv run" is
launched with "--no-sync" option.
- "/src/.venv" bound to venv_cache volume so monitor service can reuse
the venv created by robot service.

Signed-off-by: Eric Courtois <eric.courtois@gmail.com>
@ecourtois ecourtois force-pushed the 185-raspios-upgrade-base-image-to-debian-13-trixie-and-python-313 branch from 83e4f19 to 96c2d83 Compare December 16, 2025 19:26
@ecourtois ecourtois requested a review from gdoffe December 16, 2025 19:26
Copy link
Contributor

@gdoffe gdoffe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ecourtois ecourtois merged commit ecd0f89 into master Dec 17, 2025
4 checks passed
@ecourtois ecourtois deleted the 185-raspios-upgrade-base-image-to-debian-13-trixie-and-python-313 branch December 17, 2025 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file monitor raspios refactoring Code refactoring robotcam

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Raspios: upgrade base image to Debian 13 (Trixie) and Python 3.13

2 participants