[DPE-9356] Update PostgreSQL mount paths to /var/lib/pg with 16/main subdirectory by marceloneppel · Pull Request #1186 · canonical/postgresql-k8s-operator

marceloneppel · 2025-12-18T20:06:39Z

Issue

The database uses the mount points provided by Juju as the data directories. This fails the database bootstrap process when there is some content in those mount points, like lost+found.

Also, the storage mount paths use /var/lib/postgresql/*, which doesn't follow the 16/main subdirectory convention expected by PostgreSQL tooling.

Solution

Following the spec DA230, migrate storage mount paths from /var/lib/postgresql/* to /var/lib/pg/* with a 16/main subdirectory structure, and create a compatibility symlink from the standard PostgreSQL data path.

Path changes

Storage	Old path	New path
Archive	`/var/lib/postgresql/archive`	`/var/lib/pg/archive/16/main`
Data	`/var/lib/postgresql/data/pgdata`	`/var/lib/pg/data/16/main`
Logs (WAL)	`/var/lib/postgresql/logs`	`/var/lib/pg/logs/16/main/pg_wal`
Temp	`/var/lib/postgresql/temp`	`/var/lib/pg/temp/16/main/pgsql_tmp`

Key changes

Storage mounts (metadata.yaml): Update all four storage mount locations to /var/lib/pg/*
Named constants (constants.py): Extract storage paths into ARCHIVE_STORAGE_PATH, DATA_STORAGE_PATH, LOGS_STORAGE_PATH, TEMP_STORAGE_PATH and use them consistently across the codebase
Symlink compatibility: Create a symlink from /var/lib/postgresql/16/main → /var/lib/pg/data/16/main so PostgreSQL and Patroni reference the canonical path; always recreate the symlink (removing any real directory the OCI image ships first)
Patroni & pgBackRest templates: Update patroni.yml.j2 and pgbackrest.conf.j2 with new data_dir, waldir, and temp tablespace paths
Backup/restore (backups.py): Clear all storage directories during restore preparation so pg_basebackup --waldir/--tablespace options work on replicas; use actual pgdata path (not symlink) for file operations
Async replication (async_replication.py): Prevent system ID mismatch by only clearing pgdata on non-leaders after the standby leader is confirmed running; check member_started before clearing pgdata on standby units; fix disk usage check to use the actual pgdata path
Replica startup (charm.py): Defer replica startup until its endpoint is present in the members list, preventing pg_basebackup rejections; clear stale WAL/temp directories repeatedly until pgdata is fully populated (PG_VERSION exists)
Security: Fix shell injection risk by passing command arguments as lists instead of split strings
Upgrade tests disabled: Temporarily disable upgrade-related spread tests since upgrades cannot be performed when storage definitions change
New integration test: Add test_symlinks.py to verify the data directory symlink works correctly

Checklist

I have added or updated any relevant documentation.
I have cleaned any remaining cloud resources from my accounts.

Fixes: #1009

Migrate storage mount paths from /var/lib/postgresql/* to /var/lib/pg/* and configure PostgreSQL 16 data directories to use the 16/main subfolder structure. Path changes: - Archive: /var/lib/postgresql/archive → /var/lib/pg/archive - Data: /var/lib/postgresql/data/pgdata → /var/lib/pg/data/16/main - Logs: /var/lib/postgresql/logs → /var/lib/pg/logs/16/main (WAL) - Temp: /var/lib/postgresql/temp → /var/lib/pg/temp/16/main Key changes: - Update metadata.yaml storage mount locations - Update POSTGRESQL_DATA_PATH constant - Configure pgdata_path, waldir, and temp tablespace paths - Add make_parents=True to Pebble make_dir for nested directory creation - Update pgbackrest.conf.j2 with new data path - Update patroni.yml.j2 with new data_dir and waldir - Update async_replication.py cleanup paths - Update authorisation_rules_observer.py Patroni config path Note: Archive path (/var/lib/pg/archive) intentionally does not use the 16/main subfolder as it is not used in Patroni or PostgreSQL configuration. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

codecov · 2025-12-18T20:07:47Z

Codecov Report

❌ Patch coverage is 53.01205% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.55%. Comparing base (ccb262c) to head (26f96b4).
⚠️ Report is 1 commits behind head on 16/edge.

Files with missing lines	Patch %	Lines
src/charm.py	54.05%	14 Missing and 3 partials ⚠️
src/patroni.py	14.28%	12 Missing ⚠️
src/relations/async_replication.py	46.66%	8 Missing ⚠️
src/backups.py	80.00%	0 Missing and 2 partials ⚠️

❌ Your project status has failed because the head coverage (68.55%) is below the target coverage (70.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff             @@
##           16/edge    #1186      +/-   ##
===========================================
- Coverage    68.96%   68.55%   -0.42%     
===========================================
  Files           16       16              
  Lines         3809     3864      +55     
  Branches       574      588      +14     
===========================================
+ Hits          2627     2649      +22     
- Misses         980     1008      +28     
- Partials       202      207       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ix_pod Patch the OCI image health-check script (/scripts/self-signed-checker.py) to use the charm's storage path for the peer CA file instead of the hardcoded /var/lib/postgresql/data/peer_ca.pem. Make the operation idempotent and non-blocking, and call it from pebble-ready and early pod-fix flows to avoid race conditions with Pebble layer installation. Add _fix_health_check_script(container) to read, replace, and write the script. Log and handle pull/push errors; on push failure attempt an in-container sed fallback (exec ["sh","-c","sed -i 's|OLD|NEW|g' ..."]). Invoke the fixer from _on_postgresql_pebble_ready (after _create_pgdata) and from _fix_pod (early) so the script is corrected as early as possible. Update unit tests to cover the patching behaviour and the exec fallback. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>