Skip to content

Podman support and option to backup across all projects #66

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

Podman support and option to backup across all projects #66

wants to merge 15 commits into from
+87 −42

Conversation

@AlexMcDermott
Copy link

@AlexMcDermott AlexMcDermott commented Sep 16, 2025 •
edited
Loading

Hi, cool project

Contents

Podman Hostname issue

I was running into this issue and found it more reliable to use the hostname from the socket library instead.

[alexmcdermott@grom:~/homelab/containers/stack-back]$ podman exec -it stack-back rcb backup
Traceback (most recent call last):
File "/restic-compose-backup/.venv/bin/rcb", line 10, in <module>
sys.exit(main())
^^^^^^
File "/restic-compose-backup/restic_compose_backup/cli.py", line 23, in main
containers = RunningContainers()
^^^^^^^^^^^^^^^^^^^
File "/restic-compose-backup/restic_compose_backup/containers.py", line 411, in **init**
if container_data.get("Id").startswith(os.environ["HOSTNAME"]):
~~~~~~~~~~^^^^^^^^^^^^
File "<frozen os>", line 714, in **getitem**
KeyError: 'HOSTNAME'

Podman network issue

This was triggered when the internal backup container is created. Because it was trying to use the main backup containers network, without a pod being specified, and not started as part of the same compose file.

[alexmcdermott@grom:~/homelab]$ podman exec -it stack-back rcb backup
2025-09-15 21:55:10,130 - INFO: Starting backup container
2025-09-15 21:55:11,391 - ERROR: 500 Server Error for http+docker://localhost/v1.41/containers/create: Internal Server Error ("container create: container dependency b68b86daed653b6a34f74873cf14ca0550bda075cb7c21d4a5354f82ee0b84a9 is part of a pod, but container is not: invalid argument")
Traceback (most recent call last):
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/docker/api/client.py", line 275, in *raise*for_status
response.raise_for_status()
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/containers/create

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/restic-compose-backup/restic_compose_backup/cli.py", line 167, in backup
result = backup_runner.run(
^^^^^^^^^^^^^^^^^^
File "/restic-compose-backup/restic_compose_backup/backup_runner.py", line 20, in run
container = client.containers.run(
^^^^^^^^^^^^^^^^^^^^^^
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/docker/models/containers.py", line 876, in run
container = self.create(image=image, command=command,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/docker/models/containers.py", line 935, in create
resp = self.client.api.create_container(**create_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/docker/api/container.py", line 440, in create_container
return self.create_container_from_config(config, name, platform)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/docker/api/container.py", line 457, in create_container_from_config
return self._result(res, True)
^^^^^^^^^^^^^^^^^^^^^^^
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/docker/api/client.py", line 281, in *result
self.*raise_for_status(response)
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/docker/api/client.py", line 277, in *raise*for_status
raise create_api_error_from_http_exception(e) from e
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/restic-compose-backup/.venv/lib/python3.12/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation) from e
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/containers/create: Internal Server Error ("container create: container dependency b68b86daed653b6a34f74873cf14ca0550bda075cb7c21d4a5354f82ee0b84a9 is part of a pod, but container is not: invalid argument")
2025-09-15 21:55:11,396 - INFO: No alerts configured

lawndoc
lawndoc reviewed Sep 24, 2025
View reviewed changes
Copy link
Owner

@lawndoc lawndoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice simplification of the container filter loop. Please see my comment about the shared network stack

@lawndoc
Copy link
Owner

lawndoc commented Sep 24, 2025

more reliable to use the hostname from the socket library instead.

This is fine, using socket should be backward compatible

@lawndoc lawndoc mentioned this pull request Sep 24, 2025
Open
@AlexMcDermott AlexMcDermott marked this pull request as ready for review September 27, 2025 10:11
@AlexMcDermott AlexMcDermott changed the title Podman support and option of backup across all projects Podman support and option to backup across all projects Oct 15, 2025
@lawndoc
Copy link
Owner

lawndoc commented Oct 18, 2025 •
edited
Loading

Doing some testing of this PR now, sorry it's taken me so long to get to this. Everything is going well in my previous test setup. Looks like the container filter changes will also fix #62 which is a nice bonus.

Still need to test:

  • database backup with multiple docker networks after the changes to the backup container's network stack
  • the new config variable to backup all projects with one backup container

@lawndoc
Copy link
Owner

lawndoc commented Oct 18, 2025 •
edited
Loading

@AlexMcDermott your code is working as expected for individual projects, but it fails when backing up databases in other compose projects. This is because docker compose will create different networks for each compose project.

You can import other networks into your compose project, but that's not a good user experience.

Instead, I think we can give the backup container access to every network that it will need access to, instead of just the network defined for the main backup container. I actually like this better anyways, because then we don't have to mess with the backup container's network even within a single compose project.

lawndoc
lawndoc requested changes Oct 18, 2025
View reviewed changes
lawndoc
lawndoc reviewed Oct 18, 2025
View reviewed changes
lawndoc
lawndoc reviewed Oct 18, 2025
View reviewed changes
lawndoc
lawndoc reviewed Oct 18, 2025
View reviewed changes
lawndoc
lawndoc reviewed Oct 18, 2025
View reviewed changes
lawndoc
lawndoc reviewed Oct 18, 2025
View reviewed changes
@AlexMcDermott
Copy link
Author

Hi @lawndoc thanks for the review. I've implemented your feedback and tested it out with a Postgres container started in another compose stack. Let me know what you think : )

@AlexMcDermott AlexMcDermott requested a review from lawndoc November 1, 2025 00:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lawndoc lawndoc Awaiting requested review from lawndoc

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@AlexMcDermott @lawndoc