Restore doesn't function with Docker/Kamal

[oandalib]

This could be a duplicate of #33 and/or #34.

When using with Docker/Kamal we have a volume that is mounted from the host machine.

When I click on restore from the UI or run restore via the command line, the three files .sqlite3, .sqlite3-wal, .sqlite3-shm are downloaded and are correct, however the naming differs from the files currently in use (as they should, at least temporarily).

One of the steps of litestream is "renaming database from temporary location", and this is where it appears we get stuck. My guess is a permission issue.

I have found a workaround for now where I can manually rename/overwrite the three files and then restart my Docker container.

[fractaledmind]

Is any sort of warning or error thrown, or things just fail silently?

[oandalib]

For either command line or UI restore, the last log message is level=INFO msg="renaming database from temporary location" .... It seems to get stuck here without any further warnings or errors.

For the UI restore we get a successfully restored alert, but that's not accurate.

[oandalib]

I spent some additional time looking into this.

My previous comments may not be entirely accurate. I decided to do a clean deployment via Docker/Kamal and retry the above. Both UI restore and command line restore seem to be working okay, at least in terms of getting the restoration files.

For command line restore bin/rails litestream:restore -- --database=storage/production.sqlite3, the files get setup as production.sqlite3/production.sqlite3.tmp-shm/production.sqlite3.tmp-wal. Although I still need to do a Docker container restart, I'm not surprised. On restart, production.sqlite3-shm and production.sqlite3-wal are created, and everything is as expected and data is restored.

For UI restore, the files get setup as per here with the filenames having a datetime in it (e.g. production-20241025222624.sqlite3). I manually renamed the files to match the command line restore and restarted the Docker container, and everything is as expected and data is restored.

I'm very new to litestream and this gem specifically, so maybe this is expected behavior and I do understand the value of adding the datetime for knowing exactly what version it is. However, why not just save the files similar to the command line restore, without the datetime? Is the expected behavior for users to click restore on the UI and have the database restored (with the caveat of having to restart Docker if they are using Docker) or are additional steps required in which case we should update the README?

[AxelTheGerman]

Could it be that the restore might also not always work as expected because all commands are run async by default #17 and there is no way to run sync via CLI?

My docker script attempts to restore for fresh deployments but I suspect that the following rails db:prepare actually does create the database before the litestream process does.

Happy to log a new issue for this, but thought this might be related.

For now I'm attempting to write my own rake task that uses the Ruby API of the gem which does allow specifying async: false

[fractaledmind]

Interesting point. Please do report back if running the command synchronously fixes the issue in your testing.