Updated metadata, covers, and deleted books added to Kobo Sync

[AsherMaximum]

For #3298, #2685, #2816, #2509, #3133
Adds functionality to Kobo Sync for removing books from a shelf, updating metadata, and updating covers.

Sends a BookEntitlement deletion for any books that have already been synced to the kobo but are now no longer on a sync shelf (when sync only kobo marked shelves is turned on)

Also sends a ChangedProductMetadata for any books that has a modification date after the last sync, so the Kobo will get the new metadata

Added a version to the CoverImageId based on the cover file modified date. Also added an automatic thumbnail refresh to Kobo sync initialization.

[basvanberckel]

[2025-06-04 18:30:02,701] ERROR {app.py:875} Exception on /kobo/TOKEN/v1/library/sync [GET]
Traceback (most recent call last):
  File "/lsiopy/lib/python3.12/site-packages/sqlalchemy/engine/result.py", line 199, in _key_not_found
    self._key_fallback(key, None)
  File "/lsiopy/lib/python3.12/site-packages/sqlalchemy/engine/result.py", line 138, in _key_fallback
    raise KeyError(key) from err
KeyError: 'deleted'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/lsiopy/lib/python3.12/site-packages/flask/app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.12/site-packages/flask/app.py", line 919, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/calibre-web/cps/kobo_auth.py", line 173, in inner
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/app/calibre-web/cps/kobo.py", line 281, in HandleSyncRequest
    elif book.deleted:
         ^^^^^^^^^^^^
  File "lib/sqlalchemy/cyextension/resultproxy.pyx", line 66, in sqlalchemy.cyextension.resultproxy.BaseRow.__getattr__
  File "lib/sqlalchemy/cyextension/resultproxy.pyx", line 63, in sqlalchemy.cyextension.resultproxy.BaseRow._get_by_key_impl
  File "/lsiopy/lib/python3.12/site-packages/sqlalchemy/engine/result.py", line 201, in _key_not_found
    raise AttributeError(ke.args[0]) from ke
AttributeError: deleted

Getting this when running this PR on an existing database. Is there a database migration step that needs to be done?

Edit to add: I ran calibredb restore_database, which did resolve the error above but results in errors when the device tries to download the books (because of custom column tables now missing), so that in itself doesn't appear to be enough.

[AsherMaximum]

I need to look at the logic for when only kobo shelves is not turned on; I think that's what's happening here. It's checking for books that have been deleted from a shelf, but without onlykobo sync shelves turned on, that field doesn't exist.

There's no database changes for this, so not sure why you're getting an error related to that.

[faethon]

These are very useful additions! Hope this get merged in the main branch.

[AsherMaximum]

Getting this when running this PR on an existing database. Is there a database migration step that needs to be done?

The latest commit should fix this error - it wasn't an added field to the database. The query for when onlyKobo shelves is on adds a field to the results to indicate if the book is one that has been removed from a shelf, so that it can be deleted from the device.
When not using onlyKobo shelves, this field did not exist, but the code was still trying to reference it. The check just needed to be updated to only handle deleted books when onlyKobo shelves is on.

[AsherMaximum]

oops, that was not the right button to submit a comment, lol

[basvanberckel]

Looks to be working now, it grabs the updated cover on sync. Actually getting the cover to show on the device is a little spotty (it shows the blank placeholder a lot of the time, but randomly the right cover just pops up sometimes), but that's probably not on calibre-web's side

[peter9teufel]

i tried this pull request on my instance. My goal was, that when I disable the Kobo sync a shelf that has been synced to my Kobo before, I want the books and collections on my Kobo Reader to be deleted. But it didn't work out, think I misunderstood the purpose of this PR? :smiling

Best regards,
Peter

[AsherMaximum]

Deleting the books is the intention - not the collection though, haven't figured out how to do that yet.
I've tested deleting books by disabling kobo sync on it, and it does work for me, although the collections remains, just empty. It still says it has books in it, but when I open it, there is none.

If the books are in multiple collections though, and you only disable kobo sync on one of them, the books would not get deleted.

[AsherMaximum]

Looks to be working now, it grabs the updated cover on sync. Actually getting the cover to show on the device is a little spotty (it shows the blank placeholder a lot of the time, but randomly the right cover just pops up sometimes), but that's probably not on calibre-web's side

Yeah, I've had that issue too. It seems the kobo does not always cache the cover, and I haven't yet figured out why.